Talk:Softmax function

Origin

To my knowledge, the softmax function was first proposed in

J. S. Bridle, “Probabilistic Interpretation of Feedforward Classification Network Outputs, with Relationships to Statistical Pattern Recognition,” in Neurocomputing, F. F. Soulié and J. Hérault, Eds. Springer Berlin Heidelberg, 1990, pp. 227–236.

and

J. S. Bridle, “Training Stochastic Model Recognition Algorithms as Networks can Lead to Maximum Mutual Information Estimation of Parameters,” in Advances in Neural Information Processing Systems 2, D. S. Touretzky, Ed. Morgan-Kaufmann, 1990, pp. 211–217.

Would it be appropriate to mention these publications, or at least their author, in the article? --131.152.137.39 (talk) 13:40, 13 November 2015 (UTC)

Family of functions

From reading Serre 2005, it sounds that there are multiple definitions of softmax. Is this the case? What other representations are there? It looks like Riesenhuber and Poggio, 1999b and Yu et al., 2002 as referenced in Serre 2005 might give clues. JonathanWilliford (talk) 23:53, 11 August 2009 (UTC)

Content

Is this acceptable content? 95% of the article is a direct copy of the content from [1]. Jludwig (talk) 06:23, 10 May 2008 (UTC)

I agree with this concern. I am deleting most of the content of the article, per the concern of copyright violation. 128.197.81.32 (talk) 22:00, 30 July 2008 (UTC)

Furthermore, most of the terms are not defined. You can't just copy a bunch of equations into a document and not define the terms, it's terrible. People w/a background in stats can guess what most of the variables mean, but regardless, i hope whoever wrote this will return and define every variable that appears in an expression. Chafe66 (talk) 22:40, 29 October 2015 (UTC)

Derivation

Whats about the derivation of the softmax function? —Preceding unsigned comment added by 78.34.250.44 (talk) 21:28, 13 March 2009 (UTC)

John D Cook's definition is different from all of these

I followed the external link to the description of softmax as a substitute for maximum by John D. Cook. There, the softmax is described as

$\log(\sum _{j=1}^{n}\exp(q_{j}))$$\log(\sum _{j=1}^{n}\exp(q_{j}))$

not

${\frac {\exp(q_{i})}{\sum _{j=1}^{n}\exp(q_{j})}}$${\frac {\exp(q_{i})}{\sum _{j=1}^{n}\exp(q_{j})}}$

as in wkp. His version makes more sense to me. Can anyone corroborate me on this? I think the article needs fixing. But since there seem to be multiple definitions, it's hard to be clear.--mcld (talk) 15:31, 2 January 2014 (UTC)

I saw this on Cook's blog and I was highly surprised. Apparently this is a completely different function that is also called the softmax; I've never seen it in use. QVVERTYVS (hm?) 15:28, 5 February 2014 (UTC)

I checked Cook's blog post again, and found that he doesn't even call his function softmax; he calls it "soft maximum". Removed the link as it's quite unrelated. QVVERTYVS (hm?) 17:36, 5 February 2014 (UTC)

Cook's post is very informative on smooth maximum. There seems to be no natural setting for putting smooth maximum sub heading in soft max. Moving to a new page. — Preceding unsigned comment added by Yodamaster1 (talk • contribs) 16:50, 20 February 2015 (UTC)

Possible?

Could it ever be possible that the explanation of how the function works be any more incomprehensible?

Apparently, there's an extremely well developed culture in wikipedia of: everyone is expected to know a bunch of inscrutable variable-name conventions. Either that or writers really are convinced those are solidly established conventions of the likes of "+", "-", "x", etc. . I mean, not even "n" (usually used to mean "number of elements") is conventional enough in many cases (especially considering how often it is used to other meanings too).

I'm really fed up with this, and this article is among the most poor examples of this that I've found so far. — Preceding unsigned comment added by 151.227.23.87 (talk) 17:27, 1 August 2015 (UTC)