Softmax

From neurov.is/on

Jump to: navigation, search

Softmax is a neural activation function. It is an approximate maximum operation [1]. It represented as:

y=g \left( \frac{\sum_{j=1}^n x_j^{q+1}} {k+\left( \sum_{j=1}^n x_j^q \right)} \right) \text{,}

in [2] where g is a sigmoid transfer function. It is also represented as

p_i = \frac{\exp(q_i)}{\Sigma_{j=1}^n\exp(q_j)} \text{,}

where p is the value of an output node, q is the net input to an output node, and n is the number of output nodes.

The softmax operation is used to simulate an invariance operation of complex cells in [2].


[edit] References

  1. ↑ Cadieu C, Kouh M, Pasupathy A, Conner CE, Riesenhuber M, and Poggio T. A Model of V4 Shape Selectivity and Invariance. J Neurophysiol 98: 1733-1750, 2007.
  2. ↑ 2.0 2.1 Serre T, Kouh M, Cadieu C, Knoblich U, Kreiman G, and Poggio T. A theory of object recognition: computations and circuits in the feedforward path of the ventral stream in primate visual cortex. CBCL Paper 259/AI Memo 2005-036. Cambridge, MA: MIT, 2005.
Retrieved from "http://neurov.is/on/Softmax"
Personal tools