ACTIVATIONS AND SIGNALS ◆ NEURONS AS FUNCTIONS SIGNAL MONOTONICITY ■ BIOLOGICAL ACTIVATIONS AND SIGNALS NEURON FIELDS NEURONAL DYNAMICAL SYSTEMS COMMON SIGNAL FUNCTION PULSE-CODED SIGNAL FUNCTION
ACTIVATIONS AND SIGNALS ◼ NEURONS AS FUNCTIONS ◼ SIGNAL MONOTONICITY ◼ BIOLOGICAL ACTIVATIONS AND SIGNALS ◼ NEURON FIELDS ◼ NEURONAL DYNAMICAL SYSTEMS ◼ COMMON SIGNAL FUNCTION ◼ PULSE-CODED SIGNAL FUNCTION
NEURONS AS FUNCTIONS Neurons behave as functions. Neurons transduce an unbounded input activation x(t) at time t into a bounded output signal S(x(t))
NEURONS AS FUNCTIONS Neurons behave as functions. Neurons transduce an unbounded input activation x(t) at time t into a bounded output signal S(x(t))
NEURONS AS FUNCTIONS S(x) X -00 0 + +00 Fig.1 s(x)is a bounded monotone-nondecreasing function of x If c+,we get threshold signal function (dash line) Which is piecewise differentiable
NEURONS AS FUNCTIONS S(x) x -∞ - 0 + +∞ Fig.1 s(x) is a bounded monotone-nondecreasing function of x If c→+∞，we get threshold signal function (dash line), Which is piecewise differentiable
NEURONS AS FUNCTIONS The transduction description:a sigmoidal or S-shaped curve the logistic signal function: S(x)= l+e-cx S'= dS =cS(1-S)>0 (c>0) dx The logistic signal function is sigmoidal and strictly increases for positive scaling constant c>0
NEURONS AS FUNCTIONS The transduction description: a sigmoidal or S-shaped curve the logistic signal function: cx e S x − + = 1 1 ( ) ' = = cS(1− S) 0 (c 0) dx dS S The logistic signal function is sigmoidal and strictly increases for positive scaling constant c >0
NEURONS AS FUNCTIONS S(x) X -00 0 + +00 Fig.1 s(x)is a bounded monotone-nondecreasing function of x If c+,we get threshold signal function (dash line) Which is piecewise differentiable
NEURONS AS FUNCTIONS S(x) x -∞ - 0 + +∞ Fig.1 s(x) is a bounded monotone-nondecreasing function of x If c→+∞，we get threshold signal function (dash line), Which is piecewise differentiable
NEURONS AS FUNCTIONS Swould transduce the four-neuron vector of activations (-6 350 49 -689)to the four- dimensional bit vector of signal(0 1 1 0) Zero activations to unity,zero,or the previous signal
NEURONS AS FUNCTIONS S would transduce the four-neuron vector of activations (-6 350 49 –689) to the fourdimensional bit vector of signal (0 1 1 0) Zero activations to unity,zero,or the previous signal
SIGNAL MONOTONICITY In general,signal functions are monotone nondecreasing S'>=0. S(x) X -00 0 +00 This means signal functions have an upper bound or saturation value
SIGNAL MONOTONICITY In general, signal functions are monotone nondecreasing S’>=0. This means signal functions have an upper bound or saturation value. S(x) x -∞ - 0 + +∞
SIGNAL MONOTONICITY An important exception:bell-shaped signal function or Gaussian signal functions S(x)=e-cx2 c>0 S=-2cxe Soc-x The sign of the signal-activation derivation s'is opposite the sign of the activation x.We shall assume signal functions are monotone nondecreasing unless stated otherwise
SIGNAL MONOTONICITY An important exception: bell-shaped signal function or Gaussian signal functions 0 2 = − S x e c cx ( ) S cxe S x cx = − − − ' 2 , ' 2 The sign of the signal-activation derivation s’ is opposite the sign of the activation x. We shall assume signal functions are monotone nondecreasing unless stated otherwise
SIGNAL MONOTONICITY Generalized Gaussian signal function define potential or radial basis function S,(x): SU-ml2a2,门 input activation vector::x=(xi,…,xn)∈R" variance: mean vector: 4,=(4,,4) we shall consider only scalar-input signal functions:S,(x)
SIGNAL MONOTONICITY Generalized Gaussian signal function define potential or radial basis function : ( ) ] 2 1 ( ) exp[ 2 = − 2 − n j i j j i i S x x n x = (x1 , , xn )R ( , , ) i n i i = 1 input activation vector: variance: mean vector: 2 i S (x) i we shall consider only scalar-input signal functions: ( ) i i S x
SIGNAL MONOTONICITY A property of signal monotonicity:semi-linearity Comparation: a.Linear signal functions: computation and analysis is comparatively easy; do not suppress noise. b.Nonlinear signal functions: increases a network's computational richness and facilitates noise suppression; risks computational and analytical intractability;
SIGNAL MONOTONICITY A property of signal monotonicity: semi-linearity Comparation: a. Linear signal functions: computation and analysis is comparatively easy; do not suppress noise. b. Nonlinear signal functions: increases a network’s computational richness and facilitates noise suppression; risks computational and analytical intractability;