Review 1.Neuronal Dynamical Systems We describe the neuronal dynamical systems by first- order differential or difference equations that govern the time evolution of the neuronal activations or membrane potentials
1.Neuronal Dynamical Systems We describe the neuronal dynamical systems by firstorder differential or difference equations that govern the time evolution of the neuronal activations or membrane potentials. ( , , ) ( , , ) X Y X Y y h F F x g F F = = Review
Review 4.Additive activation models 衣，=-4x,+∑S,(y)ni+1, 立，=-A,y,+∑S,(x)m,+J Hopfield circuit: i=1 1.Additive autoassociative model; 2.Strictly increasing bounded signal function (S>0); 3.Synaptic connection matrix is symmetric(M=M). Cx=R+s,m,+
4.Additive activation models = = = − + + = − + + n i j j j i i ij j p j i i i j j ji i y A y S x m J x A x S y n I 1 1 ( ) ( ) Hopfield circuit: 1. Additive autoassociative model; 2. Strictly increasing bounded signal function ; 3. Synaptic connection matrix is symmetric . (S 0) ( ) T M = M = − + + j j j ji i i i i i S x m I R x C x ( ) Review
Review 5.Additive bivalent models x+1=∑S,Oy5)m:+1 y=∑S,(x)m,+1 Lyapunov Functions Cannot find a lyapunov function,nothing follows; Can find a lyapunov function,stability holds
5.Additive bivalent models = + = + + + n i ij j k i i k j p j ji i k j j k i y S x m I x S y m I ( ) ( ) 1 1 Lyapunov Functions Cannot find a lyapunov function,nothing follows; Can find a lyapunov function,stability holds. Review
Review A dynamics system is stable,ifL≤O asymptotically stable,if <O Monotonicity of a lyapunov function is a sufficient not necessary condition for stability and asymptotic stability
A dynamics system is stable , if ; asymptotically stable, if . L 0 L 0 Monotonicity of a lyapunov function is a sufficient not necessary condition for stability and asymptotic stability. Review
Review Bivalent BAM theorem. Every matrix is bidirectionally stable for synchronous or asynchronous state changes. Synchronous:update an entire field of neurons at a time. ● Simple asynchronous:only one neuron makes a state- change decision. Subset asynchronous:one subset of neurons per field makes state-change decisions at a time
Bivalent BAM theorem. Every matrix is bidirectionally stable for synchronous or asynchronous state changes. • Synchronous:update an entire field of neurons at a time. • Simple asynchronous:only one neuron makes a statechange decision. • Subset asynchronous:one subset of neurons per field makes state-change decisions at a time. Review
Chapter 3.Neural Dynamics II:Activation Models The most popular method for constructing M:the bipolar Hebbian or outer-product learning method binary vector associations:(4,,B i=1,2,…m bipolar vector associations:(XY 4=K,+ X,=2A-1 2002.10.8
2002.10.8 Chapter 3. Neural Dynamics II:Activation Models The most popular method for constructing M:the bipolar Hebbian or outer-product learning method binary vector associations: bipolar vector associations: ( , ) Ai Bi ( , ) Xi Yi i = 1,2, m [ 1] 2 1 Ai = Xi + Xi = 2Ai −1
Chapter 3.Neural Dynamics II:Activation Models The binary outer-product law: M=∑AB The bipolar outer-product law: M=∑XY k The Boolean outer-product law: M=田ABE m,=max a'b1,…,anbh) 2002.10.8
2002.10.8 Chapter 3. Neural Dynamics II:Activation Models The bipolar outer-product law: = m k k T M X k Y The binary outer-product law: = m k k T M Ak B The Boolean outer-product law: k T k m k M = A B max( , , ) 1 1 j m i m i j mij = a b a b
Chapter 3.Neural Dynamics II:Activation Models The weighted outer-product law: m M=∑wXiY Where∑w&=1 holds. In matrix notation: M=XWY Where XT=[X.Xm] Yr=[YI…lYm] W=Diagonal[w1,…,wm] 2002.10.8
2002.10.8 Chapter 3. Neural Dynamics II:Activation Models The weighted outer-product law: In matrix notation: Where holds. = m k k T M wk X k Y = m k wk 1 M X WY T = Where [ | | ] 1 T m T T X = X X [ , , ] W = Diagonal w1 wm [ | | ] 1 T m T T Y = Y Y
Chapter 3.Neural Dynamics II:Activation Models X3.6.1 Optimal Linear Associative Memory Matrices Optimal linear associative memory matrices: M=XY The pseudo-inverse matrix of: XYY-X X'XX=X X'Y-(XX XX=(XX') 2002.10.8
2002.10.8 Chapter 3. Neural Dynamics II:Activation Models Optimal linear associative memory matrices: M X Y * = XX X = X * * * * X XX = X T X X (X X) * * = T XX (XX ) * * = The pseudo-inverse matrix of X : * X ※3.6.1 Optimal Linear Associative Memory Matrices
Chapter 3.Neural Dynamics II:Activation Models X3.6.1 Optimal Linear Associative Memory Matrices Optimal linear associative memory matrices: The pseudo-inverse matrix of: If x is a nonzero scalar:x=1/x If x is a nonzero vector: X If x is a zero scalar or zero vector x*=0 For a rectangular matrix X,if ()exists: X"=X(XX) 2002.10.8
2002.10.8 Chapter 3. Neural Dynamics II:Activation Models ※3.6.1 Optimal Linear Associative Memory Matrices Optimal linear associative memory matrices: The pseudo-inverse matrix of X : * X If x is a nonzero scalar: x 1/ x * = If x is a zero scalar or zero vector : For a rectangular matrix , if exists: 0 * x = If x is a nonzero vector: T T xx x x = * 1 ( ) T − XX * 1 ( ) − = T T X X XX X