Maxout ReLU is a special cases of Maxout Learnable activation function [lan J.Goodfellow,ICML'13] 5 neuron Input Max Max 2 +-2 +一4 Max Max 4 +3 You can have more than 2 elements in a group. Maxout • Learnable activation function [Ian J. Goodfellow, ICML’13] Max 1 x 2 x Input Max + 5 + 7 + −1 + 1 7 1 Max Max + 1 + 2 + 4 + 3 2 4 ReLU is a special cases of Maxout You can have more than 2 elements in a group. neuron