This paper addresses the relationship between the number of hidden layer nodes in a neural network, the complexity of a multi-class discrimination problem, and the number of samples needed for effective learning. Bounds are given for the latter. We show that Ω(min(d,n).M) boundary samples are required for successful classification of M clusters of samples using a 2 hidden layer neural network with d-dimensional inputs and n nodes in the first hidden layer.
Mehrotra, Kishan G.; Mohan, Chilukuri K.; and Ranka, Sanjay, "Bounds on the Number of Samples Needed for Neural Learning" (1980). Electrical Engineering and Computer Science Technical Reports. Paper 94.