Algorithms & Architectures, Constructive & Pruning Algorithms
We present a problem decomposition approach to reduce neural net training times. The basic idea is to train neural nets in parallel on marginal distributions obtained from the original distribution (via projection), and then reconstruct the original table from the marginals (via a procedure similar to the join operator in database theory). A function is said to be reconstructible, if it may be recovered without error from its projections. Most distributions are non-reconstructible. The main result of this paper is the Reconstruction theorem, which enables non-reconstructible functions to be expressed in terms of reconstructible ones, and thus facilitates the application of decomposition methods.
Menon, Anil Ravindran; Mehrotra, Kishan; Mohan, Chilukuri K.; and Ranka, Sanjay, "Putting Humpty-Dumpty together again: Reconstructing functions from their projections." (1993). Electrical Engineering and Computer Science - Technical Reports. 159.