Learning From Constraints

When water moves, mollusks open shell and when something touches membrane, then shell closes. Animals are able to coordinate perception with action, which is especially clear during the hunt. In the children cognitive development, the transition from sensorimotor to more abstract representations of reality follow a stage-based principle. Is that the exclusive outcome of biology or is dictated by general principles of optimization, so that stages are nothing else than the natural solution to break the complexity of learning. Most machine learning algorithms relies on simple protocols (supervised, reinforcement, unsupervised, semi-supervised schemes), in which a considerable number of relevant interactions are missed. How about the crucial role of the teacher? The clear identification of hierarchical labels in learning environments can be thought of as a sign of intermediate goals for the agent, and the formalization of some constraints might directly give insights on the birth of deep architectures.

Further information can be found in https://sites.google.com/site/semanticbasedregularization/.