Research
The Learning Systems group at Caltech works on the theory, implementation, and application of automated learning, pattern recognition, and neural networks. We are an interdisciplinary group with students coming from Electrical Engineering, Computer Science, Mathematics, and Physics. We work on a variety of projects analyzing and synthesizing systems that can be trained to perform their task.
Learning is a topic that can be approached from different angles. For instance, one project deals with the rigorous mathematical analysis of overlearning; when a system memorizes too much detail about a limited experience. Another project develops techniques for selecting highlevel features for pattern recognition. A third project applies learning to forecasting in the financial markets.
We have an ongoing research effort in learning from hints which generalizes the basic learningfromexamples paradigm. Learning from examples assumes that the function we are trying to learn is represented to us by a training set of input/output examples, but is otherwise unknown. This poses a dilemma: We would like to use a learning model that is sophisticated enough to have a chance of implementing the unknown function, yet we want the model to be simple enough that a limited set of examples will suffice to tune it properly.
Hints come in as a learning aid to improve the situation. We usually have, in addition to the set of examples, some prior knowledge of certain facts about the system. We wish to use this side information to our advantage. For instance, we may know that is scaleinvariant and monotonic, and we want to put this information together with the training set of examples to come up with a good implementation. We want this process to be automated, so that the system is genuinely learning from hints.
Our group has pioneered the use of hints in learning, and we continue to work on all aspects of the technique. The goal is to automate the use of hints to a degree where we can effectively utilize a large number of different hints that may be available in a practical situation. We have also developed a theoretical analysis of learning from hints. The analysis is based on the VapnikChervonenkis dimension provides an estimate for the number of examples the system will need to learn.
The Learning Systems group is committed to the understanding of the fundamental components of automated learning, and to the development of reallife systems that utilize learning to achieve stateofthe art performance.
