Neural Network Architectures and Algorithms for Associative Memory, Syntax Analysis, Pattern Classification

 

Personnel

 

Dr. Vasant Honavar, Professor of Computer Science and of Bioinformatics and Computational Biology, Principal Investigator

 

Summary

Induction of pattern classifiers from data is an important area of research in machine learning which finds applications in diverse areas including automated diagnosis, bioinformatics, design of customizable information assistants, among others. Artificial neural networks, because of their potential for massive parallelism and fault and noise tolerance, offer an attractive approach to the design of trainable pattern classifiers. Constructive learning algorithms, which construct arbitrarily complex decision boundaries needed for pattern classification (and in some ways, foreshadowed the recent development of support vector machines) were motivated by: the need to overcome the limitations of learning through parameter modification within an a priori fixed network topology; and to avoid the guesswork involved in deciding suitable network architectures for different pattern classification problems by dynamically growing the network to match the complexity of the underlying pattern classification task. This research led to:

 

• Generalization (with convergence guarantees) of a large family of such algorithms designed for 2-class binary pattern classification problems to handle classification problems involving real-valued patterns and an arbitrary number of classes (Parekh, Yang, and Honavar, 2000)
• Development of a simple, inter-pattern distance based provably convergent, polynomial time constructive neural network algorithm which compares very favorably with computationally far more expensive algorithms in terms of generalization accuracy (Yang, Parekh, and Honavar, 1999).

• Development of algorithms for construction of robust, noise-tolerant neural memories for pattern storage and associative, content-based retrieval (Chen and Honavar, 1995; Chen and Honavar, 2000)
• Development of algorithms for construction of highly parallel neural architectures for syntax analysis (parsing of regular, context-free, and context-sensitive languages) (Chen and Honavar, 1999).
• Development of a biologically inspired neural architecture and an extended Kalman filter algorithm for place learning and localization in a-priori unknown environments which successfully accounts for a large body of behavioral and neurobiological data from animal experiments and offers several testable predictions (Balakrishnan, Bousquet, and Honavar, 2000).
• Development of evolutionary algorithms for feature subset selection for pattern classification design of sensor systems and actuators for intelligent agents (Yang and Honavar, 1998; Balakrishnan and Honavar, 1996; Balakrishnan and Honavar, 2001).

 

Funding

This work was supported by a grant from the National Science Foundation (NSF RIA 9409580).

Representative Publications

1. Balakrishnan, K. & Honavar, V. (2001). Experiments in Evolutionary Robotics. In: Advances in Evolutionary Synthesis of Intelligent Agents. Patel, M., Honavar, V. and Balakrishnan, K. (Ed). Cambridge, MA: MIT Press.

2. Parekh, R., Yang, J., and Honavar, V. (2000). Constructive Neural Network Learning Algorithms for Multi-Category Pattern Classification. IEEE Transactions on Neural Networks. Vol. 11. No. 2. pp. 436-451.

3. Balakrishnan, K., Bousquet, O. and Honavar, V. (2000). Spatial Learning and Localization in Animals: A Computational Model and Its Implications for Mobile Robots, Adaptive Behavior. Vol. 7. no. 2. pp. 173-216.

4. Chen, C-H. & Honavar, V. (2000). A Neural Architecture for Information Retrieval and Query Processing. Invited chapter. In: Handbook of Natural Language Processing. Dale, Moisl, and Somers (Ed.) New York: Marcel Dekker.

5. Chen, C-H. & Honavar, V. (1999). A Neural Architecture for Syntax Analysis. IEEE Transactions on Neural Networks. Vol. 10 pp. 94-114.

6. Honavar, V., Parekh, R. and Yang, J. (1999). Machine Learning. Invited article. In: Encyclopedia of Electrical and Electronics Engineering, Webster, J. (Ed.), New York: Wiley.

7. Yang, J. and Honavar, V. (1998). Feature Subset Selection Using a Genetic Algorithm. IEEE Intelligent Systems (Special Issue on Feature Transformation and Subset Selection). vol. 13. pp. 44-49.

8. Balakrishnan, K. and Honavar, V. (1996). On Sensor Evolution in Robotics. Proceedings of the First International Conference on Genetic Programming, Stanford University, CA. pp. 455-460.