Information and entropy flow in the Kalman filter (Part 2)
Abstract:
This is the second part of last week's lecture. We investigate the information theoretic properties of Kalman–Bucy filters in continuous time, developing notions of information supply, storage and dissipation. Introducing a concept of energy, we develop a physical analogy in which the unobserved signal describes a statistical mechanical system interacting with a heat bath. The abstract ‘universe’ comprising the signal and the heat bath obeys a non-increase law of entropy; however, with the introduction of partial observations, this law can be violated. The Kalman–Bucy filter behaves like a Maxwellian demon in this analogy, returning signal energy to the heat bath without causing entropy increase. This is made possible by the steady supply of new information. In a second analogy the signal and filter interact, setting up a stationary non-equilibrium state, in which energy flows between the heat bath, the signal and the filter without causing any overall entropy increase. We introduce a rate of interactive entropy flow that isolates the statistical mechanics of this flow from marginal effects. Both analogies provide quantitative examples of Landauer’s Principle.
Based on http://www.mit.edu/~mitter/publications/104_info_flow_kalman_JSP.pdf
Link to Duncan's paper http://www.dtic.mil/dtic/tr/fulltext/u2/686819.pdf
Biography:
Sanjoy K. Mitter received his Ph.D. degree from the Imperial College of Science and Technology, University of London, in 1965. He had previously worked as a research engineer at Brown Boveri & Co. Ltd., Switzerland (now ASEA Brown Boveri) and Battelle Institute in Geneva, Switzerland. He taught at Case Western Reserve University from 1965 to 1969 and joined MIT in 1969, first as a Visiting Professor and then in 1970 as Associate Professor in the Department of Electrical Engineering and Computer Science and in 1973 as a Professor of Electrical Engineering. He was the Director of the MIT Laboratory for Information and Decision Systems from 1981 to 1999 and Director of the Center for Intelligent Control Systems, an inter-university (Brown-Harvard-MIT) center for research on the foundations of intelligent systems from 1986-2000. He has held visiting positions at the Tata Institute of Fundamental Research, Bombay, India; Scuola Normale Superiore, Pisa, Italy; Imperial College of Science and Technology; Institut National de Recherche en Informatique et en Automatique, France; University of Groningen, the Netherlands; ETH, Zürich, Switzerland and several universities in the United States including the University of California, Berkeley where he was the McKay Professor in March 2000, and held the Russell-Severance-Springer Chair in Fall 2003.
Professor Mitter's research has spanned the broad areas of Systems, Communication and Control. Although his primary contributions have been on the theoretical foundations of the field, he has also contributed to significant engineering applications, notably in the control of interconnected power systems, character recognition, and automatic recognition and classification of electrocardiograms. His current research interests are theory of stochastic dynamical systems, nonlinear filtering, stochastic and adaptive control; mathematical physics and its relationship to system theory; image analysis and computer vision; and structure, function and organization of complex systems.
Professor Mitter has served on several advisory committees and editorial boards for IEEE, SIAM, AMS, NSF and ARO. He is currently Associate Editor of the Journal of Applied Mathematics and Optimization; Random and Computational Dynamics; Sankhya and the Ulam Quarterly and Editor-at-Large for Communications in Information and Systems. He is a fellow of the IEEE and was the recipient of the 2000 IEEE Control Systems Award. In 1988 he was elected to the National Academy of Engineering. He was elected a Foreign Member of Istituto Veneto di Scienze, Lettere ed Arti in 2003.