Science and information theory

By Leon Brillouin

Subjects: Information theory

Description: A new scientific theory has been born during the last few years, the theory of information. It immediately attracted a great deal of interest and has expanded very rapidly. This new theory was initially the result of a very practical and utilitarian discussion of certain basic problems: How is it possible to define the quantity of information contained in a message or telegram to be transmitted? How does one measure the amount of information communicated by a system of telegraphic signals? How does one compare these two quantities and discuss the efficiency of coding devices ? All of these problems, and many similar ones, are of concern to the telecommunication engineer and can now be discussed quantitatively. From these discussions there emerged a new theory of both mathematical and practical character. This theory is based on probability considerations. Once stated in a precise way, it can be used for many fundamental scientific discussions. It enables one to solve the problem of Maxwell's demon and to show a very direct connection between information and entropy. The thermodynamical entropy measures the lack of information about a certain physical system. Whenever an experiment is performed in the laboratory, it is paid for by an increase of entropy, and a generalized Carnot Principle states that the price paid in increase of entropy must always be larger than the amount of information gained. Information corresponds to negative entropy, a quantity for which the author coined the word negentropy. The generalized Carnot Principle may also be called the negentropy principle of information. This principle imposes a new limitation on physical experiments and is independent of the well-known uncertainty relations of quantum mechanics. The present book is based on lectures delivered before engineers of the International Business Machine Corporation at Poughkeepsie and Endicott, N. Y., and later on at different universities, especially at the University of California at Berkeley. [from author's Preface to the 1st edition]