Information is surprise. To the extent that something is predictable, it doesn't contain information. For example, let's say you're reading a book and you see the sequence "the sky is blu". There is zero surprise and therefore no information conveyed when the next letter you see is 'e'. If you already know the answer, there's no information conveyed.
In information theory, entropy is a measure of unpredictability or information content in a block of data. If the data is maximally compressed, then every single bit of data is unpredictable and the information entropy is maximized.
High entropy, maximally compressed data is indistinguishable from gibberish (have you ever looked at compressed text, for example?) - unless you have the decompression key.
To send information somewhere, you need a channel to communicate. The channel can be a wire or air or space or something else and uses some fraction of the available spectrum and bandwidth of the particular medium to send the information.
To maximize the amount of information you can send over a channel, you want to maximize the entropy of the information, and minimize the entropy of the channel. In other words, you want to make the information look as much like pure noise as possible (compress the hell out of it and/or otherwise encode it) and you want to make the channel itself as quiet and noiseless as possible.
To summarize so far: information is surprise; surprise is unpredictable; maximum unpredictability is indistinguishable from noise without the key or code; and reducing the noise of a communication channel enables a higher rate of information transmission. Except for a few minor details, that's everything there is to know about Information Theory. :-) With some innovation regarding those minor details, coupled with an excellent business model, and you could be the next Qualcomm!
In the book Knowledge and Power, George Gilder proposes that Information Theory as described above is critical for understanding economic activity and economic systems. Each of us has extensive localized Knowledge about consumption and production. On the consumption side, we know what we'd like to consume and what we're willing to trade for those goods and services. On the production side, we know what we're capable of producing and many of us have expertise, sometimes extensive, about various aspects of what's possible to produce given various inputs.
For that Knowledge to be useful, the information related to it must be transmitted to others. For the vast majority of people, the vast majority of that information is indistinguishable from noise. However, some have the "code" to understand the information being transmitted. As a simple example, the price of rubber looks like a random squiggle to almost everybody. However, to a tire manufacturer, there is critically important information embodied in those prices, especially when coupled with expected automotive demand and other related information.
The information channel that supports economic activity is critically important. First, there needs to be one. The institutions of trade, property, law, etc. need to exist and they provide the infrastructure for the channel. According to Gilder, these institutions should be maintained by government.
With the Information Theory construct, it's clearly important that the economy's communication channel has as little noise as possible. This has two implications: the institutions that form the communication channel should be as predictable as possible since unpredictability means surprise which means noise; and the institutions themselves should be as quiet and non-intrusive as possible, otherwise the institutions use up precious channel bandwidth and compete with and overpower transmissions from economic activities, impeding the flow of that information and stifling the activities that would have resulted had that information gotten through.
In Gilder's view, predictability means that the institutions have structures and rules that are unchanging or slowly changing, easily understandable, and easy to follow. This implies "rule of law" as opposed to "rule of man" (or "rule by fiat"). According to Gilder, keeping the institutions quiet means minimizing the information flow from the institutions through the channel. Minimizing regulations, minimizing taxes and keeping the rules associated with the tax regime as simple as possible helps keep down channel noise and enables increased economic communication and activity.
Gilder notes that one of the problems with government institutions generating noise is that government has a lot of Power. Its communications tend to be loud, high power communications, which uses up a tremendous amount of bandwidth, while Knowledge based economic communications tend to be quiet, low-power, numerous, and difficult to hear in the best of circumstances. Communications with the Power of government can drown out large amounts of localized Knowledge within the economy. According to Gilder, that is the fundamental relationship between Knowledge and Power. Information transmissions regarding economic Knowledge moves the economy forward, while noisy information amplified by government Power holds it back.
While I find Gilder's ideas intriguing, he unfortunately leaves all the details to the reader. As a conceptual metaphor for libertarians it's not bad, but without the mathematical relationships, empirical evidence, and plausible models, I don't think there's much one can do with it other than use it as the basis for some plausible thought experiments which I may propose in future posts. Since Gilder is an ideas guy and definitely not a details guy, I doubt we'll see anything more directly usable from him on this topic.