Information is surprise. To the extent that something is predictable, it doesn't contain information. For example, let's say you're reading a book and you see the sequence "the sky is blu". There is zero surprise and therefore no information conveyed when the next letter you see is 'e'. If you already know the answer, there's no information conveyed.
In information theory, entropy is a measure of unpredictability or information content in a block of data. If the data is maximally compressed, then every single bit of data is unpredictable and the information entropy is maximized.
High entropy, maximally compressed data is indistinguishable from gibberish (have you ever looked at compressed text, for example?) - unless you have the decompression key.
To send information somewhere, you need a channel to communicate. The channel can be a wire or air or space or something else and uses some fraction of the available spectrum and bandwidth of the particular medium to send the information.
To maximize the amount of information you can send over a channel, you want to maximize the entropy of the information, and minimize the entropy of the channel. In other words, you want to make the information look as much like pure noise as possible (compress the hell out of it and/or otherwise encode it) and you want to make the channel itself as quiet and noiseless as possible.
To summarize so far: information is surprise; surprise is unpredictable; maximum unpredictability is indistinguishable from noise without the key or code; and reducing the noise of a communication channel enables a higher rate of information transmission. Except for a few minor details, that's everything there is to know about Information Theory. :-) With some innovation regarding those minor details, coupled with an excellent business model, and you could be the next Qualcomm!
In the book Knowledge and Power, George Gilder proposes that Information Theory as described above is critical for understanding economic activity and economic systems. Each of us has extensive localized Knowledge about consumption and production. On the consumption side, we know what we'd like to consume and what we're willing to trade for those goods and services. On the production side, we know what we're capable of producing and many of us have expertise, sometimes extensive, about various aspects of what's possible to produce given various inputs.
For that Knowledge to be useful, the information related to it must be transmitted to others. For the vast majority of people, the vast majority of that information is indistinguishable from noise. However, some have the "code" to understand the information being transmitted. As a simple example, the price of rubber looks like a random squiggle to almost everybody. However, to a tire manufacturer, there is critically important information embodied in those prices, especially when coupled with expected automotive demand and other related information.
The information channel that supports economic activity is critically important. First, there needs to be one. The institutions of trade, property, law, etc. need to exist and they provide the infrastructure for the channel. According to Gilder, these institutions should be maintained by government.
With the Information Theory construct, it's clearly important that the economy's communication channel has as little noise as possible. This has two implications: the institutions that form the communication channel should be as predictable as possible since unpredictability means surprise which means noise; and the institutions themselves should be as quiet and non-intrusive as possible, otherwise the institutions use up precious channel bandwidth and compete with and overpower transmissions from economic activities, impeding the flow of that information and stifling the activities that would have resulted had that information gotten through.
In Gilder's view, predictability means that the institutions have structures and rules that are unchanging or slowly changing, easily understandable, and easy to follow. This implies "rule of law" as opposed to "rule of man" (or "rule by fiat"). According to Gilder, keeping the institutions quiet means minimizing the information flow from the institutions through the channel. Minimizing regulations, minimizing taxes and keeping the rules associated with the tax regime as simple as possible helps keep down channel noise and enables increased economic communication and activity.
Gilder notes that one of the problems with government institutions generating noise is that government has a lot of Power. Its communications tend to be loud, high power communications, which uses up a tremendous amount of bandwidth, while Knowledge based economic communications tend to be quiet, low-power, numerous, and difficult to hear in the best of circumstances. Communications with the Power of government can drown out large amounts of localized Knowledge within the economy. According to Gilder, that is the fundamental relationship between Knowledge and Power. Information transmissions regarding economic Knowledge moves the economy forward, while noisy information amplified by government Power holds it back.
While I find Gilder's ideas intriguing, he unfortunately leaves all the details to the reader. As a conceptual metaphor for libertarians it's not bad, but without the mathematical relationships, empirical evidence, and plausible models, I don't think there's much one can do with it other than use it as the basis for some plausible thought experiments which I may propose in future posts. Since Gilder is an ideas guy and definitely not a details guy, I doubt we'll see anything more directly usable from him on this topic.
17 comments:
The SETI people talk about this as well. If civilizations are communicating across interstellar distances, it's going to be maximally compressed, which (as Gilder notes) will make it almost indistinguishable from noise. Without the keys, how can you tell?
For that Knowledge to be useful, the information related to it must be transmitted to others. For the vast majority of people, the vast majority of that information is indistinguishable from noise. However, some have the "code" to understand the information being transmitted.
Hmm. I read several of the Amazon reviews and it seems Gilder is trying to battle the heavy hand of determinsm in the physical and social sciences, which makes him ok in my book. But doesn't that quote imply the "knowledge" that underpins belief in free will, individual choice and responsibility, etc. is for entrepreneurial information elites, but not necessarily for us little people, who are just hearing a lot of noise we can't make sense of? Let's go back to 2008 and the controversy over sub-prime mortgages. A lot people, mainly on the right, zeroed in on the fecklessness of people who signed up for mortgages they couldn't afford and held them responsible for making such poor choices. But if the banks and financial institutions had the "code" that let them understand that (and it's pretty hard to argue they didn't) much more easily, why should they get a pass for urging the homeowners to believe they could? If Gilder's theory explains how, say, Walmart succeeds and increases general prosperity, what does it say about the customer who is persuaded by sophisticaed art and psychology to buy something they were unaware they wanted or needed?
Also, does Gilder believe knowledge is just the sum total of a lot of information and that if I have more information (data), I am more knowledgeable by definition? Not sure I buy that. The word knowledge implies an ability to synthesize, understand and put in context. There is clearly a relationship, but are there not situations where knowledge and information can be antithetical? Unless by "code", Gilder just means ponder, muse, reflect and think.
Bret,
Another thoughtful post. I was happy to read your careful remarks in the last paragraph too.
AOG:
We can only hope the aliens are also sending greeting messages in a more simple and obvious language.
Although if you think of it, it is pretty boring to set up a system of communications where, in the more reasonable scenarios, you'll receive answers only in thousands years.
Interstellar exploration in human time scales necessarily requires technology we can barely dream about. Like the ability to build up warp-drve spacetimes and wormholes. All of them requiring either great quantities (or even greater control of smaller quantities) of negative energy densities.
I have thought of writing an essay on methods of search for civilizations using such technology, but it would be so speculative that I stopped before they consider my name to the ig-nobel prize.
Clovis;
Actually I have an old post about that and in my view we don't need much new technology, certainly none that requires new physics. Even back in the 1940s Fermi realized this, hence the "Fermi Paradox". But I don't want to totally hijack Bret's post.
AOG:
Yeah, I know the Fermi paradox, but this is not what I am talking about.
I am really talking about interspace travel (and possibly comunication) in much faster ways, with hypothetical new technology. Enterprise like.
Clovis;
My point was you don't need any of that for interstellar travel and communication.
Bret;
This is also a point about subsidiarity, in that the more dispersed the decision making, the higher the aggregate bandwidth for knowledge. Centralization inevitable restricts information flow, beyond the "Power" argument. This is one of the key points in Hayek's work.
AOG:
You sure do need the fancy new technology - if you want it in human time scales, by what I mean *one* human time scale (I want the fun while I am still alive!).
Fermi's paradox suppose many human lifes' time scale, in a diffusion-like behavior. And communications would still be severly restricted to hundreds and thousands of years.
Bret:
Maybe the main thing your analogy did not mention is the Second Law of Thermodynamics.
You mentioned entropy as a measure of unpredictability. It is also a quantity condemned to ever grow in an isolated system. Both the work of govt., - to mantain a clean channel - and of entrepreneurs (to make knowledge flow in that channel) - will be difficulted over time, thanks to the immutable Second Law.
It takes more energy - external to the system - to counter that. This energy is provided by elements of society which operate outside these simple rules implied by the analogy. In particular, only the market forces, and govt. forces, are not enough to guarantee generations of workers and citizens, "behaving well", to continually provide energy to your grid.
Clovis;
Feel free to fire away here.
Well, it is crazy to answer such an old thread, but I did, so it stays registered in your memory when you download your brain into a supercomputer.
Clovis,
Gilder does mention the 2nd Law, but makes a hash of it IMO, so I decided to leave that part out.
Entropy in Information Theory is different than entropy in Thermodynamics. Per Wikipedia: "The inspiration for adopting the word entropy in information theory came from the close resemblance between Shannon's formula and very similar known formulae from thermodynamics. ... At an everyday practical level the links between information entropy and thermodynamic entropy are not evident."
Yet Wikipedia goes on to say: "thermodynamic entropy, as explained by statistical mechanics, should be seen as an application of Shannon's information theory: the thermodynamic entropy is interpreted as being proportional to the amount of further Shannon information needed to define the detailed microscopic state of the system, that remains uncommunicated by a description solely in terms of the macroscopic variables of classical thermodynamics, with the constant of proportionality being just the Boltzmann constant. For example, adding heat to a system increases its thermodynamic entropy because it increases the number of possible microscopic states for the system, thus making any complete state description longer."
That seems like a fairly strong link between the two to me.
Now for the more specific discussion.
You wrote: "It is also a quantity condemned to ever grow in an isolated system."
Yes, but that's not applicable. The universe and solar system have growing entropy, but at the moment, the sun is pumping so much energy into our earth, that entropy is not inherently growing. That is, the earth is not an isolated system. For example, has civilization tended toward randomness in the last few thousand years? I don't think so.
You wrote: "In particular, only the market forces, and govt. forces, are not enough to guarantee generations of workers and citizens, "behaving well"..."
I do agree with that, however. Not because of the 2nd Law, however. Or if so, very indirectly. The problem is more one of looting. The system I describe has a great deal of value. There will inherently be those who want to take value and that taking either adds noise to the system, or the noise is added purposefully so that value can be extracted without being missed.
Peter asks: "...does Gilder believe knowledge is just the sum total of a lot of information...?"
No. Rather he thinks it's the other way around. People have localized knowledge. In order to make that knowledge useful to themselves and others they need to be able to convey information relevant to that knowledge to other people.
Certainly knowledge and information can be antithetical: propaganda, for example.
"Code" simply means the capacity to decipher the information in a useful fashion.
Peter wrote: "But doesn't that quote imply the "knowledge" that underpins belief in free will, individual choice and responsibility, etc. is for entrepreneurial information elites, but not necessarily for us little people, who are just hearing a lot of noise we can't make sense of?"
Not really. We are each exposed to a huge amount of information and have access to unfathomable amounts of additional information. We are each capable of understanding only an extremely small fraction of that information. This is as true for the "entrepreneurial information elites" (whatever that means?) as for the rest of us. We understand what we do because we have the "code" or knowledge to understand those bits of information. I understand a lot of information about robotics that's probably meaningless to you, while you understand a lot of information regarding legal constructs that would be unintelligible to me. You understand legal code, I understand C code containing robotics algorithms. We each play our part consuming bits of information flowing through the information channel of the economy, processing that information, and making decisions that create value based on the processed information.
One thing neither Gilder nor I touched on is the concept of bad information (like propaganda above). Unfortunately, there are plenty of falsehoods that masquerade as good information. The bad information can lead to bad decisions and, as the financial crisis showed, can be catastrophic in a systematic sort of way.
However, I notice you didn't mention the corruption of the channel by government regulations and other encumbrances like "moral hazard/too-big-to-fail." There were a great number of factors in the financial crisis, and while poor private choices possibly based on bad private information was certainly one factor, the distortions caused by government can't be completely ignored either.
aog wrote: "Centralization inevitable restricts information flow..."
Gilder (and I) totally agree. That's pretty much the entire point he's trying to make.
To take your first example, if you had an absurdist poem that read, The sky is blunt, then it would be full of information; which I don't think you want to defend.
the distortions caused by government can't be completely ignored either.
Rest assured I agree 100%. In fact, I'm not sure that regulation was even the right word. As far as I can tell, the feds were trying to manage the market by fiat. I'm willing to at least give a hearing to those advocating certain kinds of public regulation, but if there is one historical truth we have learned in the past fifty years, it is that government efforts to manage commercial enterprise, whether through market direction or nationalizations, is almost always disastrous. The only arguable exception is where a "frontier" is so lacking in population and infrastructure that it can't support private enterprise--yet. In such a case, the left can be counted on to hang on desperately long after that kind of need has reached its sell-by date.
My point was much narrower and related to his theory. If information "flows" play the systematic and key role he claims, and if his theory presupposes access to it is necessarily restricted, what does that do to classical conservative thinking about free and equal citizens striking bargains in the marketplace?
classical conservative thinking about free and equal citizens striking bargains in the marketplace?
I don't know about classical conservative thinking, but this is one of the foundational thoughts for libertarianism. It is precisely because information flows are intrinsically restricted that you need free citizens striking bargains, thereby creating the market place, because the flow restrictions prevent centralization of the information. As Bret wrote, each citizen has a lot of information that can't flow to others, so this is the only way to use that information in general.
Bret,
---
That seems like a fairly strong link between the two to me.
---
There are interesting discussions on how information relates to entropy, most of them in quantum mechanics, where there are different ways to define how the matrix of density of states relates to information measurement. But taking aside the details, for practical ends both are pretty much the same thing.
---
Clovis wrote: "It is also a quantity condemned to ever grow in an isolated system."
Yes, but that's not applicable. The universe and solar system have growing entropy, but at the moment, the sun is pumping so much energy into our earth, that entropy is not inherently growing.
---
Well, let me rephrase your point. In general our total entropy in Earth is growing. What happens is that, at the level of civilizations, we use the energy of the Sun (and its consequences, e.g. to use fossil fuels is to use accumulated Sun energy) to manipulate our
enviroment in such a way to control for that, i.e. we diminish the entropy of a subsystem, increasing it in another part of the total system, paying up for that with energy provided by the Sun.
---
For example, has civilization tended toward randomness in the last few thousand years? I don't think so.
---
I agree it has not. My point was that, taking your analogy seriously, a system composed of only beamers of information (market) and cables for that to flow (govt.), being an isolated system, would only have growing entropy, ultimately turning to chaos. They do not because they have external sources of energy to take care of that. How to care for those sources of energy is the main thing lacking in the analogy - if you complete that, than I think you'll have an effective model for society.
Our discussions here for, or against, Libertarianism, often are really on how to care for the source of energy. As I've said many times, Libertarianism would probably decay to a dysfunctional state due to errors in dealing with the power source.
---
The problem is more one of looting. The system I describe has a great deal of value. There will inherently be those who want to take value and that taking either adds noise to the system, or the noise is added purposefully so that value can be extracted without being missed.
---
"Value" is a subjective human concept, and to introduce it in your analogy is to miss the point - you usually want a physical analogy so you can clear things out of much subjectivism.
Every system has thermal fluctuations, and they are bigger as the temperature of the system grows up. Those fluctuations induce errors. To diminish the temperature, hence the errors, you need energy - this is really just the 2nd law argument above written in details.
So if the system only directs energy to sustain the beams and the infrastructure for their propagation, you'll heat up with no cooling to ever help you.
A better use of the energy is to direct part of it to also cool the system down. Maybe you can think of it as part of the price to maitain the infrastruture, and attibute it to the government part of the bill. Or you can send the bill to the beam makers, as they are also interested in having that functional infrastructure. One way or another, someone need to pay for that energy.
Statists want the bill with the govt. Old school Conservatives would prefer to place it upon the markets. Moderns day Libertarians want to turn off the cooler and forget the laws of Physics.
(Yeah, the jab may be unjustified, but I could not loose the opportunity :-)
but this is one of the foundational thoughts for libertarianism
Also for non-libertarian conservatives, although perhaps not so dogmatically that we would celebrate banks that dangle cash in front of people when they know full well paying it back will be problematic. My concern is that, in addition to promoting general prosperity, the defence of the market should also be understood as a political and social value that maximizes freedom and democracy and not just some abstract "system" that empirically results in everybody maximizing their wealth and making choices that are best for them, which is clearly not the case.
Post a Comment