Weak Signal® Research

Part II: Information Theory

Bryan S. Coffman

January 16, 1997

back to Part I: Introduction

Signals
Since we’re using the term "weak signal" it might be useful to dip into some information theory to define some terms and examine some concepts. Perhaps a side trip like this into a distantly related field will shed some light on a systematic process for searching out weak signals.

What is a signal?
John R. Pierce has written a wonderful book entitled An Introduction to Information Theory: Symbols, Signals and Noise. He defines a signal as a variable electric current deliberately transmitted by a communication system. We’ll take a broader, more metaphorical view, and call a signal an event in which some living system or other element in the environment transmits a message in the course or as a result of its actions or behavior. In this sense, "everything speaks." The relative order of items on a person’s desk sends a message. A gesture with the hand, posture, or facial expressions communicates. One company buys a controlling interest in another, seemingly unrelated one—what does it mean? Articles published in trade or professional journals contain signals. The crime rate in a city decreases. These are all signals.

Signals contain messages. Really the two are inseparable: message is one way of looking at a communication event and a signal is another. Message is content, signal is process. We use a variety of senses to detect signals: eyes, ears, touch. The message is a string or collection of symbols that comprise the transmission. The message by itself does not MEAN anything. It has no intrinsic value whatsoever. The message is simply the sequence of symbols received over time. The entire value of the message is determined by the receiver.

We are absolutely deluged by signals during our waking and sleeping hours, and these sort themselves into three groups.

Signals beyond our perception
Most signals we are incapable of receiving because they are transmitted at frequencies or across channels that we simply do not have access to. None of us sees in infrared or hears ultrasonic sounds, for example. Radio waves stream constantly through the environment but we are unaware of them because we are not built to receive them. Or, even if we could perceive the signals, we are not in a physical position to do so; activities on the other side of the building, or city go on without our knowledge. To gain access to signals outside the design and range of our personal senses, we have to construct devices that can capture them. This sounds trivial but it may be the most profound lesson to learn in weak signal research. Most of our organizations are not designed, nor have they evolved, with the proper receptors to recognize valuable weak signals. We must redesign our organizations (and ourselves) so they can "tune in" to different frequencies. Without this redesign, we’re sunk. How can you take advantage of a great new idea if you’re incapable of hearing it? How serious is this deficiency? Here's a quote from Megamistakes by Steven Schnaars:

"Video games did not come from the leaders in the board game industry. [They] were introduced by Nolan Bushnell, a 'work-in-the-garage' inventor...

"Disposable diapers came from Procter & Gamble, an industry outsider...

"The list goes on and on. Digital watches were ignored by the Swiss watchmakers. Rough-terrain or 'city' bikes did not come from the leading bicycle manufacturers. Running shoes did not come from the sneaker giants. Both diet and caffeine-free soda did not come from Coke and Pepsi... In every instance, as well as in myriad others, market leaders were amazingly myopic in their perception of emerging markets in their own backyards."

Signals within our perception but unrecognized by our mental models
The next category of signals are those that we are able to receive and process but which are ignored for one reason or another. When we drive cars, we’re aware in general of the traffic around us, the condition of the road, the weather, the scenery that we pass through. But we are not aware in detail of these things, and the more familiar the trip, the less we see and acknowledge. This is good! Imagine the traffic jam in your brain if you tried to consciously process everything you saw on the road? It’s impossible. Instead, we all construct and employ models to filter out most of these signals so we can focus on the few that truly make a difference. Like brake lights illuminating on the car in front of us. We tend to focus on signals that provide us with opportunity or represent a threat. But all of this is based on our mental models of how traffic works. Take your mental model to France or Italy or Singapore and you’ll experience an explosion of conscious awareness of what’s going on around you (or you’ll have a tremendous repair bill). New environments trigger the construction and expansion of new models, or filters. When we bring these new models back into an old, familiar environment, we will see it differently than we did before. Our models tell us what signals to discard out of hand, to the point where the signals might just as well not exist. Plants and animals in nature use this phenomenon in the form of camouflage for protection or stealth. They depend on other animals having models and sensory apparatus that will ignore the signal and move along or come within range.

Not only must we redesign our organizations to see signals that they’re not built to perceive, but we must also redesign the models they use in filtering out what they are built to perceive, but tend to ignore.

Signals recognized by our mental models that we use to modify our behavior
The third category of signals are those that we can perceive, that our models tell us to pay attention to, and which we use to adjust our behavior (what a physicist would call the state of the system in phase space). When a system receives a message it assigns value or meaning to the message. This happens always at the receiver, never at the transmitter. In a sense, the message is only a signal until it is transduced and decoded by the receiver. Then it becomes a message. Again, this may sound trite or obscure, but it points to at least two possibilities for organizational redesign: transduction and decoding. What kinds of signals are allowed to cross the organization's membrane, or boundary, and what mechanisms and languages are used to decode them? To put it more bluntly, what kinds of ideas are permissible to discuss and which are subject to dismissal or ridicule? In how many languages or disciplines is the organization versed? What is the danger of a new development taking the enterprise by surprise only because the language surrounding it is incomprehensible jargon and incapable of being decoded by insiders?

Information
We use the term so unconsciously and automatically that it's easy for two individuals to talk about information, then go their separate ways believing that they are in agreement, when they are really at odds. To one person, information is synonymous with knowledge; to another it means data. Many people would say that a message or signal is information, or at least that it carries information.

Shannon, Wiener, and Two Definitions of Information
Claude Shannon and Norbert Wiener both spent careers thinking about information from a very mathematical perspective. Wiener is one of the modern fathers of Cybernetics, the study of control in organisms and machines. He focused on information as an organizing property of living systems. Terms like "feedback loops", "lag", and "systems theory" emerged from his work into the vernacular. Peter Senge's The Fifth Discipline is a structured application of Cybernetics to the business world and organizational dynamics. Shannon was a mathematician who focused more on the communication aspects of information, in particular on algorithms for sending signals across noisy channels. The difference in how these two men defined information may yield valuable insight for those working to design organizations that excel in identifying and exploiting weak signals.

Wiener sees information as a measure of the degree to which a system is organized; a measure of its order. The DNA molecule with its structured nucleotide chains and the mechanism for translating these into proteins demonstrates a high degree of order, and therefore, possesses a large amount of information. In this sense, information is a measure of the decrease of entropy--or randomness--in a system. A pile of papers strewn on top of a desk has a higher entropy than the same set of papers arranged in labeled files or catalogued in a knowledge base. There's more information resident in the labeled files than there is in the strewn papers. This is definition of information that most of us can relate to, and that makes sense.

The scientists working in the field of complexity theory might amend Wiener's sense of information, and say that information is a measure of the degree of complexity inherent in and exhibited by the organism. A bacteria contains more information than a stone, and a human being more information than a bacteria due to the increase in complexity from one to the other.

Shannon sees things a little differently. He defines information as a measure of uncertainty as to the next message to be received in the communication or messaging event. The higher the uncertainty or surprise, the greater the information, and the greater the entropy. This is the key: Wiener sees information as a decrease in entropy and Shannon thinks of it as an increase in entropy. To be fair, They're not talking about the same kind of entropy, so there's not true dichotomy. But pondering both vantage points from the point of view of the organization may be enlightening. Still, it's hard to understand how an increase in information is a function of the level of uncertainty in communication. Since this line of reasoning is counterintuitive, a few examples are in order.

Imaging a telegrapher sending consecutive letters in the English language. Also assume that the letters combine to make ordinary English words. Acronyms, place names and abbreviations are excluded. You're the receiver. The first letter that comes across the line is an H. A quick glance at the dictionary reveals that the second letter in words that begin with an H is always one of the five vowels plus Y. So, there's not much information that the second letter will convey when it gets transmitted. In fact, many English words are easily deciphered if all of the vowels are removed from their spelling. Imagine that the second letter that's transmitted is an O. Now think about what the third letter might be. It could be most any consonant in the alphabet. We're far more interested in what the third letter might be than we were in what the second letter was. There's more uncertainty, and therefore, more information in the anticipation of the event.

For an extreme example, imagine that the first letter that gets transmitted is a Q. Now, there's no information inherent in the second letter, because there's no uncertainty as to what it might be. Every English word that begins with a Q is followed by a U. There's no point in sending the second character at all, because it's predetermined.

Notice that before a message gets transmitted, there is some degree of information, or uncertainty associated with it. After the transmission, this measure of information collapses to zero because there is no uncertainty associated with a message that's received (assuming it is received error free). In a way, Shannon sees information as "potential" messages and Wiener sees it as the ability of the organism to apply these messages to generate a higher degree of order in itself and its progeny (learning and evolution).

Here's another, completely non-scientific example I like to use. Picture a couple who have been married for thirty or forty years. They've settled into a routine, and often, if you watch them or listen to them, you can predict with a high degree of certainty their course of action following some remark or activity. There's no information contained in their habits. There's nothing wrong with this at all unless these behaviors comprise the whole of the relationship. Habits are a learning tool. They make certain acts automatic so that we can be free to accept new information and undertake new, more challenging experiences. Habits are the expression of the past work of what James Miller calls the relationship between the Associator, Memory and Decider. But what happens if habits become not only the center but the circumference of our being? There's too much order expressed. Too much homeostasis. Not enough entropy in our communications. If we read the same magazines, hold the same discussions with the same people, look at the same numbers, read the same reports, scoff at the same ideas then we have left no window in our lives for information to intrude--to stir things up and make us doubt, challenge, practice, fail, and learn again.

As it is for us individually, so it is with our organizations. In thinking about weak signal research it helps to remember the two kinds of information that we seek. We want signals and the messages they bear to accomplish two things for us:

  1. Provide us with the raw materials out of which we can learn, grow and evolve--to create ever increasing levels of "order" in our systems.

  2. Surprise us. If all we ever hear is what we expect to hear, then we can be fairly certain that we will miss opportunities and fall unsuspecting prey to threats.

Strong signals may surprise us, but their strength at least puts them on our radar screens. The more interesting surprises come from weak signals. This leads us to the problem of how to detect the weak signals in the first place. To do so, we will face one major opponent...

Noise
Recall that Shannon focused on the problem of sending a signal over a noisy channel with some assurance that it will be received intact. John Pierce describes the impact of Shannon's solution on the community of communications engineers at the time. The solution is elegant and unexpected. First I'll recap some of the solutions that were tried before Shannon's work.

If you knew there was a faint signal being transmitted on a certain channel, but you couldn't separate it from other, unwanted signals on the same channel, what options might you have?

First, you might consider boosting the power, or amplifying the signal so that other signals would fall into the background. That would solve the problem, but adding power to signals can be expensive, and if you put too much power into the signal, you might deform the channel. [The first transatlantic cable was destroyed by applying this technique.]

You could ask the transmitter to repeat the message over and over until you were sure you had received it without error, by checking successive transmissions against each other. This is also expensive, time consuming, and not a guarantee of success.

Or, you might reconstruct the channel of more pure materials so that it delivers a cleaner transmission. This will also solve the problem, but again, at a high expense. Any other options? [There are several that will not be discussed here.]

Shannon's answer was simple, but profound. In fact, it was a true weak signal--an answer that was staring everyone in the face, and seems almost trite today, but was inscrutable at the time. All you have to do is understand the noise in the system and then you can make the signal stand out. You don't have to strive to eliminate all of the noise in the channel or keep asking for a repeat of the transmission, or make the transmitter boost the power. Here's how John Pierce puts the discovery:

"'Suppose that I told you that by properly encoding my message, I can send it over even a noisy channel with a completely negligible fraction of errors, a fraction smaller than any assignable value. Suppose that I told you that, if the sort of noise in the channel is known and if its magnitude is known, I can calculate just how many characters I can send over the channel per second and that, if I send any number fewer than this, I can do so virtually without error, while if I try to send more, I will be bound to make errors.'...

"Indeed, the whole problem of efficient and error-free communication turns out to be that of removing from messages the somewhat inefficient redundancy which they have and then adding redundancy of the right sort in order to allow correction of errors made in transmission."

Aha! There's a problem. It's all well and good to talk about understanding the noise in the channel, but how can an organization have any control over adding the right sort of redundancy to the transmission of a weak signal when it has no influence over the sending of the signal?

It can't.

Unless it's the one transmitting the signal. Recall our working definition of Weak Signal® Research. One of the things our organizations must learn is how to redesign themselves to take advantage of the possibilities implied by ecosystems or sets of weak signals. Sometimes the best way to take advantage of a weak signal is to broadcast it to a wider community, to lobby for support, as it were. To invoke the law of increasing returns. Propaganda, fads, gossip, news, jokes all propagate in this manner and we're familiar with this process. They get shared in an exponentially expanding circle of transmitters and receivers. Many times they get transmitted with amazing accuracy. But remember the little game of "telephone" where children sit in a circle and pass a fairly complex sentence from one to the other by whispering? After the message goes around the circle, the children hear the original message and the often humorously corrupted final version. If our organizations are going to encourage the growth of ecosystems of weak signals by rebroadcasting them, they must learn how to encode the signals so that they propagate error-free. The ideas grow in a complex environment, but collapse in a chaotic one.

But we talk about detecting weak signals, it's true that the only part of the equation we can control is understanding the noise in the channel. But that's sufficient. Metaphorically I think of ourselves and our organizations as the channels. The maxim then becomes, "know your noise." Think about what ideas you easily scoff at or discard out of hand and hunt these down as sources for good weak signals. Be suspicious of things that you believe will never come to pass. What ideas challenge your most fundamental beliefs? Why should they be such a threat; is there nothing that can be learned from playing 'Spoze with them?

Remember the discussion above about the three kinds of signals?

  1. Signals beyond our perception.
  2. Signals within our perception but unrecognized by our mental models.
  3. Signals recognized by our mental models that we use to modify our behavior.

Detecting weak signals of type one require somewhat expensive modification of the sensory apparatus of our organizations. Detecting weak signals of type two, however, only ask of us that we examine our mental models and challenge them for the purpose of making them more global, flexible, strategic, and therefore more resilient.

Summary of Part II
Our enterprises receive signals bearing messages every day. They assign value to these messages through a complex comparison of the message with other similar messages stored in memory and through the application of an Associator that helps make decisions on what behavior the message should invoke. Some of the messages we receive are very unexpected, or surprising and may demand a more focused response. We say that such messages are high in information content and entropy. Other, routine messages are still necessary to process out of habit and for purposes of maintaining homeostasis, but they will not be sources of good weak signals. The enterprise uses messages to increase the degree of organization or order inherent in its structure and behavior. This is the other sense of information--a highly ordered organism is said to contain a high degree of information. To enable our organizations to see potentially valuable ecosystems of weak signals, we can modify the sensory apparatus, or change our mental models. The process of changing our mental models can be called, "know your noise".

 

Part 3: Sampling, Uncertainty and Phase Shifts in Weak Signals
Part 4: Evolution and Growth of the Weak Signal to Maturity
Part 5: A Process Model for Weak Signal Research

Other material on Weak Signal Research on this website

Here are some references on the world wide web for you to check out:

  • What is Information, by Karl Erik Sveiby discusses the difference between Shannon and Wiener in depth
  • Cybernetics, Human-Computer Symbiosis and On-line Communities: The Pioneering Vision and the Future of the Global Computer Network, by Ronda Hauben serves as a quick overview of Wiener's work, and John Pierce's book, An Introduction to Information Theory
  • You can download a primer on information theory using simple mathematics, titled, Information Theory Primer With an Appendix on Logarithms by Thomas D. Schneider.
  • The Millennium Clock, by Danny Hillis. It's short; just read it to help you get above all of the technical stuff for a few minutes.

Here are some books on the subject of information theory:

  • Cybernetics: or Control and Communication in the Animal and the Machine by Norbert Wiener
  • The Human Use of Human Beings: Cybernetics and Society by Norbert Wiener
  • Invention: The Care and Feeding of Ideas by Norbert Wiener
  • An Introduction to Information Theory: Symbols, Signals and Noise by John R. Pierce
  • Living Systems by James Grier Miller

copyright 1997, MG Taylor Corporation. All rights reserved
copyrights, terms and conditions

19970116170412.web.bsc

Copyright,
© MG Taylor Corporation, 1995 - 2002

iteration 3.5