[EAS]Systems on the Edge
pjk at design.eng.yale.edu
Thu Oct 10 19:20:36 EDT 2002
Mail*Link¨ SMTP Systems on the Edge
Dear Colleagues -
With his permission I'm forwarding you a mailing I received from
William Dunk, Yale alumnus and faithful EAS-INFO reader. I thought
you'd enjoy this humorous and spirited foray into the
unreliabilities and obsolescences we tolerate but shouldn't.
Not that I haven't been moved to make such comments, e.g.
but William's are more fun.
All best, --PJK
Date: 10/10/02 2:51 PM
From: William Dunk
News from the Global Province (www.globalprovince.com)
The marketplace of business ideas--a site for investors, business
executives, journalists, and elitists everywhere. See this week's
additions below. For subscription information for this free
newsletter, see bottom of page.
To learn more about William Dunk Partners, visit www.globalprovince.com/williamdunkpartners.htm.
The Laws of Lawlessness. Back in the 20th century, when things still
seemed to work, we conjured up a number of laws, sometimes
humorous, always ironic, that said we were going to hell in a hand
basket. Now in the 21st, we're in purgatory, and the laws have all
come true. The space program, apparently, gave birth to Murphy's
Law: "If anything can go wrong, it will." Augustine's Laws, the
title of a book by one-time under-defense secretary and later
Martin Marietta head Norm Augustine, more or less said: "As we get
more and more money to spend on trinkets, we put more and more
electronics in our jet planes, which condemns them to
ever-increasing breakdowns and downtime." Best of all and all but
forgotten now is Cybernetics (1948), a short, exceedingly
provocative work by the brilliant Norbert Weiner, a scientist for
all seasons. In it we learned that the second Law of Themodynamics
guarantees entropy in all systems. That is, organized things will
always fall apart. Or as our good friend Regis C. announced to all
with a chortle several years ago after a disruptive incident in the
subway: "Well, that equine elimination is just gonna happen." We
have abundant laws, written before their time, that underscore the
ultimate lawlessness of the universe and the inevitable Decline and
Fall of any system you can dream up.
The Myth of Robust Systems. Computer people have nattered on about
robust systems for half a century. But now that you know that
anything complex is subject to the slings and arrows of Weiner's
entropy, you can state positively that such assertions simply don't
hold water. There's really no such thing as a robust system. And,
circa 2002, as we make our systems more and more complex, we're
simply experiencing more and more breakdown. Moreover, since our
systems are interconnected (your house alarm is linked to an
outside monitoring service located 100 miles away, which may call
the wrong fire department when something happens), the domino
effect comes into play. One bolt of lightening in the wrong place
can bring 40 interconnected systems to a standstill.
There are all sorts of reasons that systems fall apart. In fact, the
chaps at the Sante Fe Institute in New Mexico not only study
complexity but stay up nights drumming up ways to make the complex,
which is inherently unstable, stay glued together. They, and most
of the architects who devise systems, tend to worry about design
issues, looking at how systems are wired together. Isn't it ironic
that all the people who look at complex phenomena always abide in
simple places where the biggest story of the day is that somebody
forgot to plug in the coffeepot?
Shoddy Merchandise. We mere mortals, well away from the ivory tower,
in the more complex world outside Santa Fe, can usually look to
something more down to earth if we are out to avoid breakdown. In
fact, a software guru from Santa Fe taught us that you can have
poorly designed systems that function well, if the systems have
lots of redundancy. Are there spare parts in the system, so when
one conks out another takes over? Are there enough spare parts on
your shelf (don't believe in maintenance schedules or just-in-time
delivery) so you can pull a burnt-out part out and plug in another?
Systems are put together by people often called integrators who,
either through calculation or ignorance, use lousy components in
their systems. And they're too vain to acknowledge that even the
best of systems (i.e., the systems they have built) will fail
often. Simple to say: if you can use great parts, you will have
So this is a warning to us all to watch out for any system that is
called "integrated." It rarely has rugged enough components to
work, lacks redundancy, and its creators usually over claim what it
can do, even in the best of circumstances. This yellow caution
light applies to all sorts of systems, not just the wired kind such
as computers, electric grids, or management-information systems. As
oft as not, systems fail because there's a weak link in the chain.
By the way, that certainly accounts for our worst space disasters.
For instance, many of the schools your kids go to now have
"integrated curriculums" (a.k.a. curricula). That really means that
all the courses are loosely knitted together so that your tots can
read some colonial literature in Language Arts (an unfortunate
euphemism for what we use to call English) while George Washington
is bravely losing a battle or two against the French and Indians in
a Social Studies course. But you can be sure that many children are
not getting the vital, rigorous training they need in grammar,
multiplication charts, or periodic tables. In computer training,
they're fooling around with elaborate Powerpoints, but never really
learning to keyboard (type). The politically correct textbooks they
use often border on illiteracy, even if they bear the imprimatur of
some university in the Midwest. In other words, the components of
these integrated curricula are lousy. According to some federal
statistics, 30% of college students will need to take remedial
course in reading, writing, and mathematics in order to get the
fundamentals they missed growing up.
Just as bad is the customer service system at your utility, which
lacks real-time data on when the repairs will get done and also
lacks the power to send any meaningful data to the repair
department so that the right skills are dispatched to do the fix.
Their systems lack the correct software, the right training
It's not that there aren't simple systems that work. For instance,
back in 1996 or so there was a wonderful bank in Palo Alto called
University National Bank. As Chief Executive Carl Schmitt then
said, "We're in the put and take business." He took money in and
gave money out. He did not offer an endless array of services or
contorted product options. He was in the deposit business. The
folks who worked there were exceedingly polite; I seem to remember
an Oriental rug on the floor; and you did not have to wait in long
lines. Carl gave all his customers some Walla Walla onions at
Christmas, as a way of saying thanks. He also took great pictures
himself for his annual report. Since then, Wells Fargo or one of
the huge integrated financial service institutions took it over,
and reliability is out the window. There's no longer a great
non-integrator at the helm who wants to deliver on a simple idea,
using simple, no-nonsense components. Here and there, around the
nation you can still find the occasional put-and-take, one horse
bank-- these kinds of banks tend to make money year after year.
Looking Under the Hood. This world of fragile interdependent systems
ultimately means that we will have to know what goes into anything
in order to make our lives work. Most systems and processes are
invisible now, and even if we get a list of contents, we don't know
what to make of them. Eventually we might hope for quality
branding, the equivalent of the old Good Housekeeping seal of
approval. Just as Intel has gotten computer makers to use "Intel
Inside" labels, we are going to need short-hand labels that tell us
we are probably getting good goods. This matter of quality contents
or components presents incredible opportunities for alert business
people who will increasingly grasp that obsolescence is no longer a
viable business strategy in a resource-short, environmentally
afflicted, stalled market economic environment. We need things that
last and work for a long time. But it's hard to build for a 100
years when you're used to trashing everything. Here is an almost
shocking business observation: obsolescence is obsolescent.
The first hints of making-visible-better-insides are just appearing
on the horizon. McDonald's and Frito-Lay are moving to put better
oils in their foods, and we expect they will be better able to
dramatize the Health Inside than the American Health Association or
other non-profits. The air conditioning man (if he is not part of
the national chains) is able to describe and install filtration
devices that vastly extend the life of the cooling system. UPS and
FedEx have made package deliveries transparent to the consumer, so
that one can track on the Internet an item's progress to its final
destination. A few companies are becoming more agile at making the
invisible worlds of systems and services visible to their customers.
Any product or service is just part of a system: in a world of
breakdown, we need to see whether the system works or does not
Ask the Repairman. But the insides of systems, products, services,
schools, governments, whatever, are generally not transparent. As
users, we have two choices.
1. Ask a repairman. He will probably tell you he would prefer to
work on a Toyota above all other cars. Or that four TV brands (Sony
and a few others) stand out above the pack for reliability and
repairability. Repairability often tells you whether you are
dealing with a well-wrought system. What we are saying here is that
an informed middleman is a way of improving your luck with systems.
Japanese manufacturers, similarly, once used middlemen
(distributors of products) to find out what Americans wanted in
their cars, TVs, power tools, etc.
2. Find some repair data. In a few cases, raw maintenance data of
various sorts is available. The government collects on-time and
other data on the airlines, which is not always easy to uncover but
can be unearthed. Consumer Reports assembles maintenance data on
car models that is uncommonly revealing and tells you more than all
the testing performed by CR's engineers.
In other words, until labeling gets better, you had best find out
about the reliability of systems from some sort of repair data.
It's the breakdowns that tell you what you are dealing with.
Call 911. Remember when the Monday morning quarterbacks told us that
Y2K was really a false alarm, and that the world's computer systems
did not fall apart despite the fact that computer engineers had not
anticipated, way back when, that the year 2000 would ever come to
pass. But wait a minute: systems of all kinds post 2000 are
breaking down everywhere. There are more power outages with many
more to come because we are simply not building new generation
capacity. We've been to the very edge of the Dark Ages in our
financial markets--more than once. The Cold War is over, but Don
Rumsfeld is still using the Spanish Armada to battle unconventional
forces and terrorist viruses--the wrong system and wrong weapons to
deal with an unseen enemy. Who says Y2K never happened?
Chances are you are going to run into total breakdown more and more.
Recently a retired physician checked into a hospital north of
Boston for surgery. Early one evening he rang for a bedpan, and, no
matter how much he rang or shouted, nobody came.
The following night, exactly the same thing happened. But he had a
Eureka and picked up his cell phone to call 911. The local police
were able to rouse the hospital staff and to get him a bedpan in
the nick of time.
Likewise, Don Imus, the radio talkslash host, was just as ingenious
recently. No matter what, he could not get a Time Warner cable
repairman to come to his New York apartment. Then he railed about
it on his radio/cable show and the minions of TW came running. But,
even after repairs, they knocked out the reception on one small TV
in his kitchen. The system is so flawed that even the repairmen
don't know what to do. And cable is one of the most hated services
in the United States.
The world of broken systems is also a world of broken communication
where citizens will have to be ingenious beyond belief to fight
entropy. Broken systems turn ordinary citizens into guerilla
fighters. As Norbert Weiner would have said, entropy "subverts the
exchange of messages." So you'll just have to learn to beat on your
For a description of William Dunk Partners, Inc., see
If you would like to subscribe to the Global Province, please send
an email to us at advisors at beecom.net with the subject heading
"Subscribe." If you received this message by accident or if you
wish to unsubscribe from the newsletter, please simply address an
email to us at the same address but with the subject heading
More information about the EAS-INFO