[EAS]Systems on the Edge

pjk pjk at design.eng.yale.edu
Thu Oct 10 19:20:36 EDT 2002

Mail*Link¨ SMTP               Systems on the Edge

Dear Colleagues -

With his permission I'm forwarding you a mailing I received from
William Dunk, Yale alumnus and faithful EAS-INFO reader. I thought
you'd enjoy this humorous and spirited foray into the
unreliabilities and obsolescences we tolerate but shouldn't.
Not that I haven't been moved to make such comments, e.g.
but William's are more fun.

All best,  --PJK

Date: 10/10/02 2:51 PM
From: William Dunk

News from the Global Province (www.globalprovince.com)
The marketplace of business ideas--a site for investors, business 
executives, journalists, and elitists everywhere. See this week's
additions  below. For subscription information for this free
newsletter, see bottom of  page.
To learn more about William Dunk Partners, visit  www.globalprovince.com/williamdunkpartners.htm.

The Laws of Lawlessness. Back in the 20th century, when things still
seemed  to work, we conjured up a number of laws, sometimes
humorous, always  ironic, that said we were going to hell in a hand
basket. Now in the 21st,  we're in purgatory, and the laws have all
come true. The space program, apparently, gave birth to Murphy's
Law: "If anything can go wrong, it  will." Augustine's Laws, the
title of a book by one-time under-defense  secretary and later
Martin Marietta head Norm Augustine, more or less said:  "As we get
more and more money to spend on trinkets, we put more and more 
electronics in our jet planes, which condemns them to
ever-increasing  breakdowns and downtime." Best of all and all but
forgotten now is  Cybernetics (1948), a short, exceedingly
provocative work by the brilliant  Norbert Weiner, a scientist for
all seasons. In it we learned that the  second Law of Themodynamics
guarantees entropy in all systems. That is,  organized things will
always fall apart. Or as our good friend Regis C.  announced to all
with a chortle several years ago after a disruptive  incident in the
subway: "Well, that equine elimination is just gonna  happen." We
have abundant laws, written before their time, that underscore  the
ultimate lawlessness of the universe and the inevitable Decline and 
Fall of any system you can dream up.

The Myth of Robust Systems. Computer people have nattered on about
robust  systems for half a century. But now that you know that
anything complex is  subject to the slings and arrows of Weiner's
entropy, you can state  positively that such assertions simply don't
hold water. There's really no  such thing as a robust system. And,
circa 2002, as we make our systems more  and more complex, we're
simply experiencing more and more breakdown.  Moreover, since our
systems are interconnected (your house alarm is linked  to an
outside monitoring service located 100 miles away, which may call
the  wrong fire department when something happens), the domino
effect comes into  play. One bolt of lightening in the wrong place
can bring 40 interconnected  systems to a standstill.

There are all sorts of reasons that systems fall apart. In fact, the
chaps  at the Sante Fe Institute in New Mexico not only study
complexity but stay  up nights drumming up ways to make the complex,
which is inherently  unstable, stay glued together. They, and most
of the architects who devise  systems, tend to worry about design
issues, looking at how systems are  wired together. Isn't it ironic
that all the people who look at complex  phenomena always abide in
simple places where the biggest story of the day  is that somebody
forgot to plug in the coffeepot?

Shoddy Merchandise. We mere mortals, well away from the ivory tower,
in the  more complex world outside Santa Fe, can usually look to
something more  down to earth if we are out to avoid breakdown. In
fact, a software guru  from Santa Fe taught us that you can have
poorly designed systems that  function well, if the systems have
lots of redundancy. Are there spare  parts in the system, so when
one conks out another takes over? Are there  enough spare parts on
your shelf (don't believe in maintenance schedules or  just-in-time
delivery) so you can pull a burnt-out part out and plug in  another?
Systems are put together by people often called integrators who, 
either through calculation or ignorance, use lousy components in
their  systems. And they're too vain to acknowledge that even the
best of systems  (i.e., the systems they have built) will fail
often. Simple to say: if you  can use great parts, you will have
less outages.

So this is a warning to us all to watch out for any system that is
called  "integrated." It rarely has rugged enough components to
work, lacks  redundancy, and its creators usually over claim what it
can do, even in the  best of circumstances. This yellow caution
light applies to all sorts of  systems, not just the wired kind such
as computers, electric grids, or  management-information systems. As
oft as not, systems fail because there's  a weak link in the chain.
By the way, that certainly accounts for our worst  space disasters.

For instance, many of the schools your kids go to now have
"integrated  curriculums" (a.k.a. curricula). That really means that
all the courses are  loosely knitted together so that your tots can
read some colonial  literature in Language Arts (an unfortunate
euphemism for what we use to  call English) while George Washington
is bravely losing a battle or two  against the French and Indians in
a Social Studies course. But you can be  sure that many children are
not getting the vital, rigorous training they  need in grammar,
multiplication charts, or periodic tables. In computer  training,
they're fooling around with elaborate Powerpoints, but never  really
learning to keyboard (type). The politically correct textbooks they 
use often border on illiteracy, even if they bear the imprimatur of
some  university in the Midwest. In other words, the components of
these  integrated curricula are lousy. According to some federal
statistics, 30%  of college students will need to take remedial
course in reading, writing,  and mathematics in order to get the
fundamentals they missed growing up.

Just as bad is the customer service system at your utility, which
lacks  real-time data on when the repairs will get done and also
lacks the power  to send any meaningful data to the repair
department so that the right  skills are dispatched to do the fix.
Their systems lack the correct  software, the right training
protocols, etc.

It's not that there aren't simple systems that work. For instance,
back in  1996 or so there was a wonderful bank in Palo Alto called
University  National Bank. As Chief Executive Carl Schmitt then
said, "We're in the put  and take business." He took money in and
gave money out. He did not offer  an endless array of services or
contorted product options. He was in the  deposit business. The
folks who worked there were exceedingly polite; I  seem to remember
an Oriental rug on the floor; and you did not have to wait  in long
lines. Carl gave all his customers some Walla Walla onions at 
Christmas, as a way of saying thanks. He also took great pictures
himself  for his annual report. Since then, Wells Fargo or one of
the huge  integrated financial service institutions took it over,
and reliability is  out the window. There's no longer a great
non-integrator at the helm who  wants to deliver on a simple idea,
using simple, no-nonsense components.  Here and there, around the
nation you can still find the occasional  put-and-take, one horse
bank-- these kinds of banks tend to make money year  after year.

Looking Under the Hood. This world of fragile interdependent systems
 ultimately means that we will have to know what goes into anything
in order  to make our lives work. Most systems and processes are
invisible now, and  even if we get a list of contents, we don't know
what to make of them.  Eventually we might hope for quality
branding, the equivalent of the old  Good Housekeeping seal of
approval. Just as Intel has gotten computer  makers to use "Intel
Inside" labels, we are going to need short-hand labels  that tell us
we are probably getting good goods. This matter of quality  contents
or components presents incredible opportunities for alert business 
people who will increasingly grasp that obsolescence is no longer a
viable  business strategy in a resource-short, environmentally
afflicted, stalled  market economic environment. We need things that
last and work for a long  time. But it's hard to build for a 100
years when you're used to trashing  everything. Here is an almost
shocking business observation: obsolescence  is obsolescent.

The first hints of making-visible-better-insides are just appearing
on the  horizon. McDonald's and Frito-Lay are moving to put better
oils in their  foods, and we expect they will be better able to
dramatize the Health  Inside than the American Health Association or
other non-profits. The air  conditioning man (if he is not part of
the national chains) is able to  describe and install filtration
devices that vastly extend the life of the  cooling system. UPS and
FedEx have made package deliveries transparent to  the consumer, so
that one can track on the Internet an item's progress to  its final
destination. A few companies are becoming more agile at making  the
invisible worlds of systems and services visible to their customers.
 Any product or service is just part of a system: in a world of
breakdown,  we need to see whether the system works or does not

Ask the Repairman. But the insides of systems, products, services,
schools,  governments, whatever, are generally not transparent. As
users, we have two  choices.

1. Ask a repairman. He will probably tell you he would prefer to
work on a  Toyota above all other cars. Or that four TV brands (Sony
and a few others)  stand out above the pack for reliability and
repairability. Repairability  often tells you whether you are
dealing with a well-wrought system. What we  are saying here is that
an informed middleman is a way of improving your  luck with systems.
Japanese manufacturers, similarly, once used middlemen 
(distributors of products) to find out what Americans wanted in
their cars,  TVs, power tools, etc.

2. Find some repair data. In a few cases, raw maintenance data of
various  sorts is available. The government collects on-time and
other data on the  airlines, which is not always easy to uncover but
can be unearthed.  Consumer Reports assembles maintenance data on
car models that is  uncommonly revealing and tells you more than all
the testing performed by  CR's engineers.

In other words, until labeling gets better, you had best find out
about the  reliability of systems from some sort of repair data.
It's the breakdowns  that tell you what you are dealing with.

Call 911. Remember when the Monday morning quarterbacks told us that
Y2K  was really a false alarm, and that the world's computer systems
did not  fall apart despite the fact that computer engineers had not
anticipated,  way back when, that the year 2000 would ever come to
pass. But wait a  minute: systems of all kinds post 2000 are
breaking down everywhere. There  are more power outages with many
more to come because we are simply not  building new generation
capacity. We've been to the very edge of the Dark  Ages in our
financial markets--more than once. The Cold War is over, but  Don
Rumsfeld is still using the Spanish Armada to battle unconventional 
forces and terrorist viruses--the wrong system and wrong weapons to
deal  with an unseen enemy. Who says Y2K never happened?

Chances are you are going to run into total breakdown more and more.
Recently a retired physician checked into a hospital north of
Boston for  surgery. Early one evening he rang for a bedpan, and, no
matter how much he  rang or shouted, nobody came.

The following night, exactly the same thing happened. But he had a
Eureka  and picked up his cell phone to call 911. The local police
were able to  rouse the hospital staff and to get him a bedpan in
the nick of time.

Likewise, Don Imus, the radio talkslash host, was just as ingenious 
recently. No matter what, he could not get a Time Warner cable
repairman to  come to his New York apartment. Then he railed about
it on his radio/cable  show and the minions of TW came running. But,
even after repairs, they  knocked out the reception on one small TV
in his kitchen. The system is so  flawed that even the repairmen
don't know what to do. And cable is one of  the most hated services
in the United States.

The world of broken systems is also a world of broken communication
where  citizens will have to be ingenious beyond belief to fight
entropy. Broken  systems turn ordinary citizens into guerilla
fighters. As Norbert Weiner would have said, entropy "subverts the
exchange of messages." So you'll  just have to learn to beat on your

For a description of William Dunk Partners, Inc., see 

If you would like to subscribe to the Global Province, please send
an email  to us at advisors at beecom.net with the subject heading
"Subscribe." If you  received this message by accident or if you
wish to unsubscribe from the  newsletter, please simply address an
email to us at the same address but  with the subject heading

More information about the EAS-INFO mailing list