Design by Contract: The Lessons of Ariane
by Jean-Marc Jézéquel, IRISA
and Bertrand Meyer, ISE
This article appeared in a slightly different form in Computer
(IEEE), as part of the Object-Oriented department, in January of
1997 (vol. 30, no. 2, pages 129-130).
Reader reactions to the article published in IEEE's Computer
magazine appear at
the end of the article.
Keywords: Contracts, Ariane, Eiffel, reliable
software, correctness, specification, reuse, reusability, Java,
CORBA, IDL.
How not to test your software
Several earlier columns in IEEE Computer have emphasized
the importance of Design
by ContractTM for constructing
reliable software. A $500-million software error provides a sobering
reminder that this principle is not just a pleasant academic ideal.
On June 4, 1996, the maiden flight of the European Ariane 5
launcher crashed about 40 seconds after takeoff. Media reports
indicated that the amount lost was half a billion dollars --
uninsured.
The CNES (French National Center for Space Studies) and the
European Space Agency immediately appointed an international inquiry
board, made of respected experts from major European countries, who
produced their report in hardly more than a month. These agencies
are to be commended for the speed and openness with which they
handled the disaster. The committee's report is available on the Web
in these two places:
It is a remarkably short, simple, clear and forceful document.
Its conclusion: the explosion was the result of a software error
-- possibly the costliest in history (at least in dollar terms,
since earlier cases have caused loss of life).
Particularly vexing is the realization that the error came from a
piece of the software that was not needed during the crash.
It has to do with the Inertial Reference System, for which we will
keep the acronym SRI used in the report, if only to avoid the
unpleasant connotation that the reverse acronym could evoke for US
readers. Before lift-off certain computations are performed to align
the SRI. Normally they should be stopped at -9 seconds, but in the
unlikely event of a hold in the countdown resetting the SRI could,
at least in earlier versions of Ariane, take several hours; so the
computation continues for 50 seconds after the start of flight mode
-- well into the flight period. After takeoff, of course, this
computation is useless; but in the Ariane 5 flight it caused an
exception, which was not caught and -- boom.
The exception was due to a floating-point error: a conversion
from a 64-bit integer to a 16-bit signed integer, which should only
have been applied to a number less than 2^15, was erroneously
applied to a greater number, representing the "horizontal bias" of
the flight. There was no explicit exception handler to catch the
exception, so it followed the usual fate of uncaught exceptions and
crashed the entire software, hence the on-board computers, hence the
mission.
This is the kind of trivial error that we are all familiar with
(raise your hand if you have never done anything of the sort),
although fortunately the consequences are usually less expensive.
How in the world can it have remained undetected, and produced such
a horrendous outcome?
Is this incompetence?
No. Everything indicates that the software process was carefully
organized and planned. The ESA's software people knew what they were
doing and applied widely accepted industry practices.
Is it an outrageous software management problem?
No. Obviously something went wrong in the validation and
verification process (otherwise there would be no story to write),
and the Inquiry Board makes a number of recommendations to improve
the process, it is clear from its report that systematic
documentation, validation and management procedures were in place.
The contention often made in the software engineering literature
that most software problems are primarily management problems is not
borne out here. The problem is technical. (Of course one can always
argue that good management will spot technical problems early
enough.)
Is it the programming language's fault?
Although one may criticize the Ada exception mechanism, it could
have been used here to catch the exception. In fact, quoting the
report:
Not all the conversions were protected because a
maximum workload target of 80% had been set for the SRI computer.
To determine the vulnerability of unprotected code, an analysis
was performed on every operation which could give rise to an ...
operand error. This led to protection being added to four of
[seven] variables... in the Ada code. However, three of the
variables were left unprotected.
In other words the potential problem of failed arithmetic
conversions was recognized. Unfortunately, the fatal exception was
among the three that were not monitored, not the four that were.
Is it a design error?
Why was the exception not monitored? The analysis revealed that
overflow (a horizontal bias not fitting in a 16-bit integer) could
not occur. Was the analysis wrong? No! It was right -- for the
Ariane 4 trajectory. For Ariane 5, with other trajectory
parameters, it does not hold any more.
Is it an implementation error?
Although one may criticize the removal of a protection to achieve
more performance (the 80% workload target), it was justified by the
theoretical analysis. To engineer is to make compromises. If you
have proved that a condition cannot happen, you are entitled not to
check for it. If every program checked for all possible and
impossible events, no useful instruction would ever get executed!
Is it a testing error?
Not really. Not surprisingly, the Inquiry Board's report
recommends better testing procedures, and testing the whole system
rather than parts of it (in the Ariane 5 case the SRI and the flight
software were tested separately). But if one can test more one
cannot test all. Testing, we all know, can show the presence of
errors, not their absence. And the only fully "realistic" test is to
launch; this is what happened, although the launch was not really
intended as a $500-million test of the software.
So what is it?
It is a reuse error. The SRI horizontal bias module was
reused from a 10-year-old software, the software from Ariane 4.
But this is not the full story:
It is a reuse specification error
The truly unacceptable part is the absence of any kind of precise
specification associated with a reusable module.
The requirement that the horizontal bias should fit on 16 bits
was in fact stated in an obscure part of a document. But in the code
itself it was nowhere to be found!
From the principle of Design by Contract expounded by earlier
columns, we know that any software element that has such a
fundamental constraint should state it explicitly, as part of a
mechanism present in the programming language, as in the Eiffel
construct
where the precondition states clearly and precisely what the
input must satisfy to be acceptable.
Does this mean that the crash would automatically have been
avoided had the mission used a language and method supporting
built-in assertions and Design by Contract? Although it is always
risky to draw such after-the-fact conclusions, the answer is
probably yes:
Assertions (preconditions and postconditions in particular) can be
automatically turned on during testing, through a simple compiler
option. The error might have been caught then.
Assertions can remain turned on during execution, triggering an
exception if violated. Given the performance constraints on such a
mission, however, this would probably not have been the case.
But most
importantly the assertions are a prime component of the software
and its documentation ("short form", produced automatically by
tools). In an environment such as that of Ariane where there is so
much emphasis on quality control and thorough validation of
everything, they would be the QA team's primary focus of
attention. Any team worth its salt would have checked
systematically that every call satisfies the precondition. That
would have immediately revealed that the Ariane 5 calling software
did not meet the expectation of the Ariane 4 routines that it
called.
The lesson for every software developer
The Inquiry Board makes a number of recommendations with respect
to improving the software process of the European Space Agency. Many
are justified; some may be overkill; some may be very expensive to
put in place. There is a more simple lesson to be learned from this
unfortunate event:
Reuse without a
contract is sheer folly!
From CORBA to C++ to Visual Basic to ActiveX to
Java, the hype is on software components. The Ariane 5 blunder
shows clearly that naïve hopes are doomed to produce results
far worse than a traditional, reuse-less software
process. To attempt to reuse software without Eiffel-like
assertions is to invite failures of potentially disastrous
consequences. The next time around, will it only be an empty
payload, however expensive, or will it be human lives?
It is regrettable that this lesson has not been
heeded by such recent designs as Java (which added insult to
injury by removing the modest assert instruction of C!), IDL (the
Interface Definition Language of CORBA, which is intended to
foster large-scale reuse across networks, but fails to provide
any semantic specification mechanism), Ada 95 and ActiveX.
For reuse to be effective, Design by Contract is
a requirement. Without a precise specification attached to
each reusable component -- precondition, postcondition,
invariant -- no one can trust a supposedly reusable
component. |
Reader reactions
The February 1997 issue of IEEE Computer
contained two letters from readers commenting on the article. Here
are some extracts from these letters and from the response by one of
the authors:
Tom Demarco, The Atlantic Systems Guild (co-author of
PeopleWare):
Jean-Marc Jézéquel and Bertrand Meyer are precisely
on-target in their assessment of the Ariane-5 failure. This was
the kind of problem that a reasonable contracting mechanism almost
certainly would have caught; the kind of problem that almost no
other defense would have been likely to catch.
I believe that the use of Eiffel-like module contracts is the
most important non-practice in software today.
Roy D. North, Falls Church, Va.:
Our designs must incorporate safety factors, and we
must freeze the design before we produce the product (the
software).
Bertrand Meyer's response:
What software managers must understand is that Design
by Contract is not a pie-in-the-sky approach for special,
expensive projects. It is a pragmatic set of techniques available
from several commercial and public-domain Eiffel sources and
applicable to any project, large or small.
It is not an exaggeration to say that applying Eiffel's
assertion-based O-O development will completely change your view
of software construction ... It puts the whole issue of errors,
the unsung part of the software developer's saga, in a completely
different light.
To learn more
An extensive discussion of Design by Contract and
its consequences on the software development process is in the
following book:
-
Object-Oriented
Software Construction, 2nd edition
Learn more about the using Eiffel to develop
mission-critical systems, read this book:
-
Object-Oriented
Software Engineering with Eiffel
Also, ISE's Web pages contain an introduction
to the concepts of Design by Contract.
For a contrarian perspective
For a different view of the issue (written in response to the
IEEE Computer article) see Ken
Garlington's paper. Although we disagree with Mr. Garlington's
analysis, as expressed in Usenet discussions, we feel it is part of
this site's duty to its readers to give them access to contrarian
views, letting them make them make up their own minds, for the
greater benefit of software quality.
| 
