CSC 150 Chapter 15: Selected Ethics Topics and Case Studies
information resources: as indicated below

[ previous | schedule | first ]

Topic Index
[ Codes of Conduct | Therac-25 tragedy | acceptable use | piracy | privacy ]

Ethical responsibilities of Software Engineers and Computer Scientists
 

Codes of Ethics and Professional Conduct

Association of Computing Machinery, ACM
http://www.acm.org/constitution/code.html

ACM and IEEE Computer Society
http://www.acm.org/serving/se/code.htm
 

Software Engineering case study:  THERAC-25

THERAC-25 was developed for radiation treatment of cancerous tumors.  It had two settings, a low setting called electron for tumors close to the surface and a high setting called x-ray for deeper ones.  The x-ray setting must be used in conjunction with a tungsten shield.  Controlled by a DEC (now owned by Compac) PDP-11/23 minicomputer.

It was poorly designed, and programmed in assembly language by one programmer.  There was no documentation for the code, and equipment documentation was written in French.

This poor design led to the deaths of three people during 1985 and 1986 before it was discovered.  Because death due to radiation is not normally instantaneous, they didn’t realize the problem until it was too late.

Several errors were made in system design, the most egregious being:

  1. hardware interlock was replaced with software interlock.  Hardware interlock prevented x-ray (“high beam”) mode from being activated unless tungsten shield was in place, an expensive mechanism.
  2. textual user interface (this was early 1980s) was designed to have data entry area in middle of the screen plus command line at the bottom.  When cursor moves from data entry area to command line area, the software Boolean variable “data entry complete” is set to true.  If operator then moves cursor back to data entry area without entering a command, this Boolean variable is not reset to false.
  3. Two functions, Magnet and Timer, share a Boolean variable called “bending magnet flag”.  Magnet controls setting of magnets, and takes about 8 seconds to run.  Magnet first sets the flag true, then enters a loop which calls Timer repeatedly.  Timer sets the flag false before returning.  This means the flag will be set false during the first loop iteration, even though “bending magnet flag” should be true until all loop iterations are complete!
  4. Timer simply implements a time delay.  It also checks to see if the user is editing in the data entry area, but this check is made only if the “bending magnet flag” is true! As stated above, it sets the "bending magnet flag" false before returning.
  5. The user interface was designed to display all errors with numeric code, with no indication of the severity of the error.


The scenario that led to the tragedies:

The operator mistakenly specified x-ray mode during data entry, then
a. moved the cursor to the command line area,
b. immediately noticed the mistake,
c. moved the cursor back to the data entry area
d. corrected the mistake by changing the X to E for electron (“low beam”) mode.
e. moved the cursor back to the command line area
f. performed the above steps in less than 8 seconds

When step (a) was performed, the “data entry complete” flag was set (item 2).  This also caused Magnet (item 3) to be called to set the magnets.  While Magnet was iterating, the operator performed steps b, c, d and e.  Because “bending magnet flag” was set false during the first call to Timer instead of after the last call, the check (item 4) which would have detected the operator’s actions was not performed.  Magnet completed, having set everything up for x-ray mode even though the data entry screen now shows E for electron mode.  The operator, now back at the command line, entered the command to start the beam.

The software check for tungsten shield was then performed, and this software noticed that the magnets were set for x-ray mode but the operator setting was for electron mode.  It displayed error #54, and the operator reset the computer.  This caused the tungsten shield to withdraw but left the machine in x-ray mode.  X-rays were applied to the doomed patient.

Where do responsibilities lie?

Here is my major reference
Detailed investigation  http://courses.cs.vt.edu/~cs3604/lib/Therac_25/Therac_1.html

University Computer Acceptable Use Policies

Here are two examples.

Carnegie Mellon University
http://www.cmu.edu/policies/documents/Computing.htm

Otterbein College
http://www.otterbein.edu/its/rup.asp

Intellectual property rights and piracy

Privacy

There are many more issues; we may not even get this far....
 

[ C SC 150 | Peter Sanderson | Math Sciences server  | Math Sciences home page | Otterbein ]
Last updated:
Peter Sanderson (PSanderson@otterbein.edu)