C SC 205 Lecture 3: Testing
major resources: Data Structures and the Java Collections Framework Second Edition, William Collins, McGraw-Hill, 2005
Introduction to Programming and OO Design using Java, Niño and Hosch, Wiley & Sons, 2002

[ previous | schedule | next ]

Monday September 17 (week 2)

Testing in General

• Testing is the process of discovering (establishing the presence of) defects in a program
• distinguish from debugging, which is the process of locating and correcting the defects
• we can establish the presence, but not absence of defects!
• Tester's mindset: My goal is to make the program fail!
• testing is essential to program verification - assuring that implementation meets specifications
• general process is to: develop test plan, develop test cases (inputs and expected results), conduct tests, document test results
• testing is an essential component of extreme programming

Testing is multidimensional

• can be done during any development phase: analysis, design, implementation
• must be performed at different levels: a method, a class, a class cluster, a subsystem, the complete system
• can be either static or dynamic:
  • static testing includes inspection of documents: analysis, design, program code
    • static testing is quite effective, uncovering about 60% of all defects.
    • a common practice is code inspections (meeting where several people inspect and discuss the code)
  • dynamic testing involves executing the program code
  • pairs programming combines both static and dynamic: two people at computer, one typing and running code and the other inspecting it
• can be based on either specification or implementation:
  • black box testing is based on specification
    • cannot see the source code, only its method signatures, preconditions, postconditions and invariants
  • glass (white) box testing is based on implementation
    • can see the source code and design test cases based on its structure

Testing a Class

• Each method should be first tested independently, then method sequences should be tested.
• However it is not generally possible to test a constructor or command method in isolation because these do not return a value; test them in conjunction with query methods.
• Reasonable testing sequence:
  • create objects using constructor then test query methods.
  • Correct results here confirm both query methods and constructor.
  • Then test command methods, using query methods to check results and postcondition values.
  • Finally, test sequences of method calls.
  • Remember to avoid using a faulty method in subsequent tests on other methods!
• test plan consists of set of test cases
• test case consists of: objectives, data set, expected results
• test cases should be developed carefully
  • exhaustive (complete) testing is in general not feasible or possible. Suppose method has one int parameter with no precondition. To exhaustively test it would require one test for very possible int value, and there are 2³² (> 4 billion) of them! Two such parameters would require 2⁶⁴ tests!
  • goal is to find a small number of cases which will approximate exhaustive testing
  • see if input data can be partitioned into equivalency groups, partitions in which each value appears to be handled equivalently. Examples below.
  • once equivalency groups established, select three values from each: the two boundary values and one in between. Test only those three.
  • It is not recommended that you use input data which violate method preconditions. The reason: it is not part of the supplier's contract to promise what a method will do when its preconditions are not met.
• black box example: What data would you use to test this method? (byte is 8-bit integer primitive)
  // Returns: number of digits in value
  public int digits(byte value)
• black box example: What data would you use to test this method?
  // precondition: value must be positive and have 5 digits
  // postcondition: this.mean() < 100000
  public void process(int value)
• glass box example: What does this method compute? What preconditions should it have? What data would you use to test it?
  public int pow(int value, int power) {
     int result = 1;
     for (int i = 0; i < power; i++)
        result = result * value;
     return result;
  }
• When testing loops, try to define at least 3 test cases:
  • one that causes the loop not to be executed at all,
  • one that causes one iteration, and
  • one that causes multiple iterations.
• glass box example: What data would you use to test this method? (line numbers added for reference)
  

  public int figure(int x, int y) { 1   int result = 0; 2   if (x > 0) { 3     result += 1;     } else { 4     result += 2;     } 5   if (y > 0) { 6     result += 3;     } else { 7     result += 4;     } 8   return result; }

• In the last example, is it necessary to test all possible execution paths through the method? This is known as path coverage. In statement coverage, it is only necessary that each statement be executed at least once.
• for dynamic testing, need a test harness or test driver program to stand in as class client
• When testing a class, be aware that it is not enough to simply test the methods in isolation; the sequence in which they are invoked can also lead to erroneous results.
• This occurs when an instance variable can be modified inside two or more methods. Such methods are said to be closely coupled. Methods should be called in different sequences in order to assure that their interactions are correct.

Random Number Generation and testing

• Random number generators are useful sources of test input values
• there is an entire literature on such generators
• they are really pseudo-random
• but they produce a reproducible sequence of random values
• reproducibility is key for being repeating test runs and for comparing alternative products or approaches
• Generator is given an initial seed value, to which a formula is applied to produce the first random value.
• The formula is applied to the first random value to produce the second, applied to the second to produce the third, etc.
• Each value in the sequence is a function of the previous
• However, as soon as a value appears for the second time, a fixed cycle has been established
• The cycle is inevitable but formula can be chosen to maximize its length
• Typical formula is called linear congruential
• X_n+1 = ( A X_n + C ) mod M
• A, C, and M are fixed; C is relatively prime to M
• M must be very large -- necessary (but not sufficient) for long sequence of non-repeating values
• Java provides Java.util.Random class and java.lang.Math.random() method

JUnit for testing

• A Java class to promote testing and facilitate production of test harnesses and drivers
• developed specifically for use in extreme programming
• See JUnit.org for more information and downloads