Debugging and Testing

Program Development

You want to design your program so that it is modular. It is made out of simple components that can be developed, tested and replaced if needs be. Each component of the program should be made out of simple functions that have a single functionality. Best way to develop a program is to write code iteratively - that is write short pieces of code and test.

You should write your test suite before you write your code. This gives you the opportunity to think through the functionality of your program.

Debugging

Debugging is the process of finding and rectifying errors in your code when you know that it does not work. The simplest way of debugging is insert print statements of key variables so that you know what their vaules are.

Another way is to run your code through a debugger. Set a break point in your code where you want the execution to stop. Then step through your code line by line watching the values of key variables.

Yet another way is to insert assert statements in your code. The format for that is as follows:

assert Boolean expression

In the execution of the program when the assert statement is encountered the Boolean expression is evaluated. If the expression evaluates to True then the program execution continues. If the expression is False then an AssertionError exception is raised and the program terminates at that point. Here is an example of code that uses assertions:

# File: Assertions.py

# Description: Compute the cycle length of a Collatz or 
#              Hailstone sequence

def cycle_length (n):
  assert n > 0
  c = 0
  while n > 1:
    if (n % 2) == 0:
      n = (n // 2)
    else:
      n = (3 * n) + 1
    c += 1
  assert c > 0
  return c

print ("Assertions.py")

assert cycle_length (1) == 1
assert cycle_length (5) == 6
assert cycle_length (10) == 7

print ("Done")

Once you know that your assertions are working correctly, you can turn them off by using the -O flag like so:

python3 -O Assertions.py

Testing

The goal of testing is to show the presence of bugs and not their absence. The input space for a program could be billions of numbers. We rarely test for all values of the input. We partition the input space and sample. Our test suite is made out of a judicious mix of numbers from each partition of the input space and the edge points. Let us say that the program takes in two real numbers x and y. Here are the partitions and edge cases:

x positive, y positive
x negative, y positive
x negative, y negative
x positive, y negative
x = 0, y = 0
x = 0, y != 0
x != 0, y = 0

A test suite containing seven pairs of points has a reasonable chance of exposing a bug.

Black Box Testing

In black box testing the details of your code are unimportant. The way we test your assignments, exercises in Coding Bat, and programs in Kattis is black box testing. The advantage of using black box is that the implementers of the code and the test writers could be completely different sets of people. The test suites are independent of implementation changes. Normally in black box testing we test for boundary or edge cases and a few interior cases.

Glass Box Testing

In the glass box testing, the code implementation is examined and all the assumptions are tested for. Since the implementation details are known the complete path through the code for a set of input values is tested for.

All branches of if-else branches are tested
For loops, test cases when the loop does not execute even once, execute just once, and execute many times.

Conducting Tests

There are two phases in testing - unit testing and integration testing. In unit testing components of the program are tested in isolation. If a unit depends on another unit a stub is written to mimic it. Integration testing is harder to do because the global behavior of a program is harder to describe, predict, and test.