Quick Tips: Assess Your Mistakes

In The Method and Scope of Genetics, published in 1908, William Bateson offers this advice:

If I may throw out a word of counsel to beginners, it is: Treasure your exceptions! […] Keep them always uncovered and in sight. Exceptions are like the rough brickwork of a growing building which tells that there is more to come and shows where the next construction is to be.

Of course, this quote, which predates programming languages by several decades, is not about the java.lang.NullPointerException. Instead, it is referring to a curious trait in humans: we really, really don’t like it when reality doesn’t match up with how we think it should be. It’s a very uncomfortable feeling to be wrong—in fact, we dislike it so much that our brains will just get rid of information that doesn’t fit with our worldview. This tendency is what Bateson is exhorting the young scientist to fight against: by keeping exceptions to established principles in mind, a scientist can understand how to improve existing theories to account for these exceptions.

While this advice is not directly applicable to (most) computer science at the undergraduate level, I do see one common area where this tendency to discard information about errors causes issues: debugging. If you need to debug a program, then almost by definition, something does not work the way you thought it did. And just like when studying another subject, our brains work very hard to get rid of conflicting information once we’ve found a solution. In fact, try this right now: can you remember a bug you fixed from over a month ago, what was wrong, and what the solution was? If you’re anything like me, and indeed like most people I talk to, you can probably only remember a few vague bugs, and you probably can’t remember the root cause or the solution unless you were working on it for a long time.

This is a shame, because bugs often have a lot to teach us! Most of the time ¹, bugs are caused by a failure of process. There is often some concrete action that you could have taken which would either have helped you avoid the bug or to have recognized it much more quickly. And since the ability to extract information from our errors and use them to improve how we do things is perhaps one of the most important skills any knowledge field, our tip for today is this:

If you’re really dedicated, you can do a longer review later and write everything down. But taking just 2-3 minutes just after you’ve fixed the bug, while everything is fresh in your mind, can often generate a lot of useful insights. It’s also important that you do this very soon after you finish fixing things: if you wait even 10 extra minutes, your brain will begin the lovely process of flushing inconvenient extra information from your memory.

Questions to Ask

#

What should you go over in these analyses? It’ll depend on the bug and what type of project you’re working on, but I recommend trying to answer at least three questions:

What was the root cause of this bug?

#

Whenever a bug happens, it’s because a mismatch of how you think something works and how it actually works. So what was the mismatch in the bug you just solved? Try to get the most fundamental answer: for example, in C, a common error is to forget to terminate character arrays with a null character. But the root cause of this is not “I didn’t type '\0' correctly”: it’s either not realizing that you were working with a string in the first place, or forgetting that strings in C must be null-terminated.

The closer you can get to the root cause, the better chance you have of avoiding the bugs next time (and also, the easier it’ll be to answer the next two questions!) One technique often used for this is the Five whys. To use the five whys, you keep asking “why did this happen?” in regards to the previous answer until you reach the root cause of the problem. The technique is so named because in many engineering situations (which the method was originally designed for), it takes five iterations of asking “why” to get to the root cause. However, I find that for many common programming mistakes, 2-3 “why"s is usually enough to get to the root cause.

For example, suppose our program crashes with a NullPointerException. Our analysis might look something like this:

We could extend this analysis further with more questions like “why did the library fail to read the file?” (bad filepath) or “why did we not guard against the fact that the library could return null?” (we didn’t realize that it could happen), but with just this analysis, we have a lot of information about why things went wrong on multiple levels.

Is there anything I could have done to avoid this bug in the first place?

#

The best, fastest, and most reliable way of debugging a problem is not to have the bug in the first place. If the problem with a bug was a conceptual misunderstanding, there’s not much you can do except try to remember the issue next time.

However, many bugs are preventable, even with limited knowledge. For example, if you have a bug because your variable wasn’t initialized, you can easily solve this: initialize your variables at declaration with an invalid value (e.g. -1 for the length of a list). If your problem was that you did something like this:

if (var1 == var2)
  var1 += 1;
  var2 += 1; // Oops, this statement isn't part of the if-statement!

you can solve this by always using braces, even for one-line statements. That way, if you need to expand the if-statement later, you can’t forget to add the braces since they’re already there!

One trick I developed was in this category, after spending four hours debugging a nasty k-d tree error that turned out to be caused by copy-pasting code for a different tree then forgetting to modify it. I decided that to avoid this, from now on, whenever I copy-pasted code, I would intentionally break it by deleting variable names so that the code would not run unless I remembered to change it. This trick has saved me many hours of debugging in the time since I decided to start doing it, but I would not have come up with it if I hadn’t stopped to think about the pattern of bugs I was making at the time.

Is there any way I could have recognized this bug more quickly?

#

When you first start these analyses, your answer will probably be “not really.” But as you start to see more and more bugs, you’ll start to recognize patterns in how they manifest. For example, most people who have a year’s experience in programming in any language will recognize that a program which spins forever without printing anything or crashing is probably stuck in an infinite loop.

However, there are many other patterns that you can learn to recognize. Sometimes, how wrong the output is provides clues as to what went wrong—for example, if you’re trying to sum a list but you’re always a little bit too low, you might be failing to add the last element. Sometimes, timing of the output can can be a clue: if you’re running a loop where you think each iteration should take the same amount of time, but the loop iterations seem to get slower and slower as the loop runs, this is a hint that the loop isn’t working the way you think it is! And sometimes, the absence of output can be a clue.

Think about the clues you chased while you were debugging, and try to see whether any of the things you saw could have led you to the final issue more quickly. Was there anything that you saw that made you think “well, that’s weird,” but was ultimately what clued you in to the real source? Would reading some critical part of the documentation have helped?

Ultimately, this is the hardest question to answer, and it’s very dependent on what type of code you’re writing, so it’s fine if your answer is “I don’t know” at first. But this is also a very valuable question to be able to answer, since it can potentially shave hours off of your debugging time in the future.

Final Note

#

Don’t spend too long on the any one analysis unless you want to. You should think about what went wrong and how you could have avoided it, but sometimes the answer is really “there’s not much I can do about that one.” If you get stuck trying to answer one of these questions for more than 30 seconds, just move on to the next one.

Don’t worry, there will always be another bug for you to practice on!