What Makes a Good Bug Report?

One of the more complicated “easy” things in software is providing a useful bug report. Literally anyone can encounter a “bug,” “flaw” “defect,” or even “desired-feature” while using a product and anyone can give a good explanation to the maker of the software of that problem, but people rarely do. Sometimes they feel they’re not knowledgeable enough and don’t try, sometimes they try and don’t fully understand what’s helpful, and sometimes they just nail it.

The first thing to understand to write a good bug report is that even with no technical knowledge, you can probably provide the maker of a piece of software with useful information. As long as you pay attention and report the issue as fully as you can, your input should be welcomed.

Pretend you are an auto mechanic

5080569197_6b70c8d550_oImagine you’re an auto mechanic and a customer comes to you and says “My car doesn’t work.” From the start, they’ve given you very little to go on. “Car doesn’t work” can mean literally anything from “the air conditioner doesn’t get as cold as I’d like” to “the stupid thing won’t start no matter what I do it.” So you ask for details. What you need to know is what the person is seeing happen, what they expect to happen, and you might be helped by anything they know that has happened that could contribute to the gap between those two things.

Now because of a weird quirk of this universe in which you’re theoretically an auto mechanic, it’s actually the case that you can literally never see the car. Instead, what you can see are blueprints that are used to make the cars. And when a user presses a button, their cars based on the blueprint can almost magically be reconfigured based on the changes you make. As you’d imagine, this twist presents another hurdle: you’ll probably need to know a lot about the specific car and its environment to figure out the problem. Some behavior reports like “it’s doing X and should do Y” will be good enough for you to identify and fix the issue in the blueprint. You’ll look at the car’s plan and see clearly that the flange has been getting installed upside down in all cars.

But sometimes it’ll have taken the car being driven 5,000 miles on vibration-causing backroads for the issue to occur. The fact is that the lever was being installed a few degrees off plumb which made it vulnerable to getting trapped in an open state when the car experienced lots of vibration. As you’d imagine, for a problem like that it’d be really useful to know the environment in which the driver was using their car, how they were using it, and what other things were around their car.

Enough with the cars! How do I file a good bug report?

Ok, you probably understand the analogy well enough. So here are the three most important things that a good bug report will contain:

  1. Behaviors: In as much detail as you can, describe how the application you were using behaved and how that differed from your expectation. It’s possible that what seemed broken to you was exactly the way the maker intended for it to work. So being clear about what you expected and what you were seeing helps to quickly narrow the gap between the two.
  2. Steps to replicate: This can be captured in your description of behaviors, but the most important thing to a programmer trying to fix your problem is to be able to see it with their own eyes. In general, without a method to see exactly the behavior you experienced, all a programmer can do is stare at their code puzzled. Localizing the problem to a specific part of it is generally much easier when they can make it do what you saw it do.
  3. Your environment: Some things can be omitted, but almost everything you can describe about the system on which you were running a program when you encountered a problem is potentially relevant to the problem and its cause. Obviously for most users this can be a real gap — “I was using the internet” is all some people understand about their environment — but if you can name things like the browser, operating system, and others programs also running, it can be a massive help for solving obscure problems that are hard to replicate. Generally the course of action I use when reporting my environment when talking about a bug is to give all the information I consider possibly relevant, and let the receiver ask for more details if they need it.

These three things aren’t the only things that well-estabilshed organizations will ask for. Many of them have complex forms that specifically suit their piece of software and the type of bugs they get reported. But this is a great place to start to understand what kind of things are helpful to someone trying to make their software better with your input. So get reporting!

Image Credits: hulagway, Arne Hulstein

About David Hayes

David likes learning, solving hard problems, and teaching. He bikes a lot, and lives in (and loves) Colorado. You can find him on Twitter as @davidbhayes and check out his latest hobby-project, Quodid, a monument to his love for pithy bits of wisdom.

Leave a Reply

Your email address will not be published. Required fields are marked *