What a Bug Needs

However many bugs you find, it's not enough just to find them; you have to report them well too. A bug report, sometimes just referred to as a bug, is a written step-by-step report of a glitch's location, nature, and reproduction process (no, not that reproduction process).

Writing bugs well is crucial for several reasons. For one, people outside your department — and some of the higher-ups within it — will see much more of your bugs than they do of you. To them, your bugs represent you personally, and establish your credibility — or, if they're bad bugs, lack thereof. They might say, "Oh, he just put another bug in. Pay attention; here's what he says..." or, if your bugs are written poorly, "Oh, another one of this guy's bugs. Just ignore it; he doesn't know how to test." By extension, your bugs also reflect on your teammates and your department as a whole, for better or worse.

More than that, a well-written bug will send developers directly to the problem and give them all the information they need, while a poorly-written bug will, at best, be met with more questions, and at worst, send them off in the wrong direction — either way, badly-written bugs waste time.

Before we go into the details of what separates a good bug from a poor one, let's go over the basic bug format. This'll vary from place to place, but here's the general idea.

Most places you go, bugs have three main parts: A title, a description, and a variety of drop-down menus for variables such as location and frequency. Since the variables will be completely different wherever you go (and even vary by project within each department), let's focus on the title and description here.

The title generally starts with the bug's category, followed by a colon, followed by a short summary of the issue. Example:

Collision - Out of World: User's character falls out of world upon dodge-rolling against a specific wall in Chapter 2-5.

(The term "user's character" is used in some places, but not all; others might refer to the player's avatar as "the hero," "the player" — which isn't technically accurate, but it's commonplace — or the character's specific name.)

The bug's description is a step-by-step process. Generally, the process begins by telling the reader where to find the issue, then how to trigger the issue, then what the issue is, and finally what should happen instead. Example:

1) Open Chapter 2-5.

2) Advance to the first museum exhibit and run up against the corner behind the suit of armor.

3) Dodge-roll into the corner at a 45-degree angle.

4) The user's character passes through the wall and falls out of world.

5) The user's character should never fall out of world.

Frequency 5/5.

(Some places, instead of using step numbers for Steps 4 and 5 above, preface them with the terms "Result:" and "Expected behavior:", respectively. Ask your lead or trainer how it works in your neck of the woods.)

Note the frequency above. This represents how many times you were able to make the bug happen, out of how many attempts, beyond the first time you encounter the bug. Some places will have you make five attempts to reproduce the bug, while others will be even more thorough and ask for ten. Not all places will have you put the frequency in the description — some have a drop-down menu for it instead — but wherever they want you to put it, don't leave it out.

Now, what you see above is a clear, solid, well-researched bug. It explains where the issue is, what the issue is, that the issue is specific to that area, how to trigger it within that area, and that it happens every time there.

Here's the wrong way to do it:

Say you're testing an action game, and you crash midway through the third level. The wrong thing to do is to stop testing, hop on the bug database, and enter this:

Crash: game crashes on level 3

1) Open level 3

2 Play game until it crashes

3) Game crasees.

4) Game must not crash

Before you read on, read through that again and count how many things are wrong with it.

The spelling and punctuation issues are obvious. That's important: Spelling counts. If your bugs are full of typos, it hurts your credibility; producers and developers will see your bugs and think, "He's supposed to find mistakes in the game, but he can't even find his own!" Most testers have access to some kind of spell check, so if you need it, use it. Make sure you use full sentences while you're at it.

Not only that, but look at the way the bug's worded. When you say the game "crashes," or any verb in the present tense, you're implying that it happens every time. This bug has only occurred once for the tester so far, so it's wrong to say that. (If you read carefully, you'll have seen another mistake: The bug is missing its frequency.)

And where in Level 3 does the crash happen? Do you have to do anything in particular to make it happen? Does it always happen in the same place, or does it vary from one attempt to the next? Has it ever not happened on this build?

Find out. Before you write it up. Look carefully at each bug you write, anticipate what questions people will ask about what you've written, and answer those questions within the bug itself so that they won't have to ask.

Say you play through Level 3 five more times, and you find out that the crash occurs whenever three or more enemies die at the same time. What's the next step? Write up the bug? "Crash: Game crashes if at least three enemies die simultaneously on Level 3"? No; don't write it up yet.

Can you guess what's left to do? What info you still need?

That's right; check to see if killing 3+ enemies on other levels causes the same crash. It might, or it might not; find out.

Let's say it does. Let's say it turns out that the crash happens anytime you kill three guys at once, regardless of which level you're on. The finished bug, then, would look like this:

Crash: Game crashes if at least three enemies die simultaneously.

1) Open any level.

2) Kill at least three enemies with one blow.

3) The game crashes.

4) Killing three or more enemies at once should not cause the game to crash.

Frequency 5/5

And you see that the finished bug has nothing to do with Level 3; that first version of the bug, above, would have sent the developers on an entirely wrong path. So make sure you find out everything there is to know about an issue before you write it up.

On a side note: Tell the developers how to trigger the bug, but avoid telling them how to fix it; they might take offense. Say you figure out that the reason the game's crashing in the bug above is because the combined visual effects for the three enemies' blood sprays are overtaxing the system. You can say:

3) The game crashes, possibly because the combined particle effects of the enemies' blood sprays are overtaxing the system.

But don't say:

4) The particle effects should be reduced so as not to overtax the system.

Give them all the information you have, but don't tell them what to do with it. (In cases where you believe you have a solution, but the developers don't seem to have found that solution themselves, you can bring it to your lead, who may then pass it along as a suggestion.)

There is, however, an exception to the "don't tell devs how to fix bugs" rule, and that's when it comes to text bugs. Text bugs are formatted a bit differently than most, in that they list both what the problematic line of text currently says and what it should say instead. Example:

Text: Misspelling in Cinematic 3's fourth subtitle.

1) Open Cinematic 3. [Or, if there's no way to jump directly into a cinematic, tell them to open the latest area before the cinematic, then advance to Cinematic 3 from there.]

2) Advance through the cinematic to the fourth subtitle.

3) The fourth subtitle currently reads:

You fools have fallen straight into my trap! And with you out of the way, there's no one left to stop me! I will rule the wold!

4) Change "wold" to "world", as below:

You fools have fallen straight into my trap! And with you out of the way, there's no one left to stop me! I will rule the world!

An effective text bug needs to do four things: Explain exactly where the mistake appears, quote the problematic line in its entirety, show the corrected line in its entirety, and highlight the correction (either by spelling it out, as Step 4 does above, or by bolding it, if your particular bug database allows that).

Since we're on the subject of what information a bug needs to convey, you might be wondering, "Is it possible for a bug to say too much?" The answer is, "Sort of; it's possible for a bug to have too many words."

Here's an example of an overly-wordy bug:

Graphics: The third column on the left side of the place where the ghost boss first spawns in in the second half of the mansion level gets flickery when the player sees it from the north side of the room.

1) Insert version 100203 of the "Battle Spirits" game into an Xbox 360 console.

2) Turn on the console, highlight "Battle Spirits" under "My Xbox," and press the A button to boot the title.

3) Press Start at the title screen.

4) Advance to the mansion level.

5) Defeat the hound boss in the courtyard, then go through the green door.

6) Take the upper path, turn the gold wheel clockwise one full turn, and then turn it counter-clockwise 180 degrees to open the gold door.

7) Advance to the ballroom and meet the ghost boss.

8) When the fight with the ghost boss begins, stand on the north side of the room, then turn around and look to the right.

9) Notice that, if the player stands on the north side of the room, then faces the third column to the left of where the ghost boss first spawned, the column gets flickery.

10) The third column to the left of where the ghost boss in the mansion level first spawns in should not be flickering.

Now, you probably get the idea, and this is an extreme example, but here's the point: A well-written bug conveys all needed information in the fewest possible words. This bug is three-quarters fluff. Let's go through it again, line by line, with comments in bold:

Graphics: The third column on the left side of the place where the ghost boss first spawns in in the second half of the mansion level gets flickery when the player sees it from the north side of the room.

Everything from "The third column" to "the mansion" could be shortened to, "The third column to the left of the mansion ghost boss' spawn point". Half the words, all the information. (The shortened version does omit the reference to the "second half" of the mansion level, but if there is just one ghost boss in the mansion — which, in this case, we'll say there is — then no need to specify that.) Likewise, the second half of the sentence could be shortened to, "flickers when viewed from the north."

1) Insert version 100203 of the "Battle Spirits" game into an Xbox 360 console.

2) Turn on the console, highlight "Battle Spirits" under "My Xbox," and press the A button to boot the title.

3) Press Start at the title screen.

You don't need to tell the developers, or other testers, how to turn the game on. These three steps can be cut entirely.

4) Advance to the mansion level.

Unless the bug specifically only happens if you play from the start of the game — which, for graphical bugs, is extremely rare — this step can be changed to "Open the mansion level." The difference is that "Advance to" implies the need to start from an earlier point in the game (usually the beginning, if there's no other context) while "Open this level" tells the reader that it's okay to use a debug function (usually a level-skipping function) to get there faster.

5) Defeat the hound boss in the courtyard, then go through the green door.

6) Take the upper path, turn the gold wheel clockwise one full turn, and then turn it counter-clockwise 180 degrees to open the gold door.

7) Advance to the ballroom and meet the ghost boss.

These three steps don't tell the reader anything except how to get to the ghost boss, which any tester or developer should already know. (A new tester might not know, but the bug database is not the proper venue for a walkthrough. That's up to the lead and the developers to provide — or, if they're busy, more experienced testers can help the newer ones in that regard.) Now, if the bug is conditional on doing something in a prior location, then mention that for sure. In this case, though, we can lose these three steps and not miss a thing.

8) When the fight with the ghost boss begins, stand on the north side of the room, then turn around and look to the right.

9) Notice that, if the player stands on the north side of the room, then faces the third column to the left of where the ghost boss first spawned, the column gets flickery.

There are three things wrong with these two steps. One, they're both too wordy. (General rule: You don't need to use the words "Notice that" or "Observe that." "Your shoelace is untied" conveys the same information as "Notice that your shoelace is untied," and does it more efficiently.) Two, the bug isn't consistent with its directions; it uses "north" sometimes, but then "turn around" and "left" at other times. Are we talking about the ghost's left? My left? Before or after I turn around? Three, Step 9 makes the mistake of combining the "how to trigger the bug" steps with the "what happens once you trigger it" step. The first step here should be a short summary of where to look, and at which angle, and the second should briefly describe where the flickering occurs.

10) The third column to the left of where the ghost boss in the mansion level first spawns in should not be flickering.

This one's borderline-acceptable as-is, but since you've already specified which column we're talking about, you can just say "This column should not flicker." Notice also that "flicker" conveys the same info as "be flickering," and more economically; keep that in mind for all present-tense verbs when you're writing bugs.

So the edited version of that bug would read:

Graphics: The third column to the west of the mansion ghost boss' spawn point flickers when viewed from the north.

1) Open the mansion level.

2) In the ballroom ghost boss' room, move to the north wall, then turn around and view the third west-side column.

3) This particular column flickers from this angle.

4) This column should not flicker.

We've gone from 204 words to 56, and haven't lost anything but fat. All of the necessary info, none of the clutter.

And remember, clutter can do more than slow the reader down; if you include steps that aren't necessary to trigger the bug, it can mislead them into thinking that those steps are necessary, and waste their time trying to figure out what it is about those steps that makes the bug happen.

We'll talk more about editing text later, particularly as it relates to other aspects of the job. For now, these few examples should give you a basic idea of how to conduct and organize your bug research. Keep your bugs clean, thorough, and to the point; your lead, your producer, and the developers will thank you.