Types of Bugs and How to Find Them

Your first week on the job is a crucial one. You might be new at this, but they'll be watching you very closely; they'll want to find out (1) how fast you learn, and (2) how good you are at finding bugs.

So of course, it helps if you start off knowing what you're looking for. This post is a general catalog of what kind of bugs are out there, and how you can go about finding them. (As for what to do once you've caught one, that we'll go over in the next post.)

Before we get to that, though, remember: Most of the time, the question you're paid to answer isn't, "What could be better?" It's, "What isn't functioning as designed?" We touched on this earlier, but they want to know how functional the game is, not how fun.

Occasionally, they might ask for your feedback, and that's when you can express your opinions and make suggestions. Just be diplomatic about it; developers take a lot of pride in their work, and an unkind or poorly-chosen word runs the risk of offending them. It's important to maintain good relations with other departments and teams, so even if you think the game is terrible, be polite and constructive about it, and keep in mind which suggested improvements are realistic and which aren't.

But that's on those occasions when they specifically ask for feedback. At all other times, you're strictly there to tag things that don't work. "The final boss' fire attack does no damage" is a legitimate bug; "The final boss fight should be more exciting," while it might be true, is not a bug for a tester to write. Leads and project leads, who've built more of a personal rapport with the higher-ups (and, sometimes, with the developers themselves) can sometimes get away with writing up those kinds of suggestions, but testers shouldn't try it, at least not without asking first.

Visual bugs

You'll see a wide variety of animation bugs. These can range from the blatantly obvious, as when a character freezes in time or thrashes around at crazy angles, to the more subtle, such as when a character momentarily blinks in and out between animations, plays the wrong animation at the wrong time, or faces the wrong way. In games where characters lip sync to the dialogue, the lip movements can be off (sometimes a little, sometimes a lot; you may see characters not move their mouths at all, or lip sync to the wrong line).

Animation bugs aren't just limited to characters. Objects in the environment can exhibit animation issues too; think of a rockslide, where the boulders tumble down, roll to the end of their paths, and then magically reappear back at the top of the hill. Or where the rocks don't appear to tumble at all, but your character still takes damage from them. Think of a cycling animation, such as that of a fire or a waving flag, where the object reaches the end of its animation, then loops back to the wrong frame, creating an awkward skip. Examine each object carefully, and make sure it moves the way it's supposed to under all circumstances.

Another common way to cause animation bugs is to interrupt cutscenes at awkward times. When the cutscene ends, do the characters appear in their proper places? Are there any leftover props or characters who shouldn't still be there? Hit the skip button at various times — just before, during, and just after every major action in the scene — to make sure everything's kosher. Dying just at the moment a cutscene starts is another good way to trigger bugs, if you can pull it off.

Note that animation bugs are not the same as AI bugs, which deal more with NPCs' goals, and how they attempt to achieve them, then which of their scripted animations they display while doing so. AI bugs are covered under "Gameplay bugs," further down below.

Closely related to animation bugs are clipping, or collision-related, bugs. These come in several flavors, ranging from mild to severe.

A game with customizable costumes, hair, or facial structures might have issues with one piece of equipment awkwardly overlapping another. Hats/helmets and hair are notorious for that, especially when the player can modify the size and shape of the character's head or body. Try all possible combinations, and see just how messy things can get.

Also on the subject of equipment, games with elaborate character animations don't always take the character's gear into account. If a character puts a hand to his chin, only to have the hand pass right through his armor on the way, that's a bug. Likewise, characters of a non-human size or shape tend to run into similar issues; a character with big arms and legs is more likely than most to put a fist through his own knee or some such.

Of course, characters can clip through not only their own bodies and gear, but also through pieces of the environment (and in this way, collision bugs are both visual issues and gameplay issues). Have any movable characters — either playable characters or pushable/escortable NPCs, along with any moveable objects — press up against all sides of every obstacle in every area. Chances are you'll find at least some combination that results in solid objects passing through one another.

The reverse is also true; sometimes invisible walls will stop characters' movement for no good reason. (Some invisible walls are there by design, but depending on the title, you may encounter many that are there by accident.) If these invisible walls are reasonably close to nearby obstacles, that's one thing, but an invisible obstacle in the middle of nowhere is something else entirely, especially if you're in a game where hitting a wall is dangerous.

Now, because of the various ways in which character models and environments are built, not every collision bug is easily fixable. If you find that your character's hand passes through the tip of a tree branch when he faces a certain angle, ask your lead whether or not your particular office considers such things worth bugging. (You'll hear the phrase "ask your lead" quite a bit in the posts ahead, for three reasons: One, they generally have the most comprehensive knowledge of the game, two, they'll usually have encountered similar questions in the past, and know the best answers, and three, individual offices will vary in policy when it comes to what is and isn't buggable, and at which stages in the project.)

But some collision bugs are big enough to be written up wherever you are. Ever played a game where you fell through a floor, or walked through a wall, or even rose through a ceiling, and ended up in some single-color void? Those are sometimes called out-of-world clipping bugs, and they should be found and fixed whenever possible. They're especially prevalent in games where characters have a quick "dash" or "roll" maneuver; try all such maneuvers against all obstacles, and see if it gets you anywhere you shouldn't be.

A closely related type of collision bug is one where an obstacle should prevent the player from advancing, but doesn't. If a character is supposed to chop down an obstacle, defeat all enemies around the obstacle, find a key, or whatever, it defeats the whole point if he or she can find a way to just pass right through it.

There are other types of graphical issues as well. Common bugs include characters and objects becoming invisible (particularly common if you move past them, then backtrack through the level and look at them again — and more likely to happen if you die and resume/reload/respawn somewhere along the way), flashes of visual corruption (usually in the form of a multicolored mess, sometimes accompanied by debug text; occurs most often during scene transitions), and issues with particle effects (any visual effect made up of smaller effects, such as sparks, smoke, water sprays, or magic) not appearing or behaving correctly.

Depending on the game type, you may also see many issues with the camera. We've all seen camera bugs before; the most common ones are the times your view gets obstructed behind a large, opaque piece of terrain. It's also possible for the camera to get stuck, or to zoom inside a character's head and display his or her face from the inside. Sometimes you can put the camera through a wall and send it out of the game world, revealing the empty space outside the level geometry.

In principle, finding camera bugs is fairly simple. In games with a user-controlled camera, just go everywhere, and have the camera point in every possible direction from every possible distance. Of course, in practice, that's usually a lot of three-dimensional space to cover, and sweeping an area thoroughly for camera bugs can take time. If a game includes events that suspend camera control, make sure to place the camera everywhere possible before those events occur, trigger those events at awkward times, and then — if possible — immediately trigger them again.

Games with fixed cameras can exhibit camera bugs too. For example, if the game cuts to a top-down view of your character's body when they die, have your character die under a low ceiling and see what happens. If a fixed-camera game includes a multiplayer mode, have all players run in all directions at all times and see if the camera can either keep up or restrict them properly.

Almost any game you'll see will include at least a few text bugs, and there'll be a whole separate section on those. Misspellings, incorrect punctuation, and poor grammar often abound through the scripts of early builds — especially if the developers aren't native English speakers — and fixing them is usually up to you. Just make sure you know how to fix them properly; it does no good to replace one mistake with another.

Keep an eye out also for text that overruns its onscreen borders. Characters' lines can overrun their dialog boxes, and bits of info on status screens can overlap or run outside their windows. If you can catch and bug all such issues, your game's text will look clean and professional, and you'll look good for having been the one to do it.

One other note on the subject of text: In games that include subtitled, voiced-over dialogue, you'll see sometimes that the text and the voiceover won't match. When this happens, the voiceover always wins, and the text should be changed to match it. Why? Because changing a line of text is as simple as opening up a file and typing in a correction. Changing a voiceover requires renting a studio, hiring a director, hiring an editor, scheduling the actor for another session — that's if they're available — re-recording the line, processing the recording, and then adding it back into the game.

(There are very occasional exceptions to the "voiceover always wins" rule, such as if a voiced-over tutorial gives the player incorrect instructions, or calls another character by the wrong name. In such cases, they may simply end up cutting the voiced-over line rather than re-recording it.)

Audio bugs

It goes without saying that quality sound is nearly as important to the game experience as the visuals are. To begin with, test all the sound settings on all the available TV types and sound setups you can. If the game is supposed to support surround sound, or anything higher than stereo, make sure you test that too, both with proper and improper equipment. And don't neglect mono; if possible, make sure you play through the game repeatedly in mono just to make sure nothing gets lost or garbled.

If you hear any sort of screeching noise at any point, or if the game goes silent when it's not supposed to, those are obviously bugs. But there are subtler types of audio bugs too. Listen to hear if anyone's voiceovers get cut off inappropriately, or if two of a character's voiceovers play at the same time. Make sure the right voiceovers play at the right time; simply put, characters shouldn't cheer when they lose. And make sure each voiceover comes from the correct character; if an NPC talks in a high-pitched sing-song voice for one line, then switches to a low southern drawl for the next, you've got a bug on your hands.

Of course, the subject of audio bugs covers more than just voiceovers. Make sure each sound effect plays consistently, and on cue. Make sure that triggering multiple sound effects and voiceovers at the same time doesn't interfere with the music — and on the subject of music, make it change back and forth between tracks as much as possible and see if that causes any issues.

How many variations on your average sound effect does the game include? Do footsteps always make the same sound, or does it depend on which surface you're walking on? If it's the latter, walk across all surfaces and make sure that each footstep on each surface sounds appropriate. Are there different sound effects for individual gunshots, explosions, or menu clicks? Make sure to trigger them all, under all possible circumstances, and see if any one of those variations causes any issues.

When it comes to checking for audio bugs, the key word is noise. Make as much as you can, and see what comes of it.

Gameplay bugs

For QA purposes, "Gameplay" refers to bugs that not only look or sound bad, but get in the way of the user's progress (or speed it along unintentionally).

The most obvious types of gameplay bugs are the crash and its close cousin, the hang. (The difference between the two is that a crash outright freezes everything, including the music and animations, while a hang might let the game continue on visually, but prevent any further interaction — i.e., you back out of the character menu, your character keeps standing there, the music keeps playing, and the next menu never loads.)

Crashes and hangs have all kinds of causes, some of them difficult to pin down. The cause can be as simple as one incorrect line of code; a line of dialogue might include an incorrectly-written command to call a variable, for example, or a script might refer to a voiceover file by the wrong filename. In early builds (versions of a game in test), you'll often find that some parts of the game just aren't finished yet, and if you play ahead of where the developers have gotten, you'll crash that way. (Usually, the developers will send notes, referred to as build notes, on which parts of the game are and aren't ready yet. You can ask your lead if there are any build notes for your current version of the game.)

Simple crashes are usually easy to trigger. Just get to that one part, trigger that one event, access that one menu option, or what have you, and if the game crashes there every time, you've got yourself enough to go on. But many crashes have more complicated steps, and require a combination of circumstances in order to make them occur. Get to that one part — with that one character. Trigger that one event — before you turn in that other quest. Access that menu option — after you access those five other menu options, in this specific order. And so on. Crashes can be so elaborate that it gets very difficult to pin down their causes, and a very few crash bugs may have no repeatable causes at all; in some builds, particularly early ones, just playing the game long enough can make it crash, but "long enough" might range from an hour to all day and then some, with no consistency as to when or where.

In situations like that, when you're dealing with tough-to-reproduce crash bugs, a good idea is to compare each instance of the crash bug you and the other team members have found, find out what they have in common, and use that as a basis for further experiments. (There'll be more on in-depth bug research in the next post.)

Crashes are definitely some of the most severe bugs, but a bug doesn't have to a game-breaker to be considered serious. Enough minor and moderate bugs can ruin a gamer's experience as easily as a crash can.

Think of how frustrating control bugs can be. Just try to remember how many times you've thought (or screamed) "I was blocking!" or "I pushed jump!" or "I didn't click there!" When you're testing, click there. Click everywhere. Try all combinations of buttons under all circumstances, and you'll find at least a few things wrong. Especially in multiplayer. Especially if you start making a conscious effort to stress the game, like by swapping controllers in and out of various ports willy-nilly. (Granted, you can make a case that someone who tries that hard to break a game deserves to see it broken — but still, bug it up.)

Speaking of stressing the game, make sure you check for framerate issues. We've all seen games slow down; not as much these days, but it does still happen. See how busy you can cause the game to get — a favorite method is by provoking all kinds of enemies, then having all of them chase you at once, on-camera — and try to make the framerate drop. (If so, how badly does it drop? Mention that in the bug report.) Framerate drops can be a minor annoyance, but they can also be killers if they happen in the middle of a crucial mission.

Which might remind you of that one escort mission. The one where you almost made it to the end, and then the guy you were escorting went all Leeroy Jenkins on the enemy horde and got himself killed. And then you reloaded, and he decided to press his head up against your machine gun while you were firing. And then you reloaded again, and he just wouldn't move at all.

Yeah, AI is a fragile thing in most games, easy to break or exploit. Test the limits of both friendly and enemy NPC intelligence to see how many ways you can fool them. Position NPCs in front of obstacle courses, and see if they can navigate through them. Provoke friendly NPCs by attacking them, and see if they respond appropriately. (In some games, it might be okay if they stand there and die, but nothing out-and-out strange should happen.) Damage dead bodies, including your own, and see if they get back up to die again.

Remember, you're looking for bugs that make the game too easy as well as bugs that make it too hard. Do enemies frequently blow themselves up with their own explosives? Do they waste all their ammo trying to shoot through walls? Do they stop chasing you, and stand still, if you get far enough away? Bug it all.

Speaking of difficulty, another key aspect of any game is balance. Depending on the project and policy, you may or may not have much say in how difficult or fair the game is, but if you do, use your voice to its fullest. Is a level significantly easier than the one before it? Are the final boss' attacks unavoidable? Does some random NPC have more hit points than the final boss? If you're allowed to — and check with your lead first — then bug it.

Closely related to balance is the possibility of exploits; ways to work the system and win easily. Multiplayer veterans know this category well; they've seen players fight dirty by abusing holes in the level geometry, tricking the game into making them invincible, using weapons in ways they were never designed to be used, and so on. Some exploits require quick combinations of button presses, while some are more about positioning your character in awkward, out-of-the-way spots or playing with the menu interface (think of every item duplication trick from every old RPG).

Sometimes, exploits come not from pulling improbable stunts in-game, but from oversights in the design. Is that talisman much more powerful than the devs intended? Can you find infinite recharges for your one-shot superweapon? Can a certain character's special ability combine with other spells or items to make the character invincible? Bug it.

In games with modular characters, make sure you try every combination of character stats, abilities, and equipment. If any one combination trounces all other combinations, especially in multiplayer, bug that too.

And while you're checking the stats and equipment menu, go over the game's interface with a fine tooth comb. Make sure you can get from any menu to any other menu at any time without any incidents; common bugs of this type include menus refusing to disappear, popping up at inappropriate times, or getting the player stuck. Feel free to mash buttons on the menus while you're at it; that can cause problems too.

And before we get off the subject of statistics, that's a whole category of bugs right there. Make sure each stat tracks the player's gains and losses accurately, no matter how high or low the numbers go. If you max a character's stats out — and you know some players will take the time to do that — do they stay maxed out, or do they roll back to 0? What happens if you try to buy an item you already have too many of, or if you try to buy something more expensive than you can afford? If there's more than one display for a given stat — say, if multiple pages of the status menu display the player's gold — are all those displays on the same page at all times?

While you're checking continuity in the menus, make sure you check it elsewhere too. Does the character's equipment stay the same (barring instances when there's a reason for the change)? Do the villagers' lines still make sense after each story event? Watch the dialogue closely; do characters ever refer to events they have no way of knowing about, or describe events incorrectly? If you switch costumes in one scene, does anything force the costume to change when it shouldn't? Do dead characters ever reappear later? We'll talk more about continuity as part of a later post.

When all else fails, just go crazy. Swap controllers at random. Mash the menu buttons — either the in-game ones or the console menu button — at the most inappropriate times you can think of; during cutscenes, loading screens, while initially booting the game, or loading a save file. Name your multiplayer character an inappropriate name, and see if the game catches it. Use an unorthodox controller; fight the final boss with a dance mat or a set of drums. Switch the options menu around, save it, reload an old save, switch them back, then switch them again. Run all the way to the end of a level, then run all the way back just because you can. Take the time to level up until you beat the impossible boss. Whatever.

All that should give you some idea of what's out there to find, and some of the ways you can find it. Up next: How to write a bug once you've caught one.