T
he eighteenth-century mathematician and physicist Marquis Laplace postulated that if there was an intelligence with knowledge of the position, direction, and velocity of every particle of the universe, this intelligence would be able to predict by means of a single formula every detail of the total future as well as of the total past [ReeseSO]. This is determinism.Chaos theory, Heisenberg's uncertainty principle, and genuine randomness in quantum physics have combined to prove determinism wrong. However, in the sim-plified universe of a game, Laplace's determinism actually works. .
If you carefully record everything that can affect the direction of your game uni-verse, you can replay your record and recreate what happened.
What Exactly Is Input Recording Useful For?
Game input recording is useful for more things than many people realize: reproduc-ing rare bugs, replayreproduc-ing interestreproduc-ing games, measurreproduc-ing optimizations, or creatreproduc-ing game movies.
Reproducing Bugs
Computer programs are deterministic and completely predictable, yet we frequently hear about people encountering bugs that are difficult to reproduce, and therefore dif-ficult to fix. If computers are deterministic, how can bugs be difdif-ficult to reproduce?
Occasionally, the culprit is the hardware or OS. The timing of thread switching and the hard drive is not completely consistent, so race conditions in your code can lead to rare crashes. However, the rare crashes are most frequently caused by a partic-ular combination of user input that happens to be very rare. In that case, the bug is at least theoretically reproducible, if only we can reproduce the exact input sequence again.
Videotaping of testing helps track some of these bugs, but it doesn't help at all if the timing is critical. Why don't we put that computer predictability to work, by hav-ing the computer program record all input and play it back on demand?
105
The crucial step here is that if we are to use input recording to track bugs, we have to make sure that the input is recorded even when the game crashes—especially when the game crashes! On Win32, this is typically quite easy. By setting up a structured exception handler [Dawson99], we can arrange for our input buffer to be saved when-ever the game crashes.
If we add an option to our game engine to "fast-forward" through game input (only rendering a fraction of the frames), we can get to the crash more quickly. If we also add an option to render frames at the exact same points in the game update loop, we can easily reproduce what are likely to be the relevant portions of the crash scenario.
Reproducing bugs is the one time when you will want to record and playback all of the user input, including the interactions with menu screens. Menu code is not immune to tricky bugs.
Replaying Interesting Games
The most common use of game input recording is for players to record interesting games. These recordings are used to demonstrate how to play the game, to make tuto-rials, to test the performance of new computer hardware, or to share games.
The most important thing about recording games for users to play back later is that the recording must always be enabled. It is unrealistic to expect users to decide at the beginning of the game whether they want to record the game for posterity; they should be asked at the end whether they want to permanently store the recorded game.
Measuring Optimizations
The most important thing to do when optimizing is to measure the actual perfor-mance, both before and after. Failing to do this leads to a surprisingly frequent ten-dency to check in "optimizations" that actually slow the code.
Measuring the performance of a game is tricky because it varies so much. Polygon count, texture set, overdraw, search path complexity, and the number of objects in the scene all affect the frame rate. Timing any single frame is meaningless, and a quick run-through is hopelessly unscientific.
Game input playback is a great solution. If you run the same playback multiple times, recording detailed information about game performance, you can chart your progress after each change to see how you're doing and where you still need work.
Recording the average and worst-case frame rate and the consistency of the frame rate becomes easier and much more meaningful.
Testing optimizations with game input playback doesn't always work because your changes might affect the behavior—your wonderful new frame rate might just mean the player has walked into a closet. Therefore, when using game input playback for optimization testing, it is crucial that you record critical game state and check for changes on playback.
Creating Game Movies
To create a demo reel, you can hook a VCR up to a video capable graphics card and play the game; however, the results will not be pretty. The VCR, the video encoder, and the variable frame rate of the game will lead to a blurry, jerky mess.
With game input recording, it's trivial to record an interesting game, and then play it back. With some trivial modifications to the engine you will be able to tell the game engine when you are at an interesting part of the playback, at which point you can switch from real-time playback to movie record playback. In this mode, the engine can render precisely 60 frames for each second of game play, and record each one to disk. The frame rate may drop to an abysmal two frames per second, but it doesn't matter because the canned inputs will play back perfectly.
Implementing Multiplayer
A number of games—X-Wing vs. TIE Fighter, and Age of Empires—have used input recording and playback for their networking model [Lincroft99]. Instead of transmit-ting player status information, they just transmit player input. This works particularly well for strategy games with thousands of units.
What Does ItJTake?
Game input recording is simple in theory, and can be simple in practice as well. How-ever, there are a few subtleties that can cause problems if you're not careful.
Making Your Game Predictable
For game input recording and playback to work, your game must be predictable. In other words, your game must not be affected by anything unpredictable or unknow-able. For example, if your game can be affected by the exact timing of task switching, then your game is unpredictable.
Many games use variably interleaved update and render loops. Input is recorded at a set frequency. A frame is rendered and then the game update loop runs as many times as necessary to process the accumulated set of inputs.
This model implies that the number of times that the game update loop is run for each frame rendered is unpredictable; however, this needn't make the game itself unpredictable. If you are tracking down a bug in the Tenderer, then you may need to know the exact details of how the render loop and update loop were interleaved, but the rest of the time it should be irrelevant. It is worthwhile to record how many updates happened for each frame, but this information can be ignored on playback unless you are tracking a Tenderer bug.
However, if the render function does anything to change the state of the game, then the variably interleaved update loop and render function do make the game unpredictable, and input recording will not work. One example of this is a render function that uses the same random number generator as the update loop. Another
example can be found in Total Annihilation. In this game, the "fog of war" was only updated when the scene was rendered. This was a reasonable optimization because it reduced the frequency of this expensive operation. While it ensured that the user only ever saw accurate fog, it made the game's behavior unpredictable. The unit AI used the same fog of war as the Tenderer; the timing of the render function calls would sub-tly affect the course of the game.
Another example of something that can make a game unpredictable is uninitial-ized local variables or functions that don't always return results. Either way, your game's behavior will depend on whatever happened to be on the stack. These are bugs in your code, so you already have a good reason to track them down.
One tricky problem that can lead to unpredictability is sound playback. This can cause problems because the sound hardware handles them asynchronously. Tiny vari-ances in the sound hardware can make a sound effect occasionally end a bit later. Even if the variation is tiny, if it happens to fall on the cusp between two frames, then it can affect your game's behavior if it is waiting for the sound to end.
For many games, this is not a problem because there is no synchronization of the game to the end of these sounds. If you do want this synchronization, then there is a fairly effective solution: approximation. When you start your sound effect, calculate how long the sample will play—number of samples divided by frequency. Then, instead of waiting for the sound to end, wait until the specified amount of time has elapsed. The results will be virtually identical and they will be perfectly consistent.
Initial State
You also need to make sure that your game starts in a known state, whether starting a new game or loading a saved one. That usually happens automatically. However, each time you recompile or change your data you are slightly changing the initial state.
Luckily, many changes to code and data don't affect the way the game will unfold. For instance, if you change the size of a texture, then the frame rate may change, but the behavior should not—as long as the game is predictable. If changing the size of that texture causes all other memory blocks to be allocated at different locations, then this should also have no effect—as long as your code doesn't have any memory overwrite bugs.
An example of a code or data change that could affect how your game behaves would be changing the initial position of a creature or wall, or slightly adjusting the probability of a certain event. Small changes might never make a difference, but they destroy the guarantee of predictability.
Floating-point calculations are one area where your results may unexpectedly vary. When you compile an optimized build, the compiler may generate code that gives slightly different results from the unoptimized build—and occasionally, these differences will matter. You can use the "Improve Float Consistency" optimizer set-ting in Visual C++ to minimize these problems, but floaset-ting-point variations are an unavoidable problem that you just have to watch for.
Random Numbers
Random numbers can be used in a deterministic game, but there are a few caveats.
The reason random numbers can be used is that rand() isn't really random. rand() is implemented using a simple algorithm—typically a linear congruential method—
that passes many of the tests for random numbers while being completely repro-ducible. This is called a pseudo-random number generator. As long as you initialize rand() with a consistent seed, you will get consistent results. If having the randomness in your game different each time is important, then choose a seed for srandQ based on the time, but record the seed so that you can reuse it if you need to reproduce the game.
One problem with rand() is that it produces a single stream of random numbers.
If your rendering code and your game update code are both using rand()—and if the number of frames rendered per game update varies—then the state of the random number generator will quickly become indeterminate. Therefore, it is important that your game update loop and your Tenderer get their random numbers from different locations.
Another problem with rand() is that its behavior isn't portable. That is, the behav-ior is not guaranteed to be identical on all platforms, and it is unlikely that it will be.
The third problem with rand() comes if you save a game and continue playing, and then want to reload the saved game and replay the future inputs. To make this work predictably, you have to put the random number generator back to the state it was in when you saved the game. The trouble is, there's no way to do this. The C and C++ standards say nothing about the relationship between the numbers coming out of rand() and the number you need to send to srand() to put it back to that state.
Visual C++, for instance, maintains a 32-bit random number internally, but only returns 15 of those bits through rand(), making it impossible to reseed.
These three problems lead to an inescapable conclusion: don't use rand(). Instead, create random number objects that are portable and restartable. You can have one for your render loop, and one for your game update loop.
When implementing your random number objects, please don't invent your own random number algorithm. Random number generators are very subtle and you are unlikely to invent a good one on your own. Look at your C runtime source code, the sample code on the CD, Web resources [Coddington], or read Knuth [KnuthSl].
Inputs
Once you have restored your game's initial state, you need to make sure that you can record and play back all of the input that will affect your game. If your game update loop is calling OS functions directly to get user input—such as calling the Win32 function GetKeyState(VK_SHIFT) to find out when the Shift key is down—then it will be very hard to do this. Instead, all input needs to go through an input system.
This system can record the state of all of the input devices at the beginning of each frame, and hand out this information as requested by the game update loop. The
input system can easily record this information to disk, or read it back from disk, without the rest of the game knowing. The input system can read data from Direct-Input, a saved game, the network, or a WindowProc, without the update loop know-ing the difference. As a nice bonus, isolatknow-ing the game input in one place makes your game code cleaner and more portable.
Programmers have a habit of breaking all rules that are not explicitly enforced, so you need to prevent them from calling OS input functions directly. You can use the following technique to prevent programmers from accidentally using "off-limits"
functions.
#define GetKeyState Please do not use this function
tfdefine GetAsyncKeyState Please do not use this function either
Another important input to a multiplayer game is the network. If you want to be able to replay your game, dien you need to record the incoming network data together with the user's input stream. This will allow you to replay the game, even without a network connection. The network data stream is the one type of data that can actually get quite large—a game running on a 56K modem could easily receive many megabytes of network data per hour. While this large data stream does make the recording more unwieldy, it is not big enough to be really problematic. The ben-efits of recording this stream are enormous, and the costs are quite small.
The final "input" that a game might use is time. You may want certain events to happen at a specific time, and it is important that these times are measured in game time, not in real time. Whenever your game needs to know the time—except for pro-filing purposes—it should ask the game engine for the current game time. As with the other input functions, it is a good idea to use the preprocessor to make sure that nobody accidentally writes code that calls timeGetTimeO or other OS time functions.
It is a good idea to record inputs throughout the game. That lets you use input playback to track down bugs anywhere in the game, even in the pre-game menus.
However, for many purposes you will want to store the record of the input during the game separately, so that you can play it back separately.
Testing Your Input Recording
Game input recording should work on any well-written game. Even if your game is a multiplayer game, if you record every piece of input that you receive on your machine, then you should be able to reproduce the same game.
However, if your game playbacks are failing to give consistent results, it can be difficult to determine why. A useful option in tracking down these problems is record-ing part of the game state along with die input—perhaps the health and location of all of the game entities. Then, during playback, you can check for changes and detect differences before they become visible.
Conclusion
Game input recording and playback is a valuable part of a game engine with many benefits. If it is planned from the beginning, then it is easy to add, and leads to a better-engineered and more flexible game engine. Here are some rules to follow:
• Route all game input, including keyboard, mouse, joystick, network, and time, through a single input system, to ensure consistency and to allow recording and saving of all input. This input should always be recorded. It should be stored per-manently in case the game crashes or the user requests it at the end of the game.
• Watch for floating-point optimizations or bugs in your code that can occasionally lead to behavior that is different or unpredictable in optimized builds.
• The randQ function should be avoided; use random number objects instead.
• Never change the game's state in rendering functions.
• Store some of your game state along with the input so you can automatically detect inconsistencies. This can help detect race conditions, unintended code changes, or bugs.
The sample code on the CD includes an imput system and a random number class.
References
[ReeseSO] Reese, W.L., Dictionary of Philosophy and Religion. Humanities Press, Inc.
1980. p. 127.
[KnuthSl] Knuth, Donald, The Art of Computer Programming, Second Edition, Vol-ume 2, SeminVol-umerical Algorithms.
[Coddington] Coddington, Paul, "Random Number Generators," available online at www.npac.syr.edu/users/paulc/lectures/montecarlo/node98.html.
[Dawson99] Dawson, Bruce, "Structured Exception Handling," Game Developer magazine (Jan 1999): pp. 52-54.
[Lincroft99] Lincroft, Peter, "The Internet Sucks: What I Learned Coding X-Wing vs. TIE Fighter," 1999 Game Developers Conference Proceedings, Miller Free-man 621-630.