Feedback In Game Design

This is a guest blog by Chris Hazard of Hazardous Software Inc., the makers of the upcoming time travel RTS, Achron.

Feedback is a part of every day life. It occurs when the output of something is put back in as input.

Positive Feedback

Imagine that you're going to a local sporting event on the weekend, and because of a parade you decide to leave extra early. By doing so, you've avoided the parade and gotten a better parking spot, which allows you and your friends to get out of the parking lot right after the game concludes and you go to a restaurant afterward and there's no wait. Alternatively, imagine you're running terribly late, you call your boss on the way to the airport because you forgot to bring some important documents to a big meeting and then get into an accident, paying more attention to your call than the road. You've now missed your flight, forgotten the documents, and now your boss may rethink that promotion he was about to give you.

Feedback at its finest. Yes, this was a true Tyson ad campaign.

Both of these examples demonstrate positive feedback. Positive feedback occurs whenever something, good or bad, is amplified. Negative feedback occurs when something is dampened. Examples of negative feedback are the wind resistance on a car increasing as you go faster, requiring the engine to work harder to increase your speed, or progressive taxation, where you're taxed at a higher percentage when you make more money.

Positive and negative feedback are both very important in games too. Positive feedback, when done right, quickly separates player by their skills and strategies chosen. A good player may obtain the best items and use them to quickly defeat a less-skilled player who will not have obtained good items. If positive feedback is too strong, however, the results may be random. Imagine a first person shooter where you start off with BB guns and there is a single rocket launcher on the level (with infinite ammo, of course). If a good player makes one mistake, a less-skilled player may still dominate the game.

Negative Feedback

Negative feedback is also important in game design. Think of the racing games you may have played. If you've noticed that the computer players tend to perform better when you're winning, this is a typical example of negative feedback. This particular scenario is often referred to as the "elastic band"; imagine a rubber band keeping the computer players with a certain distance of the lead. When negative feedback is done right, the player doesn't really notice and the game remains engaging and competitive.

When negative feedback is overdone, it can have many bad consequences. Too much negative feedback can prolong an inevitable loss and frustrate good players by punishing their successes. If the negative feedback is too strong, it can even yield perverse incentives where good players may intentionally play poorly at the start of the game in order to gain a long-term advantage.

Positive Feedback In Repeated Rounds of Rock, Paper, Scissor

Keeping feedback in mind when designing a game is important, but it doesn't end there. We can actually measure the effects of feedback! Let's start with a simple rock-paper-scissors game, where players repeatedly play against each other. Each round, players always start over, so there aren't really any dynamics to speak of from a game theoretic perspective (unless we want to model both players modeling each other as non-perfect random number generators, dealing with Bayes-Nash equilibria, but that's a topic for another discussion).

Now let's make the game a little more interesting. Suppose that each game is now a series of rounds of rock-paper-scissors that continues until one player has won 4 more rounds than the other. When a player wins a round, the player gets a powerup that is used during a draw (when both players play the same move). Whenever a draw occurs, the player with the most powerups wins that round. Powerups are not lost.

What does this game look like dynamically? Let's first take a look at it in a spreadsheet.

The horizontal axis is each of the game states from player 1's perspective based on the number of points advantage it has over player 2. Here we're looking at the relative strengths of the players, so if both are tied, they still remain in the center. To read the graph, first look at the current game state, then look up to see the probability that the next game state will be, as represented by the vertical axis. To begin the game, players start at 0,0 and have a 1/3 probability of each staying tied or either player having a one point advantage.

If we look at the expected value of the relative power of player 1 for each of the game states and smoothly connect the dots, it looks like the following figure.

The X axis represents the current state: how many more wins the player has than the opponent. The curve F(X) represents the expected state after this round finishes.

This figure is known as a Cobweb diagram. You start at a state on the horizontal axis, trace your finger vertically up to the point on the curve for that point on the horizontal axis, then trace your finger horizontally to the diagonal line, and then trace your finger vertically up or down to the curve again. The lower-left corner means a loss and the upper-right corner means a win. If you keep tracing your finger, you'll trace out the dynamics of the game. When you have real games that may have hundreds or thousands of discrete states and continuous positions, a continuous curve is much easier to visualize.

Games that have probabilistic results, like our rock-paper-scissor variation, do not yield such clean results either. This curve is the expected value of the game dynamics, and so a particular game may not follow it exactly; this game begins at a fixed point, but once a player wins or loses, it jostles away from the center. You could plot a colored region around the line to show all possibilities.

Fixed points are very interesting properties of feedback systems. A fixed point occurs whenever the output is exactly what is put in. The above figure has 3 fixed points, represented where the "next player relative power" function crosses the diagonal line. Examining these points can tell us how the system will behave.

By looking at the slope of the line at a fixed point (the derivative of the feedback function, or, in terms of feedback, the maximum component of the gradient with multiple variables), we can tell whether it is repelling or attracting. If the magnitude of the slope is less than 1, it is attracting, as can be seen on the two endpoints of winning and losing. If the magnitude of the slope is greater than 1, the fixed point will repel. If the slope of the fixed point is 1, it will neither attract nor repel. Fixed points that move the output toward the extremes yield positive feedback, whereas fixed points that move the output closer to the center yield negative feedback. Further, the sign of the slope tells whether the value will flip back and forth or stay on one side. This rock-paper-scissors variation, is dominated by positive feedback, as all 3 fixed points move the output toward the extremes, without flipping to another side of any fixed point.

Now, let's take a look at a game that isn't so orderly.

In this game, both of the fixed points are repelling. Even more, the fixed point in the upper right will cause the relative power to flip sides. This means that a player will build up power slowly and then lose it quickly. If the power is influential to the player winning, then this can be a fun gameplay aspect. Instead, if a good player doesn't want to do any better because he/she knows that the power will be lost, a player may just sit there waiting for a particular opportunity to arise. An example of this is if a player lost all ammo and weapons if they picked up one clip too many. Mathematically speaking, game dynamics like this are chaotic, which may be a good or bad thing depending on the specific game and circumstance.

Obviously figuring out these graphs and dynamics for nontrivial games isn't nearly as easy as figuring it out for a simple rock-paper-scissors variation. The trick is to abstract away what relative power means, which can be a heuristic metric that combines player health, items, weapons, positioning, etc. The game's AI can be used to plot an approximation of this graph. The AI starts off with each possible relative power configuration (sampling and techniques such as design of experiment can be important here, especially if you have a lot of items), and plays until it has attained a new relative power. These data points can be combined visually to see an approximation of the feedback in your game. The approximation will be as accurate as the AI is at behaving like humans, so you need to take care when analyzing the actual data. This model may be extended to encompass player skill as well by having some or all of the probabilities be a function of player skill.

Thanks Chris for your awesome post. I had been aware of the basic concept of feedback but it's great to see how it can be analyzed in a quantitative framework.

If you haven't heard of Achron yet, I recommend checking it out. A lot of people have thought about adding time travel as a mechanic to games but I don't think anyone has gotten it right for a multiplayer RTS before. These guys are even going so far as to embrace the grandfather paradox as a feature. You can find out more on the Hazardous Software YouTube Channel.