Adaptive difficulty

[ggambett]

In Betty's Beer Bar (http://www.mysterystudio.com/bettysbeerbar.php), the player can select one of three difficulty modes (Easy, Medium and Hard).

For people not familiar with the game, there's a bar, customers come in at certain rate, ask for a beer, get angry if you don't serve them, and they occasionally get drunk. Also, each level has a "sales goal" that must be reached or the player loses.

The difficulty settings control these variables in a very straightforward way - in Easy mode, customers come in more slowly, take longer to finish their beers, have a smaller probability of getting drunk, and the sales goal is lower than in Hard mode.

The game I'm working on has a similar gameplay. But this time, I don't want to include difficulty levels. The main reason is that difficulty levels are quite arbitrary, determined only from playtesting, and in any case I don't want customers frustrated by a game that is too easy or too hard.

So, I want to make the game difficulty adapt to the player instead. My general idea is to monitor some variables (time between a customer asks for a beer and it's served, for example) and combine the variables into a "skill" variable (S). Then, if the player performs between 0.90xS and 1.1xS, I keep the settings as they are. If the player falls below 0.90xS, I overestimated his skill, so I tweak the variables I control (how quick a customer becomes angry). The player will eventually become better and perform over 1.1xS, and then I'll ask more from him (for example S' = 1.1xS).

The expected result from this should be to always have the player near his actual skill, so the game will be challenging but not impossible, and also doable but not trivial.

Has anyone implemented this? Any practical tips? Does it work? I've read some old posts about someone doing this for a game in development (at that time), I'd like to know how did it turn out.

[Dom]

We do a similar thing in our new (unreleased) game. It is surprising how un-random we needed to make it play well :)

Basically, we speed the dropping of new tiles (same as new customers entering) when the screen is near empty (player doing well) and slow them down when the screen fills up (player having problems). It seems to work quite nicely as you never run out of things to do.

We do still have difficulty modes tho :)

[oNyx]

I would use normalised values [0.0-1.0].

Well, I'm not really sure what I should think of adaptive difficulty or automatic balancing or whatever you like to call it. It works quite well for party games, which are multi player per definition. They need to be interesting till the end.

Mario Kart is a nice example for that. If you are at the last place you get a very high chance for getting powerfull items and if you are at the pole position you usually get pretty useless stuff. The outcome of the race is totally open till the end - keeping the players interested all the time.

It's completely different for single player games where the difficulty ramping is stretched across several levels. Say... there are 20 levels. Person A plays a lot of games (of this genre) and reaches his maximum skill in level 3. And there is person B who'd never played a game like that and reaches his maximum skill in level 20.

It could easily happen that the game gets incredible hard for person A, whereas it's only a bit difficult for person B. It can also lead to abuse of bugs/glitches or disablilitys of the player by the game itself. That's because the game will use those pieces which works best against the player - exploiting colorblindness or bad collision detection of specific bullets/enemies.

So if you do it, don't over do it. Provide some guidance (eg ranges for each level).

Something to google for: fuzzy logic

[luggage]

Some of the games we write have to have adaptive difficulty to control payout. The games are only meant to pay 30% out so it's quite important to get it right. As the game pays more money out it gets harder until it's too hard and then the games has to get easier until it's paid out 30%.

The way we do this is to have a difficulty control that's a float ranging from 0.0 to 1.0. 0.0 is where the game is playing soooo easy it's not true, 1.0 the game is practically impossible.

Any game variables that can tweaked for difficulty are given two values. The easiest and the hardest. In your case the customers rate they turn up would be one of them but there could be lots. Decide an easy value and a hard value then interpolate based on the current difficulty. Say super easy time is 10, and super rock hard time is 0 then if the difficulty is 0.5 the actual time used should be 5.

Once you have this then it's just a matter of moving the difficulty control based on how they did in the previous game. So you could start very easy with something like 0.3. If they then score what you consider a very good score, or play a long amount of levels nudge it up abit. Probably best to move it in relation to how well they did. If they did brilliantly up it a bit quicker. Likewise if they struggle lower it. Eventually it will level out to suit the players skill level.

[princec]

I can't stand adaptive difficulty :( The ideal game doesn't actually need to adjust its difficulty at all - its basic gameplay is simply reward in itself, no matter how easy or hard it is. Consider Solitaire. Or Pac Man.

Cas :)

[papillon]

I don't know how I'd feel about it if I didn't know about it, but if I did know it would annoy me. How can I know if I'm actually getting any better at this game if it keeps changing the difficulty on me?

I'd rather have it as a setting option, so that if you wanted a challenge that would match you, you could have one, but if you just wanted to blow things up in peace you could set it to easy and not have to worry.

However, I am a fan of the "extra chance at the last minute" sort of play in some puzzle games where it draws out those last few seconds to help you pull yourself back from the edge.

[Black Hydra]

Such techniques are forms of AI.

I would suggest reading Game AI by O'Reilly. It goes over some methods from the very simple method you described of calculating skill to regulate difficulty, to making AI's that can actually strategize.

Remember to always make the game slightly harder than the player's average skill is. That way they will always be challenged.

If your really excited there are a whole bunch of things you could use as factors:
-Time since last play
-Amount of consecutive wins/losses
-Amount of loss/victory

[Applewood]

Here's one that's too often overlooked. If you have standard difficulty levels, take time to figure out what you think is right and get some testers opinions (ideally from different levels of hardcoreness in your testers).

When you're happy with your research, code up the numbers and consider the spread of difficulty to be correct. Don't change it a couple of days later - we're done and this is it. If you're really really not happy, then alter it again, I guess, but get to a point where you say "right, this is it", are happy to say that and won't listen to further criticism about the range, as you know it's right. (You can see what I'm saying.)

Then, and only then, add another level of difficulty that's quite a bit harder than the current hard. Do not try to do this as part of your original design - do it after, so it's actually "too" hard. Maybe even add yet another level (perhaps unlockable once you've 'won' on "too" hard)

You'll be amazed how many people probably won't think this super-hard level is too tough, even if you do. You play a couple of levels to test a feature out and then move on. Sometimes you play for a while to get the feel of where you're going with the gameplay. Occasionally you'll play your buddy just for a bit of ritual humiliation and pretend to value his input. If your game is really happening, you might catch yourself playing it purely for fun quite often.

Devout users will play your game 2-6 hours a day, every day for the next 5 years, if it's any good. They'll piss all over your own meager skills, and those of your testers, even without that 'insider' knowledge.

At Yuris Revenge, I can take out 7 random "brutal" enemies, playing on any custom or official map you want, using the poorest defenders (soviets against comps as they have no spies) without breaking into a sweat. If there's enough gems about, I can even do it with superweapons on - that's 7 incoming nukes to my one. I'm pretty mean vs humans too - I'm just a total addict. For players in my league we really do want a "super human, truly bizarre, can you last 5 minutes ?" setting, just to see if the answer is yes or no.

Too few games do this. There's probably more but I can only in fact think of two that really meant it - Doom and Bedlam, and Bedlam was mine.

Trust me. Say you're doing a twitch game and one particular level comprises of 100 baddies rushing straight at you through a gate. On super hard mode, as long as you can complete the level by pulling off exactly (and only) 100 left-eyelid shots with no more than a nanosecond for mouse movement between each shot, some cocky git will do it and love you for it - the fun in most games is in the trying, not the achieving.

[Applewood]

However, I am a fan of the "extra chance at the last minute" sort of play in some puzzle games where it draws out those last few seconds to help you pull yourself back from the edge.
Isn't psychology a strange beast? I too like these sorts of things even though I know they're there. You can play your mate at a racing game knowing full well that only the last 10 seconds will dictate the winner, but you still drive your heart out for a full 5 minutes beforehand.

OTOH, my wife does this bizarre thing about setting the alarm clock 10 minutes fast so she springs out of bed when the clock says 9.00, rushes downstairs and sees the kitchen clock at 8.50 and thinks "cool, 10 minutes, time for a more leisurely cup of coffee". I just look at the alarm and think "it says 9.00, so it must be 8.50"

Same(ish) scenario, different behaviour!

[Escotia]

Remember to always make the game slightly harder than the player's average skill is. That way they will always be challenged.

What makes you think that they want to be challenged?

As an example, Slingo offers no challenge to the player whatsoever. It's a game of 'random' chance but it was a very successful seller. In my experience players just like to be told that they're doing great, regardless of any metric involved.

[mrfun]

In sitations where you want players to be able to compare scores it's challenging to use adaptive difficulty without short-changing certain play styles.

One idea I'd like to add to my word game (Dungeon Scroll) is to have it notice when a very highly skilled player is at the wheel and give that player a 'power-up' that let's him (maybe optionally) SKIP levels - (granting them some points to make up for the levels they skipped as well) - this way they the game gets harder, less boring for pros and gives the player a sense of accomplishment at the same time.

I think a lot of game types could benefit from this, the only trick is figuring out a reliable 'skill metric' to choose when to award the skips, and how long they should be.

A similar idea is instead of award a skip, giving them a VERY powerful item (missiles on level 2, when normally they don't come before 6) to make the early levels less tedious and much faster (and fun) to pass. So, instead of the "make the game harder" method, it makes the game easier so he can get the to the legitimate hard part much quicker.

[luggage]

mrfun: you could just impliment an unlocking system for your levels. Say you need 100 points to unlock the next level. Well, you make sure they'll always at least get 100 pts when they complete a level - this way they can *always* move on to the next round. If they score 200 points (ie they do really well) then they can choose to unlock the next 2 levels therefore skipping one.

Obviously those numbers wouldn't work too well - you'd want to play around to get a good balance but something like that would allow better players to move through the earlier levels quickly. By increasing the amount of points you need to unlock the later levels it would also mean that they might have to replay levels to improve their score, or go to earlier levels they skipped.

[Diodor Bitan]

My favorite method is to offer multiple objectives. The player can either go for the easiest objectives or can try to achive as many as possible, which is a lot harder.

Icy Tower is the perfect example. The game can be insanely difficult or fantastically easy right from the start, depending on whether you try to get the combo jumps or not.

[Wayward]

Up and down difficulty: Personally, I find a level of difficulty (adaptive or not) that can go up and down in a sawtooth pattern way more interesting than one that just creeps up along a linear ramp. It's even more rewarding when the drop in difficulty follows your good moves; if feels like you've earned a respite.

Puyu Puyu, for example, got faster and more difficult until you reached a specific point or accomplished a big move which rewarded you by slowing the game down again. You felt like you'd earned a rest. Then the game would gradually speed up again to even higher speeds, putting you under intense pressure, before slowing again.

I guess Puyu Puyu had the inverse of adaptive difficulty: the more expertly you played the slower and easier the game become, so when your playing technique improved just a little it had a remarkable, expotential result.

RPGs have a form of adaptive difficulty. Playing earns you power in the form of experience, skills, items, and gold. That power makes the game easier. If you keep dying at a same tough point but keep growing in power, you'll eventually gain enough power to get past that tough point. RPGs also have the inverse of adaptive difficulty at the same time: the better you play (the more thorough and efficient you are), the more power you acquire and the easier the game becomes.

Greater risks, greater rewards. Expert players seeking the big payoffs will take bigger risks and consequently make the game more difficult for themselves.

Max Payne used adaptive difficulty. Scott Miller talks about it in Auto-Dynamic Difficulty (http://dukenukem.typepad.com/game_matters/2004/01/autoadjusting_g.html).

Is an adaptive level of difficulty a good thing? I'm neutral on the matter. It can work, or it can ruin a game. Some game designers and game players can be a little suspicious of the concept but accept it when they see it work. 'Catch-up' in racing games is one working example of an adaptive difficulty, but I've played some racing games where too much 'catch-up', in my opinion, spoiled the game.

Will an adaptive difficulty work for your game? I'm not sure that's something you can fully predict at the design stage - you just have to try it.

Dynamic Gameplay Adjustment (http://www.gamegirladvance.com/archives/2004/11/23/dynamic_gameplay_adjustment.html) (2004-11-23) at GameGirlAdvance (http://www.gamegirladvance.com/).

[svero]

My big problem with adaptive difficulty is defining what exactly the ideal state of difficulty should be. Is the best game one where you never lose? Or is it one where you lose sometimes? and if sometimes.. then how often?

At the simplest level it might be...

Player keeps losing -> make it easier
Player keeps winning -> make it harder

But the goal is to provide a game that's not frustrating or too easy, and to some extent when we define "keeps losing" and "keeps winning" we're once again choosing a difficulty level. Again to some extent it comes back to.. what is fun? And the answer is that there is no answer really. That is to say for some people fun might be a game where you never lose, and for other people fun might only happen when they start losing. I myself like to be challenged. A game where I always win is usually quite boring to me. So once again you end up in a .. you can't please everyone scenario.

[oNyx]

[...]
Max Payne used adaptive difficulty. Scott Miller talks about it in Auto-Dynamic Difficulty (http://dukenukem.typepad.com/game_matters/2004/01/autoadjusting_g.html).
[...]

A good read. Thanks for the link.

It's also good to remember that people don't get frustrated by dying itself - it's the fact that they have to wait (loading times) and that they have to do more or less stuff again (and again).

So... it's for sure a good idea to keep loading times short in those dying cases. Painkiller for example did a very poor job there. Every time you died everything was completely reloaded. A big fat loading penalty. 30-45seconds for loading stuff, which... uhm... already was loaded, because it's the same part of the level, which you've just played.

[GhostRik]

Sometimes it's good to hit an "impossible" level to see what you're aiming at. Like getting to the Quarks level on Robotron. "Gee, that's impossible." No, it's not impossible, one just has to develop the skill to beat it. Then, two levels later, one is surrounded by Grunts and the brain says, "There has to be a way to beat this." If a game constantly adjusted to my level of difficulty I might think "Gee, this is okay" instead of "I've gotta beat this!" (Someone once said that the beauty of Robotron's difficulty is that when you die you think "If I'd done this instead of that I would have survived," which makes you go back and try again, even against hordes of enemies. People are still playing this game over 20 years later...)

I wonder about adaptive difficulty systems because what if I decide to play badly just to keep it easy? I'm not inherently lazy (I'm currently aiming for the "pain lane" at my swim workouts), but I think I play games partially for the fun that external motivation brings.

[EpicBoy]

In my experience, for the vast majority of game players (myself included), fun == progress. If I'm constantly getting killed and having to replay the same section of the game over and over, it kills my desire to play.

Automatic difficulty adjustment is nice, but it needs to be done in such a way that I don't feel like I'm being helped. Max Payne pulls this off pretty well ... I die a few times but the next time I -just- slip through and I feel satisfied and want to continue playing. They help you over the rough spots and it really feels right.

Of course, this differs based on genre. If you're doing a story driven game, like Max Payne, it makes sense. However, for an arcade style game, like Robotron, part of the appeal is the insane difficulty. Gotta pick your design battles...