Idea: new approach to balancing difficulty level and AI

from Elemental Forums

I’ve just started playing the new 2.01 version of GalCiv2 - Dark Avatar (have played the old one before), and while doing that came upon an idea for a different approach to matching the difficulty level, and in particular the AI, to the skills of the player. The idea applies to all turn-based strategy games, and could be used in Elemental, as far as I can see.

One of the things that bothers me the most about strategy games in general, and this goes for GalCiv2 too, is that a game usually ends up being too easy or too hard, and thus either becomes boring or frustrating. In both cases, the game fails to provide me with the entertaining challenge of having to make the best game play choices in order to be able to win, which I’m looking for when I buy the game.

The problem is when I’m supposed to choose the difficulty setting before a game, as I don’t really know how normal, the “normal” setting is, or how easy/hard, the “easy/hard” settings are. Nor do I necessarily know how good I am at the game, if it’s my first time playing it, or if I haven’t played in a long time. Is the 10% economy penalty and general AI algorithm the setting that matches my own playing level? Or am I closer to the 10% economy bonus and advanced algorithm? How do I know?

So, in my experience, picking a difficulty setting that is in balance with my ability to play the game is usually impossible, leaving me with a game play experience that either lacks a challenge, resulting in the kind of boredom that makes me quit the game and not want to pick it up again, or if it’s too difficult, gives me a frustrating experience of not standing a chance to win, in which case I also want to quit and not pick it up again. Because the AI players are too far away from my own playing level, turn-based strategy games usually fail to provide me with the challenge that I was looking for in the first place. This is, in my experience, one of the primary reasons for why I quickly lose interest in what could otherwise be a great strategy game.

To solve the problem, I came upon this idea:
During play, the game measures how well the player plays the game, and adjusts the difficulty and AI accordingly. There are many ways you could handle this. If there is a playable tutorial, you could use a default easy difficulty/AI setting during that, and measure how well the player handles this challenge. The measured player skill level will then decide what difficulty/AI setting the next game will be on. Thus, the tutorial not only teaches the player how to play the game, it also teaches the game how to balance the difficulty/AI settings to the player. If there’s no playable tutorial, you could let the player choose a difficulty setting of his liking for the first game, and measure his ability against that.

Measuring could be as simple as how high a score he gets at the end of the game. A more advanced (and probably more efficient) way could be to have a number of parameters that translate into AI/difficulty settings, that you could measure, like ability to balance economy parameters, time to build civilisation, speed of exploration, ability to design and use military units, etc.

You could keep measuring this for each game he plays, and as the player develops his skills and gets more experienced, you can adjust the difficulty/AI accordingly, when a new game is started. It would then be desirable to allow the player to choose the level of challenge he is looking for, and match the difficulty to it. Thus, if a player plays the game predominantly for the feeling of empowerment, rather than that of a genuine challenge, he should be able to set the AI to play below his skill level, as he will not risk losing, but still gets some sense/illusion of challenge. Those that genuinely play for the challenge, like me, can choose a balanced setting, where the game logic tries to match the skills displayed by the player, to the best of its ability. The masochistic player should also be able to choose a difficulty level above his skills ;)

The measured skill level of the player could be computed into a number/rank that he can view. This would add an element of achievement, besides winning each game, to see how high a skill rank he can get. Further, it would be possible to use the rank to find matching players for multiplayer games.

Of course, it could still be possible to allow the player to choose a difficulty setting the old way, if he wants that, while giving him the option of this dynamic, personally tailored difficulty setting.

7,940 views 13 replies
Reply #1 from Elemental Forums Top

While I wouldn't necessarily mind this as an option, I worry that it would present a LOT of work for the devs to get anywhere close to being right. If that's true, then I'd rather they focus on gameplay. The easy way would be to base it off of score, but that is extremely flawed... For example, turtles often take a long time to win (which usually results in fewer points), but that doesn't mean they can't wipe the floor with a very tough AI.

Personally, I don't usually have this problem. I always play the tutorial, if there is one, and from there I tend to start off on the normal difficulty setting. If I'm worried that the tutorial was insufficient or that I didn't really perform well during it I might play my first game on easy. Nonetheless, I can usually figure out during my first game if I chose the right level or not. If the difficulty is too hard, I usually see my end coming and start over, after having learned from my mistakes. If it's too easy, I quickly find myself steamrolling any and all opposition, expanding much faster than my opponents; if I'm having fun with that I'll finish the game, otherwise I'll start over at a higher difficulty.

Basically, I think that after a small number of games most players can determine the optimal difficulty level by themselves much better than the computer can. People who are unwilling to play a few games before figuring out exactly where they stand are probably not the type of person who will be buying a 4X strategy game.

Reply #2 from Elemental Forums Top

The first version of this post was too hostile.

You just described rubberband AI, an ooooold idea that you don't need to write so much text explaining - any experienced gamer knows what it is. I hate it because it punishes the player for doing well, and makes the incredibly counterintuitive move of gaming the system to make yourself look weak while preparing some unstoppable burst a viable, even top-tier, strategy.

Traditional difficulty levels should be the default. Perhaps stardock could steal Fall from Heaven's "flexible difficulty" option, wherein you pick a difficulty level to start, then every x turns, your score is compared to the highest AI's score and your difficulty level is appropriately adjusted if there is a big gap. I'd never use it and I'd savagely mock people who assukme it as default when discussing balance on the forums, but it'd be an option.

Ashes of the Singularity II
Reply #3 from Elemental Forums Top

I enjoy reactive difficulty in games... It is nice to dominate once in a while but if the game is always somewhat challenging, its longevity increases drastically in my opinion.

I really like the idea of tracked skill. Ranked Games could always affect your ranking unless you quit part-way through... the idea of using this ranking as a measurement for what could be Easy, Medium and Hard is fantastic. :)

Reply #4 from Elemental Forums Top

How about just allowing the user to change the difficulty setting mid-game?  If they're doing too poorly or too well, then they could adjust the AI themselves.  They could even switch back and forth between two settings every X turns, if they want to simulate an AI that's somewhere between those two.

Reply #5 from Elemental Forums Top

Blegh, that's all I need.  A tactically inferior AI stepping itself down to my production levels to keep things fair.

 

I got a much better idea, make a much better AI than normal(this will most likely happen) and don't cheat.  That's your standard difficulty, then add identified cheating prick ai levels on top of that so the masochist in me can be a masochist when the normal AI doesn't beat me up anymore.

 

If the game is so long that you need to adjust it mid game instead of starting over when you realize you've been steamrolled or are butchering them, you're playing the wrong genre.

 

A reactive setting to go with the normal ones wouldn't be a subtraction, but gah!  I'd have to turn emo and kill myself like a retarded fuck teenager with a myspace addiction.

Reply #6 from Elemental Forums Top

I have the exact same problem, i think this is a great idea in theory and would definately solve the issue for gamers like us, BUT it would be too hard to execute and thus the only reasonable implement of this technology(at least at the moment) wouldn't be as efficient as you(and myself) aspire.

Reply #7 from Elemental Forums Top

I certainly wouldn't want Stardock throwing so much resources at such an idea that it makes the game take a lot longer to arrive... I can deal with the normal system of difficulty by the idea of something more interesting than that is always captivating.

The idea of changing the difficulty mid-game is just a manual version of a reactive difficulty. If the game had such a setting in the first place, it would most likely also feature the normal style of difficulty. Thus, no one would have to use one system or the other. I think it is important to get this sort of idea out there for Stardock to see and contemplate however. :)

Reply #8 from Elemental Forums Top

One thought: many games (though not often strategy games) have a difficulty level you can bump midgame.

It seems like a significant amount of effort for a feature that's likely to be problematic. How often will it switch difficulties? If it goes up and recalculates AI goals when would it potentially shift again? When it switches, does it either trash the progress toward those goals with new ones or get stuck with goals generated by a weaker/stronger AI X turns ago?

There's just so many complications with making this work and there's not much return on investment for the effort.

Reply #9 from Elemental Forums Top

I assume the OP speaks of SP gameplay. If I have read it right, Elemental wil have a MP component. If you wish to test your play level vs a smart AI, play MP vs other human players. It doesn't get more competetive than that... ;)

Reply #10 from Elemental Forums Top

I'm fully against any sort of reactive AI that changes difficulty within the course of a single game. The way I usually play strategy games, I start out the game significantly behind the AI in terms of points, and build up slowly but surely. Basically I provide myself with a strong foundation while trying to stay out of major conflicts, and when I'm done with that I slowly overtake the AIs pointwise. In order for the computer to really gauge the player's strength it has to understand the player's overarching strategy, and that's hard for a person to do, let alone a computer.

Ultimately, any reactive system will be exploitable. There will be some strategies that the computer will underrate, allowing you to trick the AI into being dumber than it should be. Likewise there will be some that the computer will overrate, resulting in the opposite. In some cases it'll make the game boring (too easy), sometimes it'll be frustrating (too hard), and it won't always be obvious why. I really can't imagine any big fan of 4X games being unwilling to play a few games of trial and error until they figure out what AI suits them best.

I don't think it's worth the dev time to come up with a hit-or-miss reactive AI difficulty. I think their time is better spent perfecting the gameplay, balance and UI. And ultimately, perfecting those will go a long way to solve the 'problem' of figuring out your preferred difficulty level.

Reply #11 from Elemental Forums Top

Well, a reactive AI could easily be set to only monitor wins versus losses based on a point system... If you won big time, then the AI is going to bump itself up appropriately. They could measure the current difficulty with ranks, creating a new dynamic. You could try to bump up the ranking as high as you can before the game becomes impossible. I'm sure the elite players of such games would be interested in the bragging rights, though I'm certainly not going to be one of them. @_@

In addition, I've always heard people complain about the difficulty jumps from one setting to the next. One is too easy and the next is too hard, no in between. This sort of system would create that in between.

As I said before... I'd rather Stardock focus on the game right now. Perhaps this system could be implemented later at their leasure?

Reply #12 from Elemental Forums Top

Quoting Lavitage, reply 2
You just described rubberband AI, an ooooold idea that you don't need to write so much text explaining - any experienced gamer knows what it is.

Well, a sizeable amount of text is good, because, if one reads it with an open mind and curiosity, one might be able to understand the idea behind the words. ;)

What I’m describing is not the effect of a rubberband, where the AI adjusts itself during run-time to the player’s choices, such as rubberband AI in racing games, where the AI cars increase/decrease speed to maintain a short distance to the player. It would ruin the entire point of having to make the best game play choices to win, if you are effectively making those choices for the AI at run-time as well. It would further undermine Brad Wardell’s great AI work, if the AI players were reduced to increased/decreased bonuses at run-time to mimic the players choices (i.e. cheating AI), and in this way allowing them to catch up to the player, whenever he made a smart move. So that’s not what I’m suggesting.

What I’m suggesting is that the game measures the player’s ability to make game play choices for an entire game along a number parameters that are decisive for balancing the game, and that translate into difficulty/AI settings. The measured player skill level then decides the difficulty/AI settings for the next game - not the one he is currently playing.

The only use of run-time adjusting AI that I could see as a good idea, would be during the tutorial/first game, where the game has no idea about where to place the difficulty/AI settings, and therefore has to start on a default low level. You could solve this by either making sure that the tutorial/first game is short enough, so that the player doesn’t become bored with the lack of challenge, or you could adjust the difficulty/AI settings closer to the player at run-time during the first game only, in order to maintain some degree of challenge. But once the game has tracked the players skill level through an entire game, there’s no need to adjust at run-time, and thus punish the player for making good choices, as the starting settings will already be close enough to the player’s level to provide a genuine challenge. At least that is possible if the entire game is properly balanced, and this approach to setting difficulty/AI before a new game is used.

To be a bit more specific, what I imagine is that it would be possible to benchmark the AI algorithms, in order to determine the performance of its best non-cheating decision ability. This would compute an optimum AI strategy trajectory with turns on the x-axis and game play achievement value on the y-axis. A trajectory could be calculated for each winning criteria, where the y-value is performance towards achieving the winning goal (science victory, conquer victory, etc.), which would be areas like speed of research, economy strength, etc.

When a player plays a game, you can calculate the difference between the known optimum AI y-value and the y-value for the player for each turn. When the game is finished, you can compute the average of the y-differences towards the winning goal, for the entire game, and you can use that value as the basis for the set of penalties/bonuses for the AI players, which are needed for them to match the player trajectory. Of course, you would then choose the AI algorithm with the smallest penalties/bonuses from its optimum y-value to the player’s value. The settings found in this way would then be applied at the next game - not during the game where they were measured.

This is just a simplified description of a situation that is more complex, when you get under the hood, in order to illustrate the general idea. It could also be approached in many other ways, like using zero-sum equilibrium theory, if you’re into that kind of thing.

Further, I have thought a bit more about the problem, since posting the idea, and the issue is not strictly with being able to choose the right difficulty setting. The problem is that of achieving balance in the game play challenge in the broadest sense. Choosing difficulty and AI is obviously important, but if other features are implemented that breaks the balance, like crippling starting location, or random starting bonuses of any kind that skews the balance, etc., then that needs to be addressed as well. The goal is to achieve a balanced challenge for the player, for as long as possible, so he doesn’t know whether he will lose or win, and has to optimise all game play decision to achieve victory. Every design element of the game must be taken into account to reach that.

Bottom-line is that I’m advocating taking the genre to the next level in providing the player with a balanced challenge. Strive to use all the possibilities of the computer medium to match the player’s skills, in order to provide him with the greatest degree of an entertaining experience in having to use the best of his decision-making abilities to win.

 

Reply #13 from Elemental Forums Top

Quoting E_MacLeod, reply 11
In addition, I've always heard people complain about the difficulty jumps from one setting to the next. One is too easy and the next is too hard, no in between. This sort of system would create that in between.

I think stardock actually came up with a good solution for that in galciv 2: the difficulty slider. Instead of easy/medium/hard/impossible there's a slider with something like 10 values (i think). Having a wide gradient of difficulty levels makes it easier to avoid harsh jumps between them.

Quoting E_MacLeod, reply 11
They could measure the current difficulty with ranks, creating a new dynamic. You could try to bump up the ranking as high as you can before the game becomes impossible. I'm sure the elite players of such games would be interested in the bragging rights, though I'm certainly not going to be one of them. @_@

Players interested in bragging rights can already brag in any 4X game. They just need to set the difficulty as high as it can possibly go and still kick butt. Whether the computer slowly adjusts the difficulty up until the player reaches the highest difficulty level there is, or the player chooses it from the get-go is irrelevant...

Quoting Aladin101, reply 12

This is just a simplified description of a situation that is more complex, when you get under the hood, in order to illustrate the general idea. It could also be approached in many other ways, like using zero-sum equilibrium theory, if you’re into that kind of thing.

That's just the problem. You just described a simplified version of the system, and even that seems like it would be a nightmare to implement. I'm not opposed to your idea in theory, but I hope the devs don't waste their time on such a massive experimental undertaking that ultimately won't add that much to the game. If there are enough difficulty levels, I think most players can find one that consistently provides them with a suitable challenge. At least, up until a player becomes so good that the AI is just insufficient - but a reactive difficulty wouldn't solve that problem either.