AI coding, I can’t even begin to express how excited I am

I haven’t got to design an AI since…well 2006.  It’s been a long time and the Stardock games team of 2013 is insanely awesome and the tech is just amazing too.

Now, I programmed the AI in Legendary Heroes and helped on the AI in Fallen Enchantress (I wrote worker functions for War of Magic).  But I haven’t got to sit down and make something from scratch since GalCiv II.

And wow, things are so much better now.  Virtualization, lots of cores to handle threads (GalCiv II was written for a single core with 1 hyperthread).  The things I can do now are, well they’re just plain sick.

While I am quite proud of the AI in Fallen Enchantress: Legendary Heroes, it was something I had to retrofit onto Elemental: War of Magic. It was a miracle to get it to work as well as it does because War of Magic was single threaded.  I had to make it multithreaded after the fact (if there are any programmers here, please feel free to comment to explain how nasty that is).

In GalCiv III, the AI is able to operate on a virtual machine version of the game state. So I can monkey around in real-time without it affecting other players. I can not only have every computer player have its own thread, I can have different components of the AI have their own threads and in multiplayer, I can distribute the work up across the different players.

There’s a lot more too. From crowd sourcing ship designs, tech tree strategies, planetary improvement strategies, etc. to throwing raw number crunching stuff onto the player’s video card.  We’ve come a long way from 2006.

80,893 views 39 replies
Reply #1 Top

The things I can do now are, well they’re just plain sick.

Well we are expecting sick things now then!

 

So I can monkey around in real-time without it affecting other players. I can not only have every computer player have its own thread, I can have different components of the AI have their own threads and in multiplayer, I can distribute the work up across the different players.

One thing I'm not sure how it's gonna work is in multiplayer. If two human players, say, play against two or three computer players, in which player's computer are the AI calculations taking place??

Reply #2 Top

(if there are any programmers here, please feel free to comment to explain how nasty that is).

I can well understand your reluctance to take on explaining this to everyone. Asynchronous processing is indeed a bear to program for. There are lots of special instructions to understand that are almost mandatory in making sure that your data, or even your memory assignments, don't get messed up. If you are not careful you could end up with memory overlays, or thinking you have memory you have already released back to the OS. You need locks to coordinate access to data by many coincidental threads. You need to make sure data you were about to change isn't being changed on another thread to something that only that other thread understands. And much more.

Reply #3 Top

Congrats Frogboy. Hope you don't mind with some suggestions on the Ai.

I would like to see all the players taking up a fair share of the map after the colony rush. I've noticed that about two or three Ai's are usually dominant, and at least three are usually underpowered. You might have one with a super army with 4 planets. I would like to see this to be more even. If we get something like the Yor again where you are forced to play these differently than the other species then the Ai still needs to know how to play these. An example is ab economic stradegy was not going to work for the Yor, but you were going to need a farming stradegy. This seemed to be something the Ai couldn't do.

The Ai needs to be able to adjust its taxes, so it can be able to control its approval to grow reasonably. Everyone needs to care about morale research improvements and research. Lets not have building farms to the point where the morale is brought down to a low level so the empire can't grow. I think the Ai should set the planetary populations to 20 billion each.

I think that the Ai shouldn't terraform tiles until the other tiles are used up on the planet, You should probably not upgrade improvements until your planet can afford it. If the opponent has a better improvement then it should be able to change the current structure on the planet with it. 

I think all the Ai's should build improvements on the planet.

I think that a minimum that the Ai should be concerned with is what the game suggests. What I mean is that it should have a 100 in research, economics, and military. I know that the techs are classified into subcatagorys. Planetary improvements and ship components should also be classified this way. With as small as catagories as possible. When these things are classified like this then the Ai has information it can draw on. When what it is doing doesn't work then it can start to change to a different thing in the catagory. How you do this I'm not sure maybe it could be random. Eliminating what doesn't work until it can compete with the player. The computer will need to save this information when you save the game. That way it can learn. If it has advanced espionage or is close to you it can observe what you are doing to try to counter you. The player will need to be able to reset this. You could even download this from multiplayer on the internet,

again pne way to do this is try random stuff till something works against the opponents. I think this should be as hard as it gets before it cheats on advanced algorithms or maybe have a learn mode. The player should be able to reset this.

Reply #4 Top

We are excited too, Brad. This will be an exciting ride towards the final release Anno Domini 2015. I can't wait to be a part of the alpha/beta of this game.

Reply #5 Top

Quoting Lucky, reply 2

Asynchronous processing is indeed a bear to program for.

 

It could be, but in C# and .net4.5 it's actually pretty straight forward. It even got better with concurrent lists and async/await keywords, but I'm not going too much to details. However, the question is, what do these guys use for GC3? Somehow I don't think it is the .NET, because they and you make this so big issue.

Reply #6 Top

I just hope the AI will keep things interesting on easier levels too.

Reply #7 Top

All of this sounds really great. Maybe even bit too much ambitious. How will it work, if AI will be programmed to take advantage of different computers in MP, or Virtualization if one do not have internet connection or play single player?  I can already hear people being worried about internet aspect of the game. For me personaly, if it makes AI better, I am all for it, but I live in a country where internet connection is not an issue.

However considering the fact, that game will be digitaly distributed only. And the fact that you are trying to make something revolutionary here, there need to be some sacrifices. Same with the graphics and memory, 64bit is must. So if this can do the same for the AI, it is worth doing it.

Just make sure, you take your time to do it right, and make it so, that its easy to improve in the future, without rewroting whole stuff like in the Elemental.

Reply #8 Top

Quoting Omnax1, reply 7

All of this sounds really great. Maybe even bit too much ambitious. How will it work, if AI will be programmed to take advantage of different computers in MP, or Virtualization if one do not have internet connection or play single player?  I can already hear people being worried about internet aspect of the game. For me personaly, if it makes AI better, I am all for it, but I live in a country where internet connection is not an issue.

In this case, "virtualized" means that he's taken the game state and made a copy of it. The AI can do stuff in that virtualized version and play a few turns ahead to see which one it likes the best without impacting the real game state (for example). Being 64 bit and multicore, you can do a lot of stuff that just isn't practical in a 32 bit single core design.

I don't know about the MP stuff, but they've said repeatidly that it's a SP focused game so I can't imagine the AI being handicapped if you're not playing in MP. The AI might be handicapped if you never get online as it will need to connect to the metaverse to do things like use popular player ship designs, though.

Reply #9 Top

Quoting Stringer2, reply 5

It could be, but in C# and .net4.5 it's actually pretty straight forward. It even got better with concurrent lists and async/await keywords, but I'm not going too much to details. However, the question is, what do these guys use for GC3? Somehow I don't think it is the .NET, because they and you make this so big issue.

The trick there is that if you're just using "await foo.DoStuff()" form the UI thread, you're not actually getting parallelism. In that context it uses cooperative multitasking and yields the existing thread when it's waiting on a blocking call. That maintains UI responsiveness, but isn't running things in parallel. (It can run things in parallel if you use it in some other contexts, though.)

So even in C#, they'd probably have to be using the thread pool or spinning up threads to do what a large game needs to do.

All that said, if I had to guess what they're using, I'd go with C++. Maybe Brad will be willing to enlighten us. :)

Reply #10 Top

I am excited that you are excited.  That excitement pushes inspiration and leads to higher quality work.  The only way you are going to exceed your success on GalCv2 is to have that excitement, that inspiration, and maybe a little bit of help from your friends.

 

Good luck.  I consider your "hobby" work with AI to be part of the overall evolution of AI that is occurring in all aspects of computer work.  Watson and GalCiv and all AIs in between are part of a grand effort to create Artificial Intelligence sufficient to challenge humans to become intelligent as well.  Gaming AI techniques can only help the more "serious" projects whether we readily see the connections or not. 

 

Keep up the good work.  Not only does it provide more enjoyment for us players, it is fulfilling some cosmic mission.  (Well, possibly helping some cosmic mission in some tangential way.)  (But that counts!!)

Reply #11 Top

Quoting Tridus, reply 9

The trick there is that if you're just using "await foo.DoStuff()" form the UI thread, you're not actually getting parallelism. In that context it uses cooperative multitasking and yields the existing thread when it's waiting on a blocking call. That maintains UI responsiveness, but isn't running things in parallel. (It can run things in parallel if you use it in some other contexts, though.)

 

True, not like. But the real trick is here:

var Temp=foo.DoStuff();

bar.DoSomethingElse();

var ReturnHere=await Temp;

 

And the DoStuff probably wants to run in parallel with Task.Run or something. What I'm trying to say, is that .NET is full of tools and sugar candy for thread programming. As long as you'll keep few basics in mind, it should be piece of cake.

Reply #12 Top

Quoting Stringer2, reply 11

And the DoStuff probably wants to run in parallel with Task.Run or something. What I'm trying to say, is that .NET is full of tools and sugar candy for thread programming. As long as you'll keep few basics in mind, it should be piece of cake.

Yeah, for sure. :) It's a good framework. I don't know how popular it is for games though.

Reply #13 Top

Quoting Tridus, reply 12


Yeah, for sure. It's a good framework. I don't know how popular it is for games though.

There has been few. I don't know about the really big names, but I think the Distant Worlds is one example. The cross-platform requirement is probably one of the biggest reasons someone would use something else. But in the MS only worlds .NET should be rather attractive one, because that is where MS is pushing us all.

Reply #14 Top

Are you still going to use linear bonuses for AI? Differentiating them via exponential graphs (earlier power peaks etc. for different AI's depending on difficulty) could help balance them at higher difficulty levels, where you usually see one AI performing better overall. (at least it did in Galciv2, and also did it somewhat in FE:LH) Any sort of experimenting would be awesome to be honest, and if you made it moddable, much money will be spent ;)

Reply #15 Top

Quoting Stringer2, reply 5
It could be, but in C# and .net4.5 it's actually pretty straight forward

Admittedly, the asynchronous and multiprocessor programming I did was closer to the machine level than what is required for GC3, but I have been retired for 13 years now and had not been aware that there were some newer programming languages that helped in these areas, yet I should have expected them. Even so, I wouldn't be surprised if the devs found some of the newer programming languages lacking in some instances, simply because it takes time for any programming language to "mature". I am sure Brad & co are quite busy, but maybe they can give us a simple  and approximate count of the number of work/arounds they have had to create to solve deficiencies in the programming language(s) ability to handle asynchronicity?

Reply #16 Top

Brad- Can you confirm if the AI has to actually search for planets, or if they will have knowledge of, and will bee-line towards the good worlds from the outset, as in GC2?  This is one thing that always bugged me.

Reply #17 Top

Quoting Lucky, reply 15

Even so, I wouldn't be surprised if the devs found some of the newer programming languages lacking in some instances, simply because it takes time for any programming language to "mature". I am sure Brad & co are quite busy, but maybe they can give us a simple  and approximate count of the number of work/arounds they have had to create to solve deficiencies in the programming language(s) ability to handle asynchronicity?

 

Heh, .NET ain't that new any more and you missed it only by couple years (initial release 2002). Although you are right, it has matured since then. In some cases a lot, and in some others not so much. Anonymous delegates, the async programming and the linq queries are some of the newer aspects I feel handy. I would worry more about the DirectX or what ever they are using for, because those tend to make your life very painful. 

Reply #18 Top

Mature or not .NET threading can be messy especially if you rely on it to handle all of your allocations. For goodness sake it has a hard time cleaning up when you have multiple dbs connected to it asynchronously so I would think they are relying on some manually handling this stuff to be efficient.  Especially in a game of this nature a couple of errands threads could cause memory leaks.  And since they are running on seperate threads that is going do be no easy task to properly manage even with some assistance from the language.

I too am interested in knowing about workarounds if they are using .net.

 

Still sounds like a roaring start and I cannot wait to taste the fruits of this labor.

Reply #19 Top

Quoting charon2112, reply 16

Brad- Can you confirm if the AI has to actually search for planets, or if they will have knowledge of, and will bee-line towards the good worlds from the outset, as in GC2?  This is one thing that always bugged me.

In GalCiv II, the AI didn't know where the good planets were, it just knew where the stars were. It had (and did) scout them out.

One of the things that I have not considered cheating in the past is the AI making use of undocumented patterns.  I'm not going to go into this here as even mentioning this will mean some players are going to figure this out but I have tended to make sure the galaxy seeds things in a non-random way so that if you have N stars in a particular pattern, you know that a particular star will be yellow and therefore probably have a good planet around it.

If you load up GalCiv II and look very carefully at the "random" stars you'll discover there are patterns.

Now, you could argue that's not "fair" but we also don't explicitly say that yellow stars will likely have good planets. That's just a pattern that many players came to recognize. But there are lots and lots of other patterns in the game from the way asteroids are done to the positions of anomalies to the positions of resources. There are relationships between everything. The more sophisticated the player, the better they get.

I'm going to regret giving this away as some of the more clever players will likely put up a guide (or if they're smart, they'll keep it to themselves).

Reply #20 Top

OMG, the stars, the stars!!

Reply #21 Top

Quoting Frogboy, reply 19
In GalCiv II, the AI didn't know where the good planets were, it just knew where the stars were. It had (and did) scout them out.

One of the things that I have not considered cheating in the past is the AI making use of undocumented patterns.  I'm not going to go into this here as even mentioning this will mean some players are going to figure this out but I have tended to make sure the galaxy seeds things in a non-random way so that if you have N stars in a particular pattern, you know that a particular star will be yellow and therefore probably have a good planet around it.

If you load up GalCiv II and look very carefully at the "random" stars you'll discover there are patterns.

Now, you could argue that's not "fair" but we also don't explicitly say that yellow stars will likely have good planets. That's just a pattern that many players came to recognize. But there are lots and lots of other patterns in the game from the way asteroids are done to the positions of anomalies to the positions of resources. There are relationships between everything. The more sophisticated the player, the better they get.

I'm going to regret giving this away as some of the more clever players will likely put up a guide (or if they're smart, they'll keep it to themselves).

 

 

Listen, I don't mean to sound like a jerk or anything, but what's the point of these patterns?  Are "good" players supposed to memorize these patterns in order to game the system?  What is this, Super Mario Brothers?

Reply #22 Top

Quoting Wetballs, reply 21

Listen, I don't mean to sound like a jerk or anything, but what's the point of these patterns?  Are "good" players supposed to memorize these patterns in order to game the system?  What is this, Super Mario Brothers?

No, they help program a more efficient AI without explicitly giving the AI benefits like extra resources or beneficial modifiers.  Basically, they are extra data points that the AI has access to when making its decisions that the player may or may not have access to (depending upon how much they look for and/or already know the patterns). 

Reply #23 Top

Quoting Kantok, reply 22

Quoting Wetballs, reply 21
Listen, I don't mean to sound like a jerk or anything, but what's the point of these patterns?  Are "good" players supposed to memorize these patterns in order to game the system?  What is this, Super Mario Brothers?

No, they help program a more efficient AI without explicitly giving the AI benefits like extra resources or beneficial modifiers.  Basically, they are extra data points that the AI has access to when making its decisions that the player may or may not have access to (depending upon how much they look for and/or already know the patterns). 

 

 

Yeah well, I don't think I'll ever look at yellow stars the same again.  Did everyone know they had better planets?  I never realized that.

 

Reply #24 Top

[quote who="Wetballs" Listen, I don't mean to sound like a jerk or anything, but what's the point of these patterns?  Are "good" players supposed to memorize these patterns in order to game the system?  What is this, Super Mario Brothers?[/quote]

On the other hand, these patterns ensure a reasonably fair distribution of resources (But not planets, GRRR) which is something you will definitely need with multilayer.

With regard to the AI knowing where the good planets are,,, i use that against them! What i do is, i send out colony ships and park them near planets but don't colonize them (especially lone planets in isolated systems). I then wait for AI colony ship to turn up then i colonize the planet under its nose (but doesn't work against the Yor, obviously). Because it cheated and new where the planet was, it has now wasted all those turns on a wild goose chase, sucked in!!

If i was to have colonized the planet straight away, then the AI colony ships would have magically known and turned towards other planets and gotten there much faster, possibly faster than me!

Reply #25 Top

Quoting Wetballs, reply 23
Yeah well, I don't think I'll ever look at yellow stars the same again.  Did everyone know they had better planets?  I never realized that.

He didn't say "better planets", but "good planets", which is accurate. The best planets are around purple stars.