Sins II Dev Journal: v1.65 'ARES: God of War' Preview

https://store.steampowered.com/news/app/1575940/view/660484779820450101
from Sins2 Forums

Welcome to this Dev Diary! I’m Tom Fortune, one of the engineers on Sins of a Solar Empire II, and I’ll be walking you through the reasoning and motivations behind the upcoming AI overhaul. Let’s get started. 

Cause & Effect

If you've spent some serious time playing against the Sins II AI, you likely already know the shape of the general complaints we get. The AI doesn’t really like to fight as much as we’d like. It builds a credible early empire and then has issues maintaining it. It spends wildly and in ways that don’t necessarily make sense. You show up to a planet, and structures and items are not built in a way a human would ever do. Fleets get distributed a bit strangely. Exotic factories rarely get built, research wanders instead of compounding. At higher difficulties, the answer to "make it harder" has historically been more resources rather than smarter decisions, which feels less like a stronger opponent and more like a tax.

None of this is for lack of effort. The existing AI is a layered system of specialist controllers: One for purchasing, one for fleets, one for colonization, one for research, one for defense, and several more. Each makes locally reasonable decisions on its own. That architecture has carried Sins II from launch through every expansion, and it does an enormous amount of work that players never see: keeping economies solvent, keeping fleets supplied, keeping planets defended, reacting to threats. The reason it struggles in the late game isn't that any one controller is broken. It's that there has never been a layer above the controllers whose job is to look at the whole board, decide what the AI is actually trying to accomplish, and then make the controllers cooperate toward that plan. Until now, each controller optimized its own little kingdom without strong coordination.

Introducing ARES

After more than a year of fixing symptoms, last December we decided to bite this rather large bullet and fix the cause. Rather than patch each controller into being more aggressive or more focused in isolation, we wanted to introduce a single strategic brain that owns the AI's intent for the match; and then have the existing controllers execute that intent.

The controllers are very good at how. They've never had much of a why. So, we created a top layer to sit above all of them, the Adaptive Reasoning & Execution System, or ARES for short. ARES is now our why.

Three principles drove the design:

ARES is a director, not a suggester.

When ARES is enabled, its decisions are orders. Controllers don't get to quietly re-evaluate them or fall back to their old autonomous behavior when ARES says something inconvenient. If ARES picks a target, fleets go there. If ARES says the economy needs a refinery world, the planet controller builds toward a refinery.

Strategy is goal-shaped, not rule-shaped.

Instead of a thousand if/then statements describing what the AI should do in every situation, ARES generates and scores a list of candidate goals:  capture this territory, defend this border planet, build a capital ship for your fleet, rush this tech tier. Basically, instead of a behavior tree branch that says ‘if enemy fleet > my fleet, then retreat,' ARES asks 'Is continuing this goal still worth more than the alternatives?’

It will commit for as long as those goals remain the best option. Goals are concrete things with a target, a cost, a chance of success, and a payoff. That makes them tunable and inspectable in a way that hand-written behavior trees aren't. 

Difficulty should mean better thinking, not bigger numbers.

Higher difficulty AIs should see the board more clearly, score targets more accurately, commit to plans more decisively, and tolerate more risk, not just earn 30% more credits. That's a long-term goal and it's not fully realized yet, but the architecture is built for it.

ARES sees the game through a perception filter whose accuracy is data-driven per difficulty level; and its scoring noise, goal-switching inertia, and concurrent-plan caps are all dialed in per difficulty as well. However, this has a caveat: eliminating the AI’s need for cheats does not mean that we won’t give them some anyway. We envision a future of Sins' AI where no difficulty needs cheats, but some may be given anyway to further increase difficulty. In short, impossible truly becomes impossible. 

What Makes ARES Tick?

So how does this new layer work? Here's a quick overview of what happens every time ARES thinks:

  1. Perception: Everything ARES learns about the world passes through a per-difficulty perception filter: Fog of war, intel staleness, capability flags for things like, "Can I detect chokepoints?" or, "Can I estimate enemy income?" Lower difficulties see less and guess worse.

  2. World State: ARES rebuilds a single snapshot of where it’s at: Estimated game phase (rough boundaries, not strictly defined), economic phase, peak vs. current income, fleet strength relative to neighbors, supply utilization, attrition, front lines, chokepoints, expansion candidates, per-enemy threat assessments, and which gravity wells border whom.

  3. Engagement Appetite: Based on the world state, ARES decides how aggressive it feels: willingness to commit forces, tolerance for losses, appetite for risk. This number feeds into how it scores its options next.

  4. Goal Planning: ARES scores its candidate goals: Military ("capture this," "defend that," "destroy their fleet") and economic ("expand here," "establish a factory world"); ARES then picks the best ones up to its concurrent-goal caps and commits. Once a goal is active, it persists until completed, abandoned for cause, or dominated by something dramatically better.

  5. Directives: ARES writes orders into a per-domain output bus: Attack directives, defense directives, fleet composition directives, economy directives, research directives, colonize directives, diplomacy directives; and the pre-existing controllers then consume them. Directives are rebuilt every tick; they are guidance, not state. The persistent state lives in the goals.

  6. Feedback: Before scoring next tick's goals, ARES reads back what the controllers couldn't do last tick: Research it couldn't afford, structures blocked by missing infrastructure slots, ships that needed a research subject the AI hadn't unlocked, etc. ARES then folds those signals into the next round of planning. This is what closes the loop. The strategic brain learns from the executors instead of just shouting at them.

The result, is an AI that picks a recognizable plan, commits to it, builds the economy that plan needs, builds the fleet that plan needs, and presses the plan instead of getting distracted. You can tell from watching the map. Fleets converge on real objectives. Defense forms up at the right gravity wells. Research advances along a coherent track. Pressure mounts in one place instead of dribbling everywhere.

What’s Still Being Worked on?

ARES is in the player's hands as an opt-in because it's functional and competitive, not because it's finished.

Here’s an honest list of what we know still needs improvement:

  • Late-game fleet composition

    • Capital and super-capital build targets flow through correctly, but the AI is still under-investing in titans and command ships relative to where a strong human player would be by the established and late phases. We want the AI's late-game fleets to look like late-game fleets, not just to be larger early-game ones. Expect updates on this very soon.

  • Fleet engagement commitment

    • ARES decides when to attack pretty well, but its in-combat commitment, "Do I press, do I disengage, do I reinforce?",  is still leaning on the existing fleet behavior. We have a phase of work planned to bring that decision under ARES as well, so the fleet doesn't break off when ARES wants it to push.

  • Econ-to-war signaling

    • Economic state already informs which military goals look attractive, but there are patterns the AI still misses (e.g. recognizing when an enemy's economy is brittle enough to be worth a targeted strike, rather than a frontal assault).

  • Difficulty as cognition

    • As discussed, we want difficulty to scale ARES's quality of thought more than its income. Some of this is wired (perception, scoring noise, concurrent goals, re-evaluation cadence), but more difficulty-scaled tuning is still ahead of us.

  • Team play

    • ARES is an angry fella. It really struggles to recognize the difference between an ally and an enemy, and in team games, can generate war goals against allies; and wind up getting stuck on an unresolvable goal. This should be resolved, but, keep an eye out!

  • Tuning, broadly

    • Every number in ARES - scoring weights, posture thresholds, attrition limits, role templates, goal caps, demand profiles - is data-driven precisely so we can keep tuning during opt-in based on what we see in real games.

  • Multiplayer

    • Multiplayer works, however, there are issues with hot-joining at the moment causing desyncs. Do so at your own risk. Everything else: single-player, full multiplayer from the lobby, is fair game.

Conclusion

ARES is going to keep evolving throughout the opt-in period and beyond. We're not shipping this and walking away. We'll be iterating on it in public, based on what you tell us and what we see in our own playtesting. That's the whole point of the opt-in: not for you to play a finished feature, but to help us find the rough edges. The games where ARES felt sharp, the games where it did something baffling, the difficulty that felt miscalibrated, the faction that didn't feel like itself. All of that is gold to us right now.

There's no substitute for real games. If you've got a save or replay where ARES did something especially smart or especially stupid, send it our way, ideally in the Discord.

Thank you for your continued support and we’re eager to get your feedback!

sins dev journals
7,888 views 5 replies | Pinned
Reply #1 from Sins2 Forums Top

I think the core problem for the AI or 'controllers' lay in that it does NOT understand core concepts of the game, about what is happening.
I've long 'farmed out' the impossible AI completion marks on every single map. I've only build larger fleets (above 400 fleet supply), because I had the resources, but definitely DID NOT need them, even on larger maps, where the AI clearly had time to develop.
My fleet 'concept' is extreme sustainability (Vasari Alliance - Skirantra Carrier - Nanite Armor 'spam' with capital only). I know the counter to my fleet, the AI doesn't even seems to be aware of the concept of having fleet specializing around some kind of synergy or effect.

Tbh, the AI doesn't seem to be aware of anything, I don't see how those individual 'controllers' actions make any sense, it has fked up planets, no practical economy, resource flow management, it just builds the first thing available to be built for an 'effect', most planet items seems to be useful in a vacuum, but most are completely useless in practice, considering the limited planet and capital ship item slots; a 'player' will always end up having planet templates and capital ship item templates (at least, I do, because most items just don't provide any significant value compared to the items I end up using), differing 1, in extreme cases, maybe 2 items... the AI just slams the cheapest and most useless items available at the moment.
You could have long solved this problem with the AIs, if you've actually looked at the practical effect of the stuff you've put into the game. You'd only need about 3 game testers (now 4 with Eidolon), 1 for each race, maybe 1 for each faction. Specialize them on testing the game flow, rush potential, late game potential, fleet focus (like advent heavy CC focus, my Vasari immortal fleet composition), item synergy, final planet layouts, economic and military templates for planets, etc.
Then feed the effective templates into your 'controllers' as the 'goal'... voala, you have strong, itemized capital ships, actual synergistic fleets, economically functional planets with actually useful planet... heck, you'd even solve some in-race/in-faction imbalances among their own stuff, because you'd see how little certain stuff is actually used in practice, so you could make value tuning or complete item reworks so they become actually usefull.

It's insane how you're seemingly not willing to do the most sensible thing for game development: self-checking.

Based on what you've written about this ARES thing, this whole concept seems to ignore the facts, that even the individual decisions for cerain controllers make no sense. If the planet builder/ economist refuses to build asteroid miners with all the insane AI resources after 2-3 hours in the game or refuses to expand/attack towards completely undefended enemy or neutral sectors and just brainlessly suicides into enemy defenses... you have much deeper problems with the AI, than just lacking long term, strategical thinking. A simple 'upper layer' to synchronize goalmaking won't solve the problem of having an incompetent, drooling economic handler or a braindead military commander, that just wastes resources, suiciding into heavily defended planets and not even being able to comprehend what a specialized fleet is.

You could have even made the AI to build micro-fleets based on function, like support/ light/medium/heavy/missile weaponry and such and deploy them accordingly, moving them separately even to it can have a 'raider' fleet to harass new espansions of the enemy and a main combat fleet with a frontline and a backline fleet at least and simply giving an 'if' to the controller to not build supports if there are nothing to support, aka frontline is dead or has low numbers, then focus on rebulding those numbers over building ships, that would just support (heal/buff) an actual combat fleet.

I'm really hopeful about the developments of ARES, but I also have my worries, that it'll be just an immense flop and would be considered a waste of resources from the development part, because you've failed to recognize these much deeper issues about the AI, which I've already provided really simple concepts to fix.

Your welcome for freeloading on ideas, I don't care, I just like this game and want it to have a functional AI :)

Reply #2 from Stardock Forums Top

The observations and the fix are two different things and I only agree with one of them. An AI that stacks the cheapest item it can afford and then walks a fleet into a defended planet is not doing well, and nobody sensible is going to argue otherwise.

Where it falls down is the templates. You want testers to work out the strongest build per race and then hand those to the AI as its goal. What you get out of that is an opponent that plays one way competently, and the first person who works out what that way is has beaten it permanently. That is roughly what Rebellion turned into for me. Its AI was not stupid, it was predictable, and you only have to solve predictable once.

The thing you are describing that actually bothers me is not that it picks the wrong build. It is that it picks the same build no matter what I am doing. Give it a perfect template and it still will not notice I have left a second front undefended for twenty minutes.

The testing is worth doing anyway, just not for the reason you gave it. If four people play each race seriously and half the items never get picked, you have found an item problem, and I will leave the arguing about which items to somebody else. That is a balance pass with an AI justification stapled to the front of it.

+1 Loading…
Object Desktop - Get all of our enhancement utilities in one suite
Reply #3 from Sins2 Forums Top

I understand your point, but please you also try to understand, that I did not make any restrictions about giving the AI only a single template.
The in-house testers could reveal a LOT of things, that could be improved in the game (like useless items, when there could/should be actual options to pick), ships, that are just simply not worth using in any situation (don't know what's it like in multiplayer, I have my personal reasons I don't want to play the game that way), if there are ships like that or te opposite, ships that are used in extreme quantity, because of their sheer general power. These things can easily be explored with a small in-house test group. But that's just an idea on how to get a significant amount of information about the state of the game. They can thest by whatever standards, races/ factions rush capability, specific mechanics under- or overperforming, etc.
I've just provided an idea.
If we realy want to expand the topic of templates, not just the rudimentary idea of having them, then for example:
 - AI's could 'flag' a planet as frontline/ backline, frontlines aren't realy to be subcategorized significantly more, than that, but maybe we can go with supply node and fortress node (hope the names are telling enough), so the AI could define worlds to that and have a priority list of items for those specializations in general. Then for backline, there could be loads of main and sub specializations, they could create a fully committed extreme specialized world for exotics, or do a more general eco world setup, which could simply depend on the empire size, like above 15 planets, it will build an actual refinery only world (or even multiple ones, but the 15th colonized world could be a trigger for it), but before that, it will use more general eco worlds with 1-1 refinereries here and there.
A template isn't that complex to not have (if there could be) more, than one, like ice and crystal worlds could use the same templates, but ocean and earth-like planes a different, but both would be within the economic type.
I saw how these things are defined in the files, it wouldn't take THAT long with a dedicated coding personnel.
Ships could have semi-specialized roles through items too, like Kortuls can be specialized to be a more supportive (Weapon Jam + disruption) or more damage (Power Surge + disruption) or a (personally non-recommended) shock type with PS + Jam, while AM lasts, it provides strong support and high dps, but eats through AM quickly and then the whole flet's relative power level dropps significantly, all of the types could use slightly different 'end stage' item loadouts.

I'm not saying, that every single ship and planet should get a race/faction specific predefined loadout, but some specialized direction would definitely help the AI to build an effective support capital ship or an actually functional economic world.
How detailed the templates are ending up is entirely the decision of the developers, I'm just literally giving them free ideas.

I also have a methodology for testing stuff actively on the AI, simply they play against each other with the same race/faction they are playing in a team and they are doing their build they've theorycrafted recently for testing, or testing a fleet setup or whatever they think, could be an effective strategy for that faction. Meanwhile the AI does it's own stuff and you can easily see in action how and what the AI does, even analyze it in a replay.

I'm not saying it's not a lot of work, but this methodology could easily reveal problems, that are plagueing the game currently in item-balance within faction, cross faction and the AI behaviour.

I'm just afraid, because the original post talks about the 'Controllers' as if they're actually working well individually, but they're clearly not. And if the devs are going to create a solution, that doesn't actually resolve the problems with the controllers NOT actually doing well even within their 'own little world', the devs couldd waste an immense amount of resources, while not achieving proper/expected results.
Which could, in the long run, lead to the abandonment of the game by both the disappointed players and the developers, because the game doesn't generate enough profit to make it worth keeping the staff for it.

Also, in terms of fleet templates, I've actually gave you the method right away on how to allow the AI to create variety already, give it templates for 'micro-fleets' of frontline, support, artillery, raid, etc. AND templates for playstyle in each faction, that combines these micro-fleets in different setups. The ARES then can add an extra layer of variety of slightly overriding protocols, like making a support fleet follow a raider fleet instead of the 2 frontline and 1 artillery fleets, or creating an extra support fleet, so 2 frontlines can be supported to have singificantly better longevity/effective strength.

My ideas are nothing more, than just LEGO pieces, you can gave the AI commands to build whatever you want from them, but you at least have to define what those pieces are, so then you can actually create the plan to build something out of those pieces.

I hope my concepts are understandable, I'm prone to rabble a lot :)

Reply #4 from Stardock Forums Top

That is fair, and I argued against a smaller version of what you said. A library of templates with a layer choosing between them is not one optimal build, and it does answer most of what I meant by playing one way.

Where I would still push is on where the variety comes from. Everything you have described is triggered by the AI's own state. Fifteen planets, so build the refinery world. Two frontline fleets, so add a support fleet. That is a far bigger repertoire than it has now and it would take longer to work through, but none of it is looking at me, so it is still something you learn once.

What made me stop playing against it was never a bad build. It was doing the same thing three games running and never once being made to stop.

Your point about the controllers is the strongest thing in your post and I think you are right about it. If the individual pieces are making decisions that do not hold up on their own, a layer above them coordinating those decisions inherits the problem instead of fixing it. That holds whatever happens with templates, and it is a better argument than the one I was making.

Reply #5 from Sins2 Forums Top

My ideas are just a starting point in fixing the issues, even if partial or temporal, I don't see the the code related to it and don't know the actual, detailed decisionmaking guidelines these 'controllers' are making their decisions, I only see the end results and can only deduce the possible reasons behind those bad decisions, then trying to provide the above mentioned fixes I think, could be a starting point.

My bigest worry is about the opening post's statement about the individual 'controllers' working well...

I'm playing on low fleet supply and super slow economy and everything, except the actual ship movements... so the AI clearly has ample time to react, calculate incoming damage on the capital ships, calculate how much effective damage it dealt, how much it recives in a set timeframe and decide if it's worth continuing the fight or even just retreat with a capital ship... there are no triggers outside of actually reaching crippled hull with a capitalship, when it's already too late.
It literally just builds a single fleet and attack moves it into a planet and every ship in the fleet is just attacking the first enemy in range.
It is definitely NOT something that can even remotely be considered 'doing good in it's own world' in terms of fleet control/ making tactical decisions or anything... Advent capitals autocasting Pacify on focus fired targets and other stuff are just clear indications of it not working well. And there are insane amounts of examples, where it's clearly just one 'controller', that could affect the decision and it's braindead.

I know the limits of programming input, but I also know that stuff can be reworked so they work better.

In Sins 1 I've made a mod for myself, where I've analyzed the AI's ability triggers and reworked abilities so they are less wasted by autocasting -> making them much better in the hands of the AI -> making the AI more challangint to defeat, and it worked. Example: Ankylon Titan has a disruption field, that has a long CD and only puts a debuff on the initially close targets and the trigger is to have 3 or 5 targets in range... so you can have a 50 ship fleet engagement and the AI's titan will waste an insane AoE damage and controll spell (ability use blocker) on 3-5 ships on 'contact' of the enemy fleet. My solution -> I found mechanics in other abilities, like one, that makes an ability to be activatable, you activate it, and you have a constant effect, then deactivate it and you lose the effect. I've combined it with self debuffing (AM drain/ sec). So I've reworked the AoE one time long CD debuff, to be a constant buff on the Titan, that drains it's AM, has a low initial cost to be activated and once active, it generates the same AoE field every 5 sec, but the duration of the debuff is 10 sec instead of 60 or something.
The actual combat effectiveness was the same, enemy could move out of the range and 'shake off' the effect if it wants to, but the Ankylon can also follow up and move in range to apply the debuff AoE over and over again to more ships in range.
The ability's cost or actual combat effect did not change, just a usability for AI autocast became less 'exploitable' (not, that I needed to exploit it, my problem was that it autocasted it dumbly and I've essentially made the ability a bit more... dumb-proof).

So the main idea with providing actual templates to work for for the AI or these 'controllers' is to give some dumb-proofing. It probably won't be immune to long-term predictability, although there aren't realy that many choices in the game anyways, so with a good pool it can work pull from, could achieve a state, that would at least make it act sensible, even if not smart, at least it won't be able to do realy dumb and wastefull things.