Hello and welcome to the 59th Victoria 3 development diary!
With the release of the game just a little more than a month away (which honestly feels surreal), one of the last remaining things for us to do alongside bug fixing is to put the final polishing touches on the AI for release.
As such, this feels like a good time to talk about the AI of Victoria 3 - how it’s designed, how it operates, and what tools modders have available to tweak it.
However, I won’t really be talking about which exact amount of difficulty the AI provides or which level skill it plays at.
Firstly, because perception of AI performance is highly subjective and what is a ‘good AI’ means something different to just about every player, and second because there’s still a few issues to be ironed out before release, one example being AI-controlled countries too frequently experiencing revolutions (something that should hopefully be greatly improved now, but which has yet to be verified by QA).
So then, how does the AI in Victoria 3 work? Well, just as in our other Grand Strategy Games, the AI plays the same game you do.
It uses all the same features, faces all the same restrictions, and does not receive the advantage of any ‘cheats’ (with some exceptions if you use certain Game Rules, but more on that later in this dev diary).
In addition to this, there are four design principles that guides how the AI should act in Victoria 3:Roleplaying, not min-maxing: The AI in Victoria 3 should ‘roleplay’ rather than try to play optimally or ‘play like a player’.
AIs should not exploit the game and should be acting according to their internally set goals - as an example, it’s perfectly okay for a small German state controlled by the AI to want to be absorbed into a unified Germany, even though being annexed is clearly not the optimal way to play Victoria 3.
Transparency and clarity: The AI’s gameplay strategy and decision-making should be comprehensive to a player - if an AI dislikes the player or won’t go along with a player’s proposal, the player should be able to understand why, and the internal strategies that the AI follows according to the ‘roleplaying’ principle should be clearly shown to the player.
Moddability, within reason: The AI should be as moddable as it is possible to make it without ruining the performance of the game.
While it will likely never be possible (at least with current-gen hardware) to actually expose the AI’s core decision-making loop to modders, anywhere where we can allow modders to hook into its decisions through triggers, defines or scripted values, we should do so.
Not too random, not too deterministic:
The AI should generally make decisions based on something I like to refer to as ‘semi-randomness’: What this means is that an AI looks at the choices it can make, assigns a score to each, and then throws a set amount of randomness into the mix based on how unpredictable we want it to be for that particular type of decision.
For example, if the AI is choosing between two different choices that have base scores 50 and 100, and the randomness factor (R) for this decision is 1. 0, this means that the actual score ranges for each of the two is between (X / ( 1 + R ) ) to ( X * ( 1 + R ) ) -
ie, between 0. 5x to 2x the base score, or 25-100 vs 50-200 in this particular example.
In other words, while the AI is far more likely to pick the option scored at 100, it’s not guaranteed to do so - the option worth 50 could end up scoring higher after randomization.
However, if the AI was picking between two options scored 100 and 10 instead, it would always pick the 100-value option with a randomness of 1 as the randomness simply can’t beat out the gap in base scores.
This is a type of weighting we use in all our games that lets the AI make ‘sensible’ decisions without being completely deterministic.
But enough preamble, let’s dig into the actual AI mechanics in the game! The most central mechanic to the AI in Victoria 3 is the AI Strategies, which follows from all three of these principles.
Every AI country in the game is at all times governed by three AI Strategies - one Administrative, one Political and one Diplomatic.
Generally speaking, the Administrative strategy decides how the AI develops its economy (for example, should it focus on building plantations for cash crops or try to Industrialize? )
and balances its budget, the Political strategy determines which IGs and political reforms the AI favors, and how boldly it’s willing to pursue those political reforms, while the Diplomatic strategy determines what geopolitical direction the AI wants to go in (for instance, does it want to pursue colonization or maintain a stance of well-armed isolationism? ).
Which AI strategies an AI country picks is semi-randomly determined from a set of weights - as an example, countries with a powerful Intelligentsia Interest Group are more likely to pursue a Progressive Political strategy, but never guaranteed to do so.
Qing at the start of the game tends to have extremely conservative strategies - their goal is solely to maintain internal stability and defend what they already haveThe AI strategy system is much more than just broad-strokes AI guidance though: It contains all sorts of values that can be tweaked and hooked into different strategies.
Every AI has a hidden ‘default’ strategy which sets values such as how much it values an Obligation from another country, which Wargoals it’s interested in, how aggressive it should be (and against who) and what its baseline Neutrality is in Diplomatic Plays (more on that later).
These default values can then be overridden or added onto by the active AI strategies.
For example, the AI has a default scoring for the Conquer State war goal against any country against which it has the ‘Conquest’ strategic desire, based on the GDP and population of the state, whether the AI can reach it, whether it’s adjacent, whether it has a port if it’s not adjacent, and so on.
Then, if this AI country has the ‘Unify Germany’ diplomatic strategy, this strategy will add an additional value for every North and South German culture state.
Some of the scripted logic that goes into AI evaluation of how much an Obligation is worth, something I’m certain modders are going to tinker with!
I mentioned Strategic Desires - this is something like a mini-strategy, which each AI country can have against every other Diplomatically Relevant (ie, country they can diplomatically interact with based on Interests, neighbor status and so on) country.
Strategic Desires are semi-randomly determined, with weighting for each set in the AI strategies - for example, an AI country with the ‘Acquire Colonies’ AI strategy is more likely to want to conquer or subjugate Unrecognized countries than one that has ‘Maintain Power Balance’ as its AI strategy.
A major upside of this whole system is that it makes it much easier when we want the AI to pursue specific objectives, such as completing Journal Entries - instead of having to write specialized code for behaviors such as Britain seizing a treaty port from China in the Opium Wars, we simply set the appropriate weights in the AI strategies and the rest handles itself.
The Ottomans have a Belligerent attitude towards Egypt because of their very low attitude score combined with their ‘Conquer’ Strategic DesireAI strategies and Strategic Desires are of course not static across the entire campaign.
Some are set in the history files, while others are allowed to be semi-randomly determined as mentioned above, but over time you can expect them to change.
This change can be sped along by certain events - each individual AI Strategy and Strategic Desire has a ‘change points’ meter that causes it to be re-rolled when that meter hits 100, and there are a number of things that can happen to a country which immediately adds a bunch of change points and may trigger a re-roll.
For example, if a country declares bankruptcy, all other AI countries will receive a number of change points on their Strategic Desire towards the country that declared the bankruptcy and will be more likely to roll a non-friendly desire, as bankrupt countries are seen as unworthy allies and easy pickings.
Another, less dramatic example is that whenever a country successfully enacts a new law, a number of change points are added to its Political strategy.
Hitting 100 change points doesn’t necessarily mean a country’s Strategy or Strategic Desire will change though - it is entirely possible that the outcome of the ‘re-roll’ is to simply pick the same old thing over and over again, especially if the AI’s scripted preference for that strategy is high.
Now that we’ve talked about how the AI functions on a strategic level, let’s switch gears and talk about how it operates in another area, namely the economy.
The economic management of Victoria 3 can be quite complex, and as such, making AI for it is no trivial task either.
There’s a lot of factors to consider, even for the most straightforward-seeming action: As an example, let’s say the AI is considering whether to build a Tobacco plantation. Simple, right?
We just look at the estimated labor costs involved versus the current market price of Tobacco and make a determination on whether it’ll turn a profit? Well, no.
For one, that market price is going to change as soon as the new building is complete and starts adding more Sell Orders of Tobacco to the Market.
What’s more, goods substitution also comes into play here: The increased supply of Tobacco will, over time, lead to Pops buying more Tobacco as part of their Intoxicants need, increasing the demand for Tobacco while lowering it for other goods such as liquor (which then in turn would influence the AI’s decision making regarding any buildings with production methods that produce liquor).
So then, how do you account for all these factors without crashing performance every time the AI wants to build something?
The solution is a system called Spending Variables, which is a rather unassuming name for how the AI predicts and charts the course of its economy and market.
Each AI country’s Spending Variables tracks values such as the Buy Orders and Sell Orders that are going to be generated by buildings which are currently in the construction queue or in the process of hiring employees.
This means that if the AI is facing a shortage of Tools, and starts building a Tooling Workshop, it can now understand that once the Tooling Workshop is completed, the shortage of Tools will be resolved, and thus doesn’t start constructing a second or third Tooling Workshop - and can, for example, queue up a construction after the Tooling Workshop that is going to use Tools as an input, even though that would clearly be a terrible idea under the current market conditions.
The Spending Variables of course track a whole lot more data than just buy and sell orders, particularly any complex calculation that doesn’t need to be made all the time - such as determining whether the AI feels that it’s under a heightened military threat and thus should prioritize military spending.
Spending Variables are tracked and recorded even for player countries, where they are used for the purposes of prediction - in the shown example, you can tell that expanding the Wheat Farms in Peloponnese is actually going to result in reduced profits for the building due to the price impact on Grain, which means the AI knows this too!
When it comes to making spending decisions based on these variables, the AI actually uses a system that was actually inspired by the development process for Victoria 3 and the issue tracker that we use (JIRA).
The AI collects a list of all changes to the government spending it could do, in both directions.
For example, subsidizing a particular building, constructing Barracks in a particular state, or reducing the size of its Government Administration in another state.
It then assigns each of these actions a Priority and a Score.
Priorities are a tag that determines the general circumstances under which the AI will consider increasing or decreasing their spending on this particular item.
When increasing spending, the AI will always pick an item of higher priority over an item of lower priority, with the reverse being true when decreasing spending - low-priority items are cut first.
It’s quite common for the same ‘item’ (for example, the same building type in the same state) to have a different priority when it comes to increasing vs decreasing spending, as the AI should generally be reluctant to downsize a building it spent a bunch of money to construct unless it absolutely has to.
The priorities are as follows:Must-Have: This is something that the AI considers so critical it’d rather go bankrupt than be without it.
The AI will always increase spending on a Must-Have, and never decrease spending.
Must-Have is used sparingly, with an example being that the AI will never consider downsizing Government Administration buildings when in a Bureaucracy deficit, as lacking Bureaucracy decreases tax revenue and thus would likely just worsen the situation.
Should-Have: This is something that the AI believes is very important and should be acquired as soon as they have a small surplus.
The AI is only willing to cut spending on a Should-Have if they’re in debt and can’t find any other way to balance the budget.
Wants-to-Have: This is something the AI considers fairly important and will be willing to increase spending on if they have a modest surplus, but isn’t willing to go into debt for and will cut spending on if they have a budget deficit (not counting temporary expenses, of course) and aren’t sitting on a large pile of gold reserves.
Nice-to-Have: This is something the AI considers low priority and will only be willing to increase spending on if they have a large surplus and nothing better to do with their money.
They will cut spending on nice-to-haves the moment their surplus isn’t looking so good. Should-not-Have:
This is something the AI has determined it doesn’t want at all. It will not increase spending on this item under any circumstances and will cut spending on it as soon as possible.
Score is a numerical value that is semi-randomly determined and serves as a tie-breaker between two items of the same priority.
So for instance, the Belgian AI might consider Barracks in either Wallonia or Flanders and a Naval Base in Flanders to all be Should-Have, but which of the three it actually ends up increasing spending on is determined by the score of each after randomness is factored in.
Together, these two allow the AI to make decisions in a way that should neither be too deterministic nor too random, and makes it easier to handle special cases than a purely score-based system would.
AI spending data isn’t shown in the interface as we consider that to be *too much* transparency, but can be gleaned using the console.
Here, you can see that Prussia thinks it has a good surplus of Bureaucracy and doesn’t see any pressing need for more Government Administration buildingsMoving on from the economy, let’s talk about how the AI behaves in Diplomatic Plays, as this is an area where a bunch of things have happened quite recently.
If you’ve been following the official streams and various developer AARs, you may be familiar with the way that the alpha implementation AI used to decide which side to back in a Diplomatic Play: It would assign a score to each side, then add the value of a sway offer on top of that to determine if it was willing to be swayed (and sometimes join a side it strongly preferred even without being swayed).
This system has now been enhanced with a few new values that govern AI behavior in Diplomatic Plays: Neutrality, Sympathy, Confidence and Boldness.
We’ll go through each one in turn and explain how they influence AI decision-making when taking part in a Diplomatic Play. Neutrality:
This is a calculated value that determines how much an AI country wants to stay out of a particular Diplomatic Play, and is based on factors such as which Strategy the AI is pursuing, whether they’re in debt, the shape of their army, and whether they consider the conflict to even be worth caring about at all.
In order for an AI to take a side or allow themselves to be swayed, the score of joining one side must now beat out both the score of the other side and their Neutrality Score - so if Austria and Prussia are opposing each other in a play, and Russia gives Austria a score of 30 and Prussia a score of -10 while their Neutrality Score is 50, they will be unwilling to join the play on their own accord and will demand a sway from Austria worth at least 21, while for Prussia the sway offer would need to be worth at least 61.
Sympathy: This is a value that is semi-randomly determined for each non-leader participant towards the two leaders, and which is added directly to their willingness to join that side in the play.
Countries tend to have higher sympathy for the defender, lower sympathy for enemies of their allies, and higher sympathy for countries that are facing off against an enemy with high infamy.
Sympathy will also change over the course of the play: Any side which adds additional wargoals will generate sympathy for the opposing side based on the Infamy of the wargoal added, and similarly any side which receives support from an uncommitted country will generate sympathy for the other side based on how powerful that new backer is in relation to the other countries involved in the play (conversely, losing a backer increases sympathy for the side that now finds itself with less support).
With nothing more in it for them than a little-valued Obligation, Great Britain has no interest in getting involved in this West African dispute as their Neutrality is simply too highConfidence: This is a calculated value that only applies to the leader of each side of the Diplomatic Play, and determines how they feel about their chances to come out on top if the Play should escalate to war.
This is primarily determined by the relative strength and mobilization level of the two sides’ militaries, but also by factors such as how well they can bear the cost of a war, how internally divided they are and how capable they think their own national forces are to fend off incursions into their territory.
The lower the AI’s confidence, the more willing they are to back down, and the more keen they will be to try and secure additional support from uncommitted participants. Boldness:
This is a hidden value that is semi-randomly determined for each participant in the Diplomatic Play.
Countries with certain Strategies, higher-ranking countries, countries with strong militaries, and countries that are facing annexation in the Play tend to have higher Boldness.
Boldness plays a role in multiple different AI decisions such as how early it mobilizes, how willing it is to join plays without being offered anything, but the main role of Boldness is to be compared against Confidence to determine whether the AI is willing to back down rather than let the play go to war.
For this, Confidence and Boldness are added together and the lower the result, the more likely the AI is to back down.
There’s some other factors to this as well, such as how much the AI stands to lose from additional wargoals that could be pressed if the Play escalated to war, but Boldness serves as a way to provide ‘predictable randomness’ - that is, an AI Modena facing off against Austria alone would have such a bottomed-out Confidence score that it would need an very high random roll for Boldness to let the play escalate to war, even if they’re facing annexation.
Facing the prospect of war with both Prussia and Russia, there is a chance here that Austria is going to back down here if they can’t secure more support - though they might also decide that Bohemia must be defended at all costs given its integral status in the EmpireAI is a personal passion of mine (I actually started out as an AI programmer for Paradox back in 2013, almost 10 years ago now! )
and I could probably keep talking about it forever, but this dev diary is getting awfully long, so to wrap it up I just want to cover the AI Game Rules that are available in Victoria 3, since they’re a bit different from the usual difficulty settings you might be used to.
Because of the complex and interconnected nature of Victoria 3’s economy, and our general commitment to simulation, we didn’t want to simply have ‘High Difficulty’ mean that all of the AIs start producing more goods or have a bunch of income created out of nothing, or ‘Low Difficulty’ mean the same for the player.
Instead we decided to focus on how the AI interacts with the player and the rest of the world on a geopolitical level.
Following this philosophy, the two AI Game Rules we’ve added are called ‘AI Behavior towards Players’ and ‘AI Aggression’.
AI Aggression goes Low - Standard - High and is relatively straightforward: the higher this is, the more aggressively the AI will act in general towards all other countries.
It affects which Strategic Desires they tend to prefer, how likely they are to start Diplomatic Plays to carry out those desires and how willing they are to take on Infamy.
AI Behavior towards Players, on the other hand goes Lenient - Standard - Harsh and affects only how the AI views player-controlled countries and is the closest thing to a Difficulty setting that we have.
With the regular setting of Standard, the AI treats the Player the same as if they were another AI country, with no special considerations.
However, you can also set it to Lenient if you want a ‘friendlier’ experience - this will make the AI more likely to have friendly Strategic Desires towards player countries, make them more likely to support the player in Diplomatic Plays, and will make them unwilling to conquer the player’s Homelands unless they have a claim on it.
Conversely, if you’re an experienced hand at the game and want the AI to treat you like the agent of chaos that you are, you can put this setting to ‘Harsh’ - this makes the AI far more suspicious of the players’ intentions, less willing to support them in Diplomatic Plays unless they have something to very clearly gain from it, and more willing to knock the player down a peg or two should a good opportunity to do so arise.
I call this combination of AI Game Rules the ‘Metternich Mode’Alright then!
That’s it for this dev diary - I hope you find it enlightening, and I hope you find the AI in Victoria 3 fun and interesting to play with and against when the game releases.
We still have some things to fix before then, and we’re of course going to continue improving, polishing and increasing the moddability of the AI post-release as we do with all our games.
I’m particularly interested in hearing about any additional ideas for post-release AI Game Rules that the community may have, as I think that allowing players to tailor their individual AI experience is a great way to deal with some of that subjectivity I mentioned at the start of this dev diary.
For now though, farewell and join us again next week as Michael introduces us to Modding in Victoria 3!
0 Comments:
Leave a Reply