How Microsoft Flight Simulator 2024 uses machine learning AI, and how much of your data it might need

Microsoft Flight Simulator 2024 launches on Steam and the Microsoft Store today. Developed once again by Asobo – otherwise celebrated for their stinking rat hordes – it builds upon the 2020 game by “[taking] advantage of the latest technologies in simulation, cloud, machine learning, graphics and gaming”, in the words of the launch announcement release.

We’ve got a review in the works, but code has landed late, so our write-up might take a while. In the shorter term, I thought you might like to know how, exactly, MFS 2024 makes use of “machine learning” technologies, taking into account the energy cost of such wacky gadgetry and the creeping relationship between increased reliance on automated tools and laying people off. More immediately, you might like to know how much of your internet package it’ll devour as you play.

Microsoft Flight Simulator 2024 shares many of its moving parts with the 2020 game, which made use of tech from Blackshark.ai, Microsoft’s Azure tools and Bing Maps to analyse satellite and aerial imagery and generate terrain maps and objects such as buildings. While a lot of this is delivered by the initial download, some of it is streamed to you on demand.

The major change with MFS 2024 is that it streams a lot more, while simultaneously trying to offer a lot more visual opulence. “We did surveys and asked people, how can we do this – what can we improve?” MFS head Jorg Neumann told me in an interview earlier this month. “And one of the areas that was very consistent was well, it would be nice if the ground would look a little bit better. Because in MFS 2020 we had the whole world and it looked good from a certain altitude, but once you landed, it didn’t look super-detailed.”

MFS 2024 tries to address this by way of a new online machine learning platform that generates terrain detail on demand, based on rules given by the developers. “So we looked at the textures, basically, and the way it works is we had a team that marked up 28,000 tiles,” Neumann explained. “So we picked some tiles that were representative of Earth – some of them were forest, some of them were desert, some of them were rivers, those types of things, and then we mark them up and say, ‘this is gravel, this is low grass, this is bush’.

“And then you feed that into a machine learning system, and you say ‘this type of colour and this type of pattern is this’ – you feed it in, and you get everything back, you look at the results, and then you have human intervention again. You basically say: ‘Oh OK, got this wrong.’ And then you retrain it. We did that in four different stages.”

What does that process of human intervention look like in practice, I asked? What’s the trick to avoiding the kind of uncanny distortion we often see in today’s prompt-based generated art, to say nothing of the previous game’s cosmically horrible terrain deformations?

“It’s just picture analysis, really,” Neumann said. “It’s just a lookup table and you say, is this sand? We defined, I think, 26 different surface types – red sand or brown sand to asphalts, those types of things. And I don’t know about distortion but you know, you basically say, ‘oh this one is this, and this one is this’, and it got some stuff wrong sometimes, you know – it understood gravel to be sand. And then you have to retrain it a little bit.”

Once generated in response to one player’s movements, MFS 2024’s terrain data is then available to other pilots – it’s streamed into their games when they’re close by. This makes the system more efficient, and gives rise to the peculiar thought that when you fly over a remote area in the game you are bringing that location to life for everybody else.

One inevitable fear I had, on hearing about this system, is whether the expanded role of machine learning has come at the cost of jobs. While it varies by the tool, discipline and company, there’s a link between today’s AI tools and corporate cost-cutting, with larger video game publishers such as Microsoft dismissing tens of thousands of people even as they spend billions on software that theoretically automates aspects of game development, such as the creation of art assets for buildings and vehicles.

According to Neumann, however, MFS 2024’s expanded usage of machine learning has had “no impact” on the workforce. “The only thing we’ve done is grow,” he went on. “Flight Simulator 2020, it was a team of 100. Flight Simulator 2024 is 200 people at Asobo, a total of 800 people [at supporting studios].” It would not have been possible to mark up the game’s map for the generator ‘by hand’, Neumann insisted. “We would have spent our entire lives and then some just marking up terrain, which is not really the core of the thing. So it’s just a tool that helps us.”

The more immediate concern is that the reliance on streaming will require the beefiest of broadband connections. A few months ago, there was a report that MFS 2024 would require you to stream as much as 81GB an hour. Neumann told me that this was a calculation based on an unoptimised build running on the most demanding settings, and that the average data use would be “way, way less than that” – not least because most players will spend the bulk of their time high up, coasting through already-loaded “aerial data” that doesn’t have to be streamed. Asked to give a rough average figure for a player’s data use, he suggested a relatively modest 5GB an hour.

While 5GB an hour is still quite a chunk, Neumann feels the expanded streaming system is more efficient from the player’s perspective, because it means the initial download can be a lot smaller – around 50GB. It also means you won’t have to download massive expansions over the next few years – instead, the game’s representation of Earth can be overhauled and expanded remotely, with the new bits being streamed to each player as required.

“Flight Simulator 2020 was around 120 gig, and then we added all these world updates, and each world update hovers between 10 gigs and 30 gigs, so we’re now at hundreds of gigs,” he said. “And we’re going to keep going. So just imagine world update 38 in 2028, or something. A terabyte download – it just doesn’t scale.”

In practice, Nuemann noted, many MFS 2020 players didn’t feel inclined to visit the vast majority of its simulated Earth. “An airport is two gigabytes, and we have a ton of airports, right,” he said. “Under the old scheme, you had to download every airport, even though you had no intention of ever visiting this place. I know a lot of people that just fly from their local hometown to where they grew up in college. And they don’t want to explore the entire planet. They just want to hang out in the area that they know.

“Their data consumption [in MFS 2024] is going to be tiny. So we actually think we’ve saved a bunch, and that’s actually much better. The brute force huge client thing – I don’t think this really scales. Just imagine if our texture resolution goes up again.”

While Neumann’s arguments about efficiency sound reasonable to me in the abstract, it’s obviously unclear right now how players will play Microsoft Flight Simulator 2024, and how they might use or abuse its dependence on streaming generated terrain. One, more whimsical scenario that occurs to me is that disreputable pilots might decide to troll the developers by carrying out low-altitude flights en masse, increasing the burdens on the machine learning and streaming systems.

That open-endedness is worrisome because generative AI at large has a mounting energy and emissions cost, and Microsoft especially are trying to walk the line between adhering to relatively ambitious carbon negative plans while also plunging inordinate sums into technologies such as OpenAI’s various roided-up chatbots and Microsoft’s flagship Copilot app.

The most prominent generative AI tools soak up a lot of power, both in the course of “training” them on datasets with human oversight, and when actually running them on data centres. For all their rhetoric, Microsoft’s greenhouse gas emissions were actually 30 per cent higher in fiscal year 2023 than the previous year.

While Neumann didn’t share any estimates about overall energy usage in our chat, I doubt that Microsoft Flight Simulator 2024’s new machine learning and streaming systems will have an emissions footprint on par with, say, giving your keyboard a Copilot button. But it’s all part of the picture. Look out for more on the subject in a piece later this month, in which Neumann and I discuss the philosophy of flight simulation, future community-driven features, and the sheer weirdness of generating an entire planet in real time.