You may have seen Grand Theft Auto V realism mods before, but probably not as effective as this. Kotakureports that researchers Stephan Richter, Hassan Abu AlHaija and Vladlen Koltun have developed a technique that adds an uncanny level of photorealism to the game using machine learning. The method not only adopts more natural colors, but improves reflections, tweaks road texture smoothness and otherwise adds subtleties to GTA V's normally hyper-vivid look.

The visual upgrade required a new approach to AI enhancement. While the concept of using real-world footage to guide the algorithms isn't new, the researchers found that existing methods either produced artifacts (and were frequently "unstable") or were too slow to be usable. There was often a wide gap between the footage used to train the AI and the in-game scenes. The new method grabs similar-looking patches to provide better reference points for enhancement, such as similar-looking cars and people, while keeping the frame rates relatively high.

You're not about to install a mod like this any time soon. While the scientists found their approach to be the strongest and most consistent of any AI-based system they'd seen, its photorealism is still dictated by the availability of samples. You'd need much, much more footage to improve indoor scenes and otherwise see the upgrades in every aspect of GTA V's gameplay. And there's still a significant performance hit — the unoptimized code needs half a second of inference, and that's while using a top-tier GeForce RTX 3090 graphics card.

The developers believe you could tightly integrate their machine learning system into game engines to speed it up, though. In that sense, this may represent the future of game graphics. Just as technology like NVIDIA's DLSS is making 4K gaming more viable, you could see games routinely lean on AI to add a photorealistic look that would otherwise be difficult to achieve.