A new patent from Microsoft shows how they want to record your facial expressions, show them to an AI, and have it redesign the game you're playing. It's not only your face they want, but also any words you might be muttering, growls you might be making, or, dare I say, even purrs.
The broader technology described in the patent is a process for game designers to prompt an AI to build out the narrative elements of a game, including objectives, sidequests, NPCs, dialogue, and items. Rather than stop at that first pass, the process would continue when players are in the game, analysing how they're playing to then generate new content based on their actions and reactions.
A quick peek behind the curtain: I, too, am using this technology and the rest of this article has been rewritten to match exactly the facial expression you are making right… now.
US patent 20260183672A1, pithily titled Generative Narrative Game Experience With Player Feedback, doesn't describe a technology Microsoft have already made – it is more a case of describing a process of how this tech would work if they were to make it. As ever with patent documents, they are as much a case of laying the groundwork to say "I thought of that first" to a judge when one of your competitors does a thing, as much as they are something you actually intend to do. They also often have the quality of talking to a child describing the spaceship they would make if they were given half the chance – "...and its legs have pipes filled with jelly, so when it lands it can turn the ground into trampolines". As such, you can't read a patent document as the company has done a thing so much as the company has thought about doing a thing.
Anyway, onto the thing Microsoft wants everyone to know they have thought about.
The patent describes a few ways the process would work. In one, it's predominantly on the dev side. The patent suggests that a designer could prompt an AI, using written input, images, and audio files, and the engine would then create characters, narratives, and worlds "at a high level" that the designer would finesse. That finessing would either be by directly editing what the model created or by giving it more prompts to refine what was in front of them. Once the designer was happy with what they had made, they could implement it within a game.
The authors suggest the prompt could be "merely a general theme, idea, or goal for the game or characterize the general nature of the game, or it may include some narrative details, such as main characters, a basic storyline, etc." The model would then go on to create game content – NPCs, items, dialogue, lore descriptions, and objectives and sidequests. This would then be given back to the designer as "a visual representation of the game [...] in the form of text, images, and/or graphs".
Personally, I find the vagueness here delightful. All I can picture is the creators of Deep Thought, the massive supercomputer in the Hitchhiker's Guide to the Galaxy, prompting the machine for the answer to the ultimate question: Life! The Universe! Everything! And receiving the answer 42. I can imagine a designer sitting at their desktop, writing 'Computer, build me a game about the theme of grief', only to be presented with a flowchart 'Main character loves person > loved person dies > Main character = sad". Thank you, computer.
سەرچاوەی فەرمی بە زمانی ئینگلیزی: https://www.rockpapershotgun.com/microsoft-want-to-record-your-face-and-feed-it-to-a-generative-ai-telling-it-to-go-off-and-design-a-game-from-your-frown
