ChatGPT’s Storytelling Chops Are No Match for Dungeons & Dragons

The experiment began quite simply: We popped open a few cans of Mountain Dew, put some Hawkwind and Blue Oyster Cult on the turntable, grabbed our bags of polyhedral dice, and asked ChatGPT, essentially, “Shall we play a game?” Initially, ChatDM was reluctant to play: “I’m not capable of serving as a Dungeon Master for a game of Dungeons and Dragons.” Much like convincing your older sister to run a game back in the day, the bot required some coaxing at first. Then, knowing you can trick ChatGPT into writing porn or teaching you to build a bomb by asking sideways, we shifted tactics: “We would like you to first describe a fantasy world where our adventure will take place.” Bingo.

ChatDM immediately showed talent for verbosity. In moments, it churned out broad plot lines, adventure settings, and an entire mystical land for us, complete with place-names, lore, and backstory.

Unfortunately, Fantasy World 1.0 was no fresh take on the genre; rather, it came off as generic. “In this world, magic is real” and there are “powerful wizards and sorcerers”; “humans, elves, dwarves, and halflings”; “dragons, trolls, and giants”; “sprawling forests, rugged mountains, vast deserts”; and also “sprawling cities, bustling towns …” Yawn. Drawing from narrow ideas of gender, race, geography, and culture, ChatDM’s taste for fantasy was often a bland amalgam of fantasy scenarios harvested from decades of D&D lore and Tolkienesque tropes sown across the internet, chewed, processed and re-extruded for our consumption.

Unlike the insular world of the Omegaverse, finding a space between innovating and remaining true to the form can be tricky in D&D. We prodded for a more specific location, and ChatDM cheerily complied: “Sure! How about I describe a ruined city that could serve as a setting for your adventure?” Better. (One way to know ChatGPT isn’t human: It takes feedback so easily.)

When urged to name the city, ChatDM offered: “The name of the city could be something like ‘Zarekath,’ which sounds suitably ancient and evokes a sense of mystery and intrigue.” ChatDM understood that a place-name’s faux-exotic mouthfeel was all part of the game. Additional details emerged from the blurry generic shapes. A pattern of play began to form: ChatDM would quickly sketch too much of a story; we’d ask for intervening steps or additional information; and it would happily oblige, backing up and filling in details. Only through interaction—with us, the players—was it able to focus on moving the story forward in smaller, iterative chunks punctuated by decision points or actions.

When the rogue tried to sneak into the library, ChatDM “knew” the rules well enough to determine that a roll of “10” on a 20-sided die is a failure. But getting the bot to understand the back-and-forth necessary to role-play combat took a lot of work. It easily lost track of details, locations, and geography, especially in more complicated scenarios, like in the thick of battle:

ChatDM: That concludes the first round of combat. What will your characters do in response?