Watch how ChatGPT is tricked into generating Windows 95 keys

ChatGPT and other AI chatbots seem a little bit like magic sometimes. But they’re not, especially when asked to do a fairly basic computational task that you’d think they could handle with relative ease—like generate a Windows 95 license key.

From our AI shootout of ChatGPT versus Microsoft Bing versus Google Bard, we know that AI chatbots can outperform our expectations on some tasks, and surprisingly struggle with others. Enderman, a YouTuber who typically plays around with various older Windows builds, set out to see if they could generate a brand-new Windows 95 key. (This is probably illegal, but anyway…)

The obvious method is to simply ask ChatGPT to generate the requisite license keys, but OpenAI made its chatbot too ethical for that. So an alternative was necessary. As Enderman points out, Windows 95 key generation uses a known algorithm, combining various strings of numbers and letters together in a known way to create the 20-character code.

The method doesn’t appear to be all that complicated—three random characters, a year string, the letters “OEM,” and so on. The trouble, however, is simply explaining that to ChatGPT in a way that it understands, and in a way that the algorithm won’t generate bugs in return. You can see how Enderman fared in the video below (spoiler: he does succeed) but it’s worth slowing down or even pausing the video to examine the process.

ChatGPT’s struggles underscore several points: first, the AI chatbot is a large language model, which means that it was designed to parse natural language and output a result. As our shootout revealed, AI chatbots still can’t always get this right, either because the input was poor, or the algorithm simply didn’t understand it correctly. We might tolerate some inprecision in how an AI chatbot explains the causes of the American Revolution, but generating a license key is similar to a math problem: the answer is either correct, or it isn’t. Perhaps oddly, this isn’t always an AI’s strong suit.

What you might take away from all of this is what some are realizing: AIs benefit from clear, precise language inputs. If you’re a lawyer, writer, historian, or some other arts major who practices their communication skills regularly, this brave new world of AI chatbots may benefit you. Some are even adding “prompt engineer” to their resumes—and why not, when the ability to issue clear directions might up their value substantially? This YouTuber discovered this the hard way. But since the basic version of ChatGPT is free to use, why not give it a whirl yourself?