AI chatbot’s MBA exam pass poses test for business schools

Elon Musk has long dismissed the MBA as irrelevant or damaging, but now a company backed by the outspoken tech entrepreneur is threatening to directly undermine the value of the flagship business degree: the artificial intelligence chatbot ChatGPT.

Christian Terwiesch, a professor at the University of Pennsylvania’s Wharton, one of the oldest and most prestigious US business schools, decided to put to the test growing concerns about ChatGPT’s power, and found to his surprise that it could outperform some of the students on his operations management course, a core MBA subject.

In his white paper “Would Chat GPT3 Get a Wharton MBA?” published this week, he concluded: “Chat GPT3 would have received a B to B- grade on the exam. This has important implications for business school education,” citing the need to overhaul exam policies, curriculum design and teaching.

The chatbot, which has been temporarily overwhelmed by a surge in queries in the past few weeks, has sparked concern from many academics including those in business schools that students will use it to cheat in their essays and exams.

“I’m one of the alarmists,” said Prof Jerry Davis at the University of Michigan’s Ross Business School, which has convened a faculty meeting on Monday to discuss its implications. “Our whole enterprise in education is being challenged by this, and it’s only going to get more challenging. Time for a top-to-bottom rethink.”

Francisco Veloso, dean of Imperial College Business School in London, said: “We are having serious discussions and a working group is analysing the implications of ChatGPT and other similar tools that we know our resourceful and inventive students are using, and we will be formulating policies around that soon.”

While stressing that the growing use of AI technology was inevitable and even largely desirable, he called for clear disclosure policies in class on whether students had used ChatGPT, and predicted mitigation measures including “going back to handwritten work, as well as more oral and class — or at least synchronous — discussions.”

I was just overwhelmed by the beauty of the wording — concise, choice of words, structure. It was absolutely brilliant . . . but the math is so horrible.

Microsoft, the software giant co-founded by Bill Gates, who dropped out of Harvard University without even finishing his undergraduate degree, is currently considering a $10bn investment in OpenAI, the company behind ChatGPT on top of the $1bn it laid out in 2019.

Many are predicting the technology will radically shake up a broader range of activities beyond education, including internet searches and the world of work.

Musk himself, the founder of Tesla and one of the original funders of OpenAI, has argued that MBA graduates lack sufficient critical-thinking skills and focus too much on boardroom meetings and financial performance at the expense of getting close to product and walking the factory floor.

Ironically, Terwiesch concluded that while ChatGPT proved impressively literate and analytical in drafting answers to the questions he gave it on operations management and process analysis, its numeracy skills were far more limited. He did not test it against the full MBA curriculum, which includes marketing, finance, accounting and other subjects.

“I was just overwhelmed by the beauty of the wording — concise, choice of words, structure. It was absolutely brilliant,” he told the FT. “But the math is so horrible. The language and intuition are right, but even relatively simple middle-school math it got so wrong.”

But he stressed that it could quickly improve its answers when given clues, and more broadly the technology offered considerable scope in the future including in drafting and marking tests to free teachers for more valuable student support.

He also suggested an application of ChatGPT that could threaten the many business school alumni who pursue careers as consultants churning out reports and recommendations.

Current students could hone their judgment against the chatbot’s strong performance in “playing the role of that smart consultant (who always has an elegant answer, but oftentimes is wrong),” Terwiesch’s report said.

Kara McWilliams, head of the ETS Product Innovation Labs, which applies AI to learning and assessment and has developed tools to identify AI-generated answers, said: “We really need to embrace advanced technologies in education. Remember when the calculator came into play and there was a big fear about using it? I’m of the mind that AI isn’t going to replace people, but people who use AI are going to replace people.”

She argued that ChatGPT could help faculty with lesson planning, syllabus creation and developing notes for lectures. “They will be able to get a lot of menial higher education tasks off their plates so they can focus on learning. There is a real opportunity for enhanced personalisation.”

Andrew Karolyi, dean of Cornell University’s SC Johnson College of Business, said that while many academics had been taken by surprise by ChatGPT and that codes of conduct and academic integrity statements would need to be updated, “my own hope is that professors will bring up the topic actively in their classrooms to involve the students in framing AI as a valued learning tool.”

“One thing we all know for sure is that ChatGPT is not going away,” he said. “If anything, these AI techniques will continue to get better and better. Faculty and university administrators need to invest to educate themselves.”

ChatGPT replied to the FT that it was “unlikely” to kill the MBA.

“While AI and machine learning can automate certain tasks and make them more efficient, they do not yet have the ability to fully replicate the complex decision-making and critical-thinking skills that are developed through MBA programs,” it said. “Additionally, MBA programs provide networking opportunities and access to industry professionals that cannot be replicated by technology.”