Study examines productivity effects of ChatGPT when used by college-educated professionals

A pair of economists at MIT have conducted an experiment designed to determine if the use of ChatGPT by college-educated professionals can make them more productive. In their study, reported in the journal Science, Shakked Noy and Whitney Zhang designed and conducted a study in which college-educated professionals engaged in incentivized writing tasks.

In the eight months since ChatGPT became mainstream, it has incited a storm of debate. Some have suggested AI-based applications will make life easier because it can carry out tasks that others do not want to do. Others suggest that such applications are going to make many jobs obsolete. This idea is not new. What is new is that this time, job losses could occur in the professional sector rather than in sectors involving labor-intensive tasks.

In this new effort, the researchers noted that many of their colleagues were using ChatGPT to improve their productivity in writing projects. Some had suggested that using ChatGPT increased the quality of their written work, as well. Thus, rather than losing their jobs, the use of AI appeared to be making them better at their jobs.

The researchers wondered if such use was widespread among college-educated professionals. To find out, they designed and carried out an experiment in which 453 volunteers in such positions completed two types of writing assignments, a press release and a policy report, with the option of using ChatGPT as an assistant. Eighty percent of them chose to do so. A second group of peers reviewed the work as a means of measuring productivity and quality.

The researchers found that volunteers using ChatGPT took 40% less time to complete their assignments than those who did not use the app. They also found that those who used the app produced results that were judged to be 18% higher in quality. The researchers acknowledge that they did not conduct fact-checking on the writing produced by the volunteers; thus, it is not known if the increase in efficiency and quality came at the cost of accuracy.

© 2023 Science X Network