Among the responses to Wharton Business School’s final MBA exam lay a “superbly explained” essay whose author relied less on critical thinking than it did on artificial intelligence. The mastermind behind the essay was ChatGPT, OpenAI’s new cutting-edge machine-learning Generative Pre-trained Transformer that operates on trillions of bytes of training data.

ChatGPT has not only written college essays comparable to human-written ones, but it has also passed multiple law school exams, the U.S. Medical Licensing Exam and the AP Computer Science A test.

Since becoming free to the public in November 2022, the AI writing assistant is also increasingly being used in Paly classrooms for discussion board posts, English essays and science Claim-Evidence-Reasoning responses, sparking discussion over where to draw the line between ChatGPT as a plagiarism tool and a useful educational one.

ChatGPT emulates human language

In 2015, Tesla CEO Elon Musk and former Stanford student Sam Altman founded OpenAI, ChatGPT’s parent research institution.

Their focus was on creating Generative Pre-trained Transformers, models trained on large swaths of text data to emulate human language patterns that have evolved into ChatGPT. OpenAI’s GPT models are designed to produce human-like text in widely varying contexts.

Unlike pre-programmed virtual assistants like Siri that search the internet to generate responses, ChatGPT generates original, human-like responses with its predictive machine learning technology, according to OpenAI.

Laura Biester, a Ph.D. researcher in natural language processing at the University of Michigan, said ChatGPT uses a multi-layer neural network to produce an output.

“ChatGPT takes in hundreds of billions of words, and then it’s trained to essentially predict the next word in a sequence,” Biester said. “So given the sentence I’m saying right now, (the model will) try to predict each subsequent word. ChatGPT is trained to do that on this huge amount of text.”

Stanford freshman and Paly alumnus Neil Rathi, who studies human and machine language processing, said ChatGPT’s model is similar to the way humans process words using the Surprisal Theory.

“Surprisal Theory is essentially that humans are trying to predict the upcoming word in the sentence,” Rathi said. “So if I have a sentence like, ‘The children went outside to blank,’ the next most likely word in my sentence is ‘play,’ (the word) almost every human will (say) if they speak English.”

Biester said ChatGPT’s large, comprehensive training set, combined with its input of the conversation’s previous history, allows it to remember context and perform complex tasks like writing essays and translating between languages.

Despite these advances, Rathi said the threat ChatGPT poses to society is inflated.

“The public hasn’t really seen GPT, GPT-2 or GPT-3,” Rathi said. “They’ve gone from essentially nothing to ChatGPT. It’s not that meaningful if you’re a researcher, but the public has seen this massive increase in the ability of AI models, so (ChatGPT) has become this very interesting thing for people and newspapers to fixate on.

But Rathi said ChatGPT only imitates human language and does not understand human nuances like context and emotions.

“These language models are not trained to understand language the way that a human can, so what they’re doing is more of an emulation of the output that a human can do,” Rathi said. “They’re producing language that looks like a human could have produced it, but they’re not producing it in the way that a human produces it. So, they’re not necessarily the most effective ways of getting artificial general intelligence or AGI.”

Rathi said ChatGPT’s lack of human comprehension should make it evident to teachers when someone uses it for school assignments.

However, Biester said by introducing human input into the model, OpenAI has potentially introduced bias into ChatGPT.

“Anytime that you’re using human annotations for a model, there’s some opportunity for bias,” Biester said. “I don’t think OpenAI has given any information about demographic information or where the people (filtering content) are from.”

Rishi Bommasani, a Ph.D. student at Stanford University, said the type of people who label data could impact the model, specifically those who train ChatGPT to refuse to answer offensive questions as per its content filter.

“One important class of (considerations) is trying to make sure the model doesn’t generate toxic content,” Bommasani said. “An important ethical question is who labels this toxic content and under what conditions.”

Usages

As an artificial intelligence tool that generates readable text with a human-like tone, ChatGPT can complete assignments on everything from understanding and developing code to composing a polished English essay by following a given prompt.

Biester said the tool can be used like how a calculator is used in math since the emergence of new technology doesn’t cause the fundamentals of a subject to go away.

“Even if you assume that students have access to a calculator, they certainly still need to learn and (understand) the core concepts to do well on the exams,” Biester said.

And junior Mihir Menon said he thinks ChatGPT can be a negative or positive tool depending on how it is used and regulated.

“While the calculator may have stripped away the challenge of manual arithmetic, it has allowed students to tackle more advanced or complex math topics much more efficiently,” Menon said. “So, if we are able to leverage ChatGPT-type tech to handle surface-level thinking so that students can move on to (more) challenging topics, it will be beneficial.”

As a computer science instructor, though, Biester said she is concerned students might use ChatGPT to write code for short programming assignments.

“We’ll need to change how we’re doing things because if you’re able to take output from ChatGPT and run it against a bunch of test cases, that solves your whole homework problem,” Biester said. “It makes it so much harder for students to stay motivated (to complete their own work).”

Regardless of the program’s ability to listen to instructions and follow prompts, Biester said ChatGPT has its downsides, especially because it cannot mimic human creativity.

“There’s still a lot of creativity that’s needed in computer science, and ChatGPT only focuses on how to write a small function rather than the architecture of the overall system you’re building,” Biester said. “So I don’t think we’re at the point now where it is not going to be creating the next new algorithm, and there’s definitely still a need for people who are trained.”

Rathi also said ChatGPT is a useful tool for solving simple problems but does not replace human cognizance.

“Unfortunately, when you ask ChatGPT a math question, you aren’t going to be able to control the output because all it can do is immediately give you the answer, so you don’t have to do any work,” Rathi said. “I’ve asked it to explain or provide the first step to help me work through the problem, but it cannot listen to those instructions to act in the role of a tutor.”

School reflects on applications

Though ChatGPT has limitations, it is still popular among students. According to a Schoology survey conducted by The Campanile, 84.3% of 70 Paly students have heard of ChatGPT and 50% have used it for either recreational purposes or school assignments.

With ChatGPT becoming increasingly prevalent in the classroom, teachers say they have had to reevaluate their curriculum and teaching methods, particularly in the English curriculum where teaching methods focus on establishing a defensible argument and a strong line of reasoning with specific contextualization.

English teacher Keith Tocci said while ChatGPT tends to write generic responses that lack sophistication, at some point in their life, students will face a situation where they have to respond to something new.

“AI can’t necessarily do that because it’s working with what has been done before,” Tocci said. “Even if it can produce (written work), I would not trust a robot to have as nuanced (of) a response.”

While ChatGPT-generated essays may lack sophistication, sophomore Keerath Pujji said it can be a useful resource to provide feedback on paragraph structure and grammar.

“If people are using it to further their writing skills and gain new vocabulary and sentence structure, it could be good,” Pujji said. “But otherwise, I don’t see students using it in a way that would be beneficial to their learning.”

Pujji also said AI could change teaching in a way which disadvantages students, if teachers start using it to create lesson plans and grade writing assignments.

“Personally, I think this takes away the personal aspect of teaching, and it could lead to the future of education being completely AI-based,” Pujji said.

Accessibility increases plagiarism

Because ChatGPT does not copy from any source, it is difficult to detect the use of AI in writing with traditional methods such as Turnitin.com’s Similarity Report. Because of this, English Instructional Lead Shirley Tokheim said plagiarism is a significant challenge with ChatGPT.

Tokheim said the English department is exploring alternative ways to assess student writing to avoid the risk of plagiarism.

“We’ve talked a lot about different options and focusing more on the process or having students write their essays in class or write pieces of them in class by hand,” Tokheim said. “The biggest challenge right now is to figure out what works and what doesn’t, especially given that we’ve spent the past few years since entering COVID-19 developing all the online tools for assessing and writing.”

Tokheim said it is also important for students to be authentic with their work and challenge themselves as writers to build their confidence.

“Don’t be so quick to give up your ability to think critically,” Tokheim said. “Don’t turn it over to a machine. Really hold on to your willingness to struggle with knowledge and writing and presenting your ideas in a way that is hard. Don’t give that up so quickly, because then you really lose that ability to think clearly. That is something writing helps students do.”

To help discourage plagiarism with AI, Pujji said the honor code should prohibit the use of AI assistants.

“I would probably consider using ChatGPT to brainstorm ideas or to create a very basic outline for an essay, but anything beyond that is unethical,” Pujji said. “Without putting it as academic dishonesty, students wouldn’t be punished for using AI to write their essays, and this would open up the stage for many more Paly students to use it freely.”

And Menon said using text-generation models, even when allowed, should be cited.

“Even when it is allowed, everyone should acknowledge if they used it — similar to citing a source,” Menon said. “Teachers should know whether or not their students’ work was aided by ChatGPT.”

Curriculum should incorporate AI

The breakthrough launch of ChatGPT has fueled debates over how to wield the power of AI in classrooms.

Though it may be too early to see the full repercussions of ChatGPT in school systems, Menon said high school curricula should adapt to the growing technological advancements and should maneuver around the use of AI.

“Assignments, essays, problems, etc. should be restructured so it wouldn’t be beneficial to use ChatGPT,” Menon said. “Additionally, all classes could shift toward more discussion or presentation-based learning, which is an example of (a space) where ChatGPT wouldn’t be useful.”

Bommasani, the former head teaching assistant of multiple language learning classes at Stanford, said ChatGPT could affect how computer science curriculum is taught at the university.

“The concepts we’re trying to teach won’t really change, but how we teach it might change more significantly,” Bommasani said. “One of the great things in computer science is you can teach the same concept in many different ways, so we’ll learn some new ways of teaching (concepts) inspired by ChatGPT out of necessity because the old way is maybe less effective.”

However, Rathi said the capabilities of ChatGPT and AI chatbots are not pressing problems since these technological models will have major limitations for the foreseeable future.

“It’s not something to necessarily be worried about right now, since (when) people are using ChatGPT, you can tell they’re using it,” Rathi said. “But if you give it a couple of years, AI is progressing so fast that we might get to a point where it is going to become useful. At the same time, AI has been progressing at a high rate because (developers) have been adding more data (to) these models, but there’s going to be a ceiling at which either we have no more data we can give them or the amount of data is too saturated, and we can’t get better performing models.”