Since the rise in popularity of Gen AI chatbots among the youth, “just ask ChatGPT” has been a common response whenever students encounter difficulties in school. Sparked by their concerns regarding the over-reliance of Gen AI chatbots as a lifeline in educational settings, students Celene, Ednalene, Kimberley and Li Ying from Nanyang Technological University (NTU) have reached out to approximately 2,000 students since last October to ensure that students make the most out of their use of Gen AI chatbots. Their solution? CheckGPT.
CheckGPT is a communications campaign that helps students address inaccuracies in responses provided by generative artificial (Gen AI) chatbots like ChatGPT. The framework is simple – don’t just chat, CHECK! Through CheckGPT, these students have developed the 4 ‘C’s framework, which combines prompt engineering and verification.
Are chatbots truly reliable?
Granted, chatbots are able to generate quick, articulate, seemingly logical answers, to the point where it seems too good to be true. But for all the convenience that these chatbots provide, are there any downsides to this technology?
As it turns out, there are! And there have been multiple instances of it across the past few years – a lawyer in the US submitting ChatGPT’s made-up cases to court, Microsoft’s Copilot falsely branding a journalist as the offender of the crimes he covered, as well as a Yonsei University candidate using ChatGPT to cheat his admission exam but ultimately failing due to incorrect solutions provided by the chatbot. These are all cases of Gen AI chatbots hallucinating, a term referring to non-existent or false information generated by Gen AI chatbots in their responses.
Additionally, Gen AI chatbots do not see the human language, and instead only see numbers, also known as vectors. Through these vectors, chatbots can classify and identify patterns in text, before generating words based on probabilities. Ultimately, Gen AI chatbot responses are based on probabilities and not certainties, making them prone to inaccuracies!
The case for CheckGPT
Fueled by this, the CheckGPT team developed the 4 ‘C’s framework, ‘Context, Clarity, Constraints, CHECK!’
The first three Cs guide students through prompt engineering, which is the process of forming better prompts so that the Gen AI chatbot is more likely to give you an accurate answer. ‘Context’’ involves assigning the chatbot an expert, knowledgeable role and providing background information. ‘Clarity’ means providing clear, specific instructions, from specifying the task objective, to narrowing down the topic of interest, or even indicating the academic level of response. ‘Constraints’ is about establishing limitations if necessary, such as the output length and structure, or the type of information the chatbot should reference.
The last and most important ‘C’, ‘CHECK’ is the final step to preventing misinformation. Like with any information we come across, it’s crucial to cross-reference multiple other credible sources before we utilise or trust it.
Digital literacy needed more than ever
As more schools and students adopt this technology, there is a need for digital literacy classes and campaigns to not only highlight plagiarism and ethical use, but also the shortcomings of such technology and how we can better leverage it. Considering their rapid development, Gen AI chatbots may become the norm in years to come – some might even argue that they have already become the norm. CheckGPT aims to facilitate a fundamental understanding and foundation of good habits centered around the use of Gen AI chatbots.
Since the campaign’s launch in October 2024, CheckGPT has reached students through sharing sessions and roadshows at universities, polytechnics and secondary schools. Their Instagram page also shares information on Gen AI chatbots in a digestible manner for online audiences. CheckGPT has also created an informational pack as a resource for students, teachers, parents and anyone interested to learn more about Gen AI chatbots and the 4 ‘C’s framework. This informational pack is available both as an online magazine and a physical booklet that can be printed out.
The CheckGPT team believes this campaign is integral to the ongoing conversation about how Gen AI chatbots should be used in educational settings. Ultimately, CheckGPT hopes to bring to light that sometimes, the solution to a problem isn’t as simple as turning to ChatGPT. Instead, by understanding how this technology fundamentally works, students can make better use of it as a resource in their learning journeys and be equipped with the skills needed to navigate the developing scene of Gen AI.
