Thoughtful choices when upper-secondary school students use AI in their schoolwork
Many students use ChatGPT in their schoolwork. New research from Halmstad University shows that science and technology students in upper secondary school have good strategies in their use of AI. This is even though they overestimate their knowledge and actually have very little understanding of how such an advanced language model really works.
“Teachers and parents need to understand how students use AI because students already have access to AI tools and will use them even more in the future.”
Federico Valeri, PhD student
“Teachers and parents need to understand how students use AI because students already have access to AI tools and will use them even more in the future”, says Federico Valeri, Doctoral Student in Informatics at Halmstad University.
Because AI development is so fast, very few previous studies have focused on upper secondary school students’ use of large language models (LLMs), such as ChatGPT, in biology, physics, chemistry, engineering and mathematics. However, the researchers behind the current study were not surprised that the use of ChatGPT is proving to be widespread. In a survey, almost 90 per cent of students in science and technology programmes responded that they had used some form of generative AI tool, such as ChatGPT, in school in the past month. 35 per cent stated that they had used AI at least five times during the period.
Concepts
Over 60 percent of students use AI to understand concepts, which is significantly more than those who use AI to summarise texts. Above all, they used ChatGPT in biology.
“Summarising texts could be a way to relieve a large part of the work, but it can also be problematic for learning if reading is an important part of the learning process. However, the use of AI to support understanding concepts does not pose the same problems”, says Federico Valeri.
The survey responses show that over 62 per cent of the students feel that they have sufficient knowledge to use AI to support their studies in the natural sciences and technology subjects.
“This is remarkable, especially considering that it turned out that most people had very little understanding of how AI models actually work.”
The researchers conducted in-depth interviews with 12 students to gain more understanding of how and why the students used ChatGPT.
“Overall, the interviews showed that the students do not know much about AI, although some have a greater interest and knowledge of algorithms, mathematics and statistics. Despite this, the students had sensible and well-thought-out strategies for interacting with the tool”, says Federico Valeri and continues:
“It’s a bit surprising, we thought that good knowledge of AI would have a greater impact on students’ strategies.”
Well-known strategies
Depending on how the questions are formulated to ChatGPT, so-called prompts, the quality of the answers can vary tremendously.
“The students used what is similar to the well-known strategies for prompting. Both those who were ignorant of AI and those who were more knowledgeable had good strategies.”
Federico Valeri describes good strategies as providing a lot of context, for example that you are an upper secondary school student and that it is about the subject of biology.
“If, on the other hand, you just write ‘what is the greenhouse effect’, you don’t get such a good answer”, says Federico Valeri and continues:
“Students know that ChatGPT can make up facts, so they don’t ask pure factual questions. Students also work iteratively, that is, first they ask a question and then if they don’t get a good answer, they refine the questions gradually. It is also a good strategy.”
The students’ answers showed that they had initially used ChatGPT more like a regular search engine.
“Gradually, they used the tool in a smarter way, even though they hadn’t learned more about the tool technically. The students also stated that, if they had been in class and had more subject knowledge, they became better at using ChatGPT for school assignments.”
Text: Kristina Rörström
Photo: Dan Bergmark
About the research
A survey was answered by 450 students in March 2024 and in-depth interviews with 12 students were conducted in April 2024 at the natural science programme and the technology programme at upper secondary schools in southern Sweden. The questions were about the STEM subjects, i.e. science, technology, engineering and mathematics.
Behind the study are Federico Valeri, Doctoral Student, Pernilla Nilsson, Professor, and Anne-Marie Cederqvist, Senior Lecturer, all at Halmstad University.
Read the full article in Computers and Education: Artificial Intelligence
Exploring students' experience of ChatGPT in STEM education
External link.
The study is part of Federico Valeri’s dissertation work and is funded by the Swedish Research Council.
