The 2022 surge in generative artificial intelligence (GenAI), marked by the launch of ChatGPT, continues to bring critical considerations to how educators assess learning. Generative AI can create articulate, logical, and reflective text, as well as images, code, audio and video (UNESCO, 2024). How individuals and institutions choose to address and adapt to this changing technology is both a personal and organizational consideration (MacPherson, 2023). 

Well-designed assessments are essential for determining if students have learned. Traditional assessment methods, such as essays and multiple-choice exams, have long been valued. However, they come with limitations. These types of assessments require a considerable time to develop, provide only a snapshot of performance, and can sometimes lack flexibility and authenticity (Swiecki et al., 2022). With the increasing use of AI tools, new challenges arise, especially in maintaining academic integrity, which prompts a reevaluation of assessment methods. This blog proposes alternative assessment methods and offers ways that educators can make the most of GenAI. 

GenAI Recap

GenAI is a kind of artificial intelligence that produces content by drawing on extensive datasets from various sources. Scroll through the timeline to see the history of AI.

AI imitates human abilities, with wide-ranging capacities to produce text, images, videos, music, and software codes. GenAI generates new content in response to prompts written in a conversation style. It builds content by statistically analyzing the distributions of words, pixels, or other elements, identifying common patterns, and replicating them (for example, determining which words typically follow others). GenAI outputs often look reasonable but cannot be trusted to be accurate, appropriate, or ethical. Errors will be overlooked unless the user has a substantial knowledge of the topic in question (UNESCO, 2024). Some common capabilities and limitations are summarized below.  

Capabilities	Limitations
Crafts writing (text: prose, poetry, dialogue, code).	Produces hallucinations (confidently states things that aren’t true).
Provides examples and references.	References may be ‘hallucinated’.
Creates questions and outlines.	The free version of ChatGPT lacks web access, so it lacks access to resources (post 2021).
Summarizes text.	The biases present in the training data are mirrored in the responses generated.
Provides feedback on text, including editing, clarity, and structure.	Responses may differ depending on how the user’s prompt is worded and presented.
Explains concepts and ideas at various levels of comprehension.	When using ChatGPT 3 (the free version), users may encounter delays in access during periods of high demand. Bing and ChatGPT 4 do not face such delays.
Translates text to and from several languages.	Plagiarism checkers are not effective in detecting content generated by AI because of their dynamic nature and complexity.
Functions like a conversation, building outputs through an iterative process, which builds upon previous prompts.	GenAI generates content based on patterns in the training data but lacks common sense and contextual understanding.

GenAI in Postsecondary Education: Update

By early 2023, institutional statements were released across Canada, framed in academic integrity and cautiously approaching GenAI. Later in 2023, institutions began accepting GenAI. Now, many Canadian higher education institutions have resources and guidance about how to (re)design curricula and assessments to integrate GenAI and prevent academic misconduct. The guidance includes incorporating language in course outlines or syllabi (Desforges & Usher, 2023). In alignment with this process, Vancouver Island University (VIU) provides the following guidelines and resources.

Designing Assessment with GenAI

Postsecondary educators use GenAI tools to enhance instructional practices, focusing predominantly on content development, activity creation, rubric design, and assessments. Examples include question creation, fictional case studies and learning scenarios. Some faculty encourage students to use GenAI tools as a tutor or writing aid to proofread their work. In addition, faculty members are promoting critical analysis skills by tasking students with evaluating ChatGPT responses to questions for quality, bias, and accuracy (Veletsianos, 2023; Robert, 2024).

---

When designing assessments, consider the following principles.

• Authentic assessment: Does the assessment promote disciplinary ways of thinking and problem-solving?
• Learner-centred: Does the assessment align with the goals, interests, and lived experiences of the learners?
• Universal Design for Learning (UDL): Are assessments designed with accessibility in mind, ensuring all learners can participate?
• Constructive alignment: Does the assessment clearly demonstrate that learners have achieved the course learning outcomes?
• Assessment for learning: Do the assessments offer opportunities to apply learning in different contexts and provide chances for further learning?

Short Term Approaches

• Invigilated and observed In-class work. Consider the potential challenges an in-class assessment could pose for students, especially those who rely on academic accommodations. For instance, a tightly timed writing task might provoke anxiety for students dealing with cognitive overload or difficulty maintaining focus (MacPherson, 2023).
• In-class work that integrates AI.
• Revise assessments to emphasize tasks AI cannot perform well.
• Revise grading schemes and rubrics (MacPherson, 2023).
• Incorporate formative assessment into your approach, and make sure to assess the process itself. Unlike traditional methods that usually focus on evaluating the final product, such as an essay, lab report, or exam paper, it’s important to also assess how students reached that point (Swiecki et al., 2022).

Redesign Assessment with AI

• Build AI literacy into assignments.
• Use AI to gather ideas for assessments.
• Generate examples.
• Design rubrics.
• Create activities, for example, case studies and learning scenarios.
• Build question banks for tests and assignments (MacPherson, 2023).

Prompting Tips to Generate Teaching and Learning Resources

GenAI prompts work best when you present a logical sequence to a practical problem.  

• Use simple clear language. 
• Include examples to guide the desired response. 
• Include context. 
• Refine and iterate as necessary. 
• Be ethical to avoid prompts that may generate inappropriate, biased or harmful content. 
• Evaluate and review the implications of responses, and take steps to confirm responses with evidence. 

The video below also shows how to use AI to make teaching easier and more effective, with example prompts and guidelines.

Invitation

Join our 1-hour online workshop on May 01, 2024, where we’ll guide you through practical and time-saving applications of AI to enhance student learning. Discover how to efficiently leverage AI tools to generate a variety of educational resources, including practice questions, reflection prompts, and interactive activities. 

References

Adobe. (2024). Firefly Image (2). https://firefly.adobe.com

Desforges, S. and Usher, A. (2023) AI in Canadian higher education-an update. AI in Canadian Higher Education-An Update | HESA. higheredstrategy.com

Hopfenbeck, T.N., Zhang, Z., Sun, S.Z., Robertson, P., & McGrane, J.A. (2023). Challenges and opportunities for classroom-based formative assessment and AI: a perspective article. Frontiers in Education.

MacPherson, P. (2023). Generative artificial intelligence in teaching and learning at McMaster University. Pressbook. Ecampus Ontario.

Robert, J. (2024) EDUCAUSE AI landscape study. EDUCAUSE. The Future of AI in Higher Education | EDUCAUSE

Swiecki, Z., Khosravi, H., Chen, G., Martinez-Maldonado, R., Lodge, J., Milligan, S., Selwyn, and Gašević, D. (2022). Assessment in the age of artificial intelligence. Computers and Education: Artificial Intelligence. Volume 3, 100075, ISSN 2666-920X, https://doi.org/10.1016/j.caeai.2022.100075

UNESCO.(2023). Guidance for generative AI in education and research. Guidance for generative AI in education and research – UNESCO Digital Library

Veletsianos, G. (2023). Generative artificial intelligence in Canadian post-secondary education: AI policies, possibilities, realities, and futures 2023 Special Topics Report. 2023-AI-Report_revised_final.pdf (d2l.com)