AI and Educational Assessment 101
Authors: James D. Basham, Ph.D., University of Kansas, Angelica Fulchini Scruggs, Ph.D., University of Kansas, and Matthew T. Marino, Ph.D., University of Central Florida; info@ciddl.org
One of the most common inquiries CIDDL receives is about Artificial Intelligence and assessment. The integration of AI throughout the education system is affecting nearly everything educators do daily, from small tasks such as emailing students to planning instruction. But one of the largest concerns educators often have about AI is how to design effective assessments. Well, just like nearly everything else with this powerful technology, the key is to understand the basics and then design to support the most effective human interaction.
How does AI Impact Educational Assessment?
Assessment has always evolved, just slowly enough that we could pretend it wasn’t changing. AI has quickly uncovered the illusion. It doesn’t just tweak assessment; it exposes how limited many of our legacy approaches have been.
For decades, the “one test, one score, one moment” model dominated, largely because it was efficient, not because it was particularly good at capturing learning. Frameworks like Universal Design for Learning (UDL) have begun to move the field in a more positive direction, emphasizing multiple means of action and expression. Instead of asking every student to demonstrate understanding in the same way, UDL opened the door to demonstrations of learning through writing, speaking, creating, and designing. AI accelerates this shift from “standardized performance” to “authentic demonstration.”
Here’s where things start to get interesting and a little uncomfortable.
- AI is pushing us away from traditional assessments (Khlaif et al., 2025; Salinas-Navarro et al., 2024)
When students can generate a passable essay in seconds, the value of asking them to produce one under generic conditions drops quickly. This isn’t the end of the assessment, but it’s a forcing functional change in the process. Educators are being nudged (sometimes dragged) toward more meaningful tasks: applied problem-solving, iterative design, and performance-based demonstrations that require thinking, not just output. - Automated grading and scoring (with caveats) (Gardner et al., 2021; Kofinas et al., 2025)
AI can score structured responses, short answers, and even aspects of writing at scale. That’s a win for efficiency. But it’s not neutral. These systems reflect the data and assumptions on which they’re built, which means educators still need to play referee. The opportunity isn’t to replace teacher judgment, it’s to free it up for the parts that actually require expertise. - Personalized and real-time feedback (Kofinas et al., 2025)
We know feedback is a critical aspect of the learning process. Unfortunately, a single educator cannot provide immediate feedback. This is where AI starts to earn its keep. Instead of waiting days (or weeks) for feedback, students can receive immediate, targeted input while they’re still learning. That shift, from feedback as autopsy to feedback as coaching, is a big deal, especially for learners who need more scaffolding or timely reinforcement. - Self-testing and self-scoring (Francis et al., 2025).
If students are engaged with the content and want to learn, AI lowers the barrier to checking their own understanding. Practice quizzes, adaptive questioning, and instant explanations can support metacognition if students are taught how to use them well. Otherwise, it becomes another “click until correct” exercise. Tools don’t create reflection; design does.
What are the Concerns with AI in Educational Assessment?
Of course, there are real concerns that have to be acknowledged in the assessment. AI doesn’t magically make assessments better; it changes the pressure points.
- Academic integrity (Evangelista, 2025; Xia et al., 2024)
The most immediate concern is straightforward: students can use AI to complete work that is supposed to reflect their own thinking. But the deeper issue isn’t just “cheating”, it’s misalignment. When tasks can be easily outsourced to a tool, it may signal that the task isn't measuring what we actually care about. Responses like moving back to paper-and-pencil or locking down the test might feel like a form of control, but they often push assessment back toward rigid, time-bound formats that don’t reflect authentic learning and have other implications for accessibility. - Validity and fairness (Evangelista, 2025; Gardner et al., 2021)
If AI is involved, either by students or embedded in scoring systems, we have to ask: What is actually being measured? An assessment loses validity when performance reflects tool use more than understanding. At the same time, AI-driven scoring systems can introduce inconsistencies or bias depending on how they are trained and applied. If we don’t examine these systems carefully, we risk measuring something—but not the right thing. - Transparency and trust (Evangelista, 2025; Topping et al., 2025).
AI systems are not always clear about how they make decisions. When students receive feedback or scores generated by AI, they (and educators) need to understand where those judgments are coming from. Without transparency, trust erodes quickly. And once trust is gone, assessment becomes performative rather than meaningful. - Access and consistency of use (Martin et al., 2025; Francis et al., 2025)
Not all students have the same familiarity with or access to AI tools, and not all are taught how to use them effectively. This creates uneven conditions, with some students benefiting from support while others do not. Ironically, attempts to ban AI outright can make this worse, pushing use underground or removing supports that some learners rely on to demonstrate what they know. - The unintended consequence of restrictions (Francis et al., 2025; Kooli & Yusuf, 2024).
Efforts to eliminate AI from assessment environments often lead to tightly controlled, one-size-fits-all formats (e.g., heavily proctored, timed, text-only tasks) or environments (e.g., lock-down browsers). These approaches may reduce AI use, but they can also limit how students show their understanding. In trying to protect assessment as a product, rather than supporting the students in demonstrating what they know. Additionally, these formats and environments may make the assessment process inaccessible to students who require disability accommodations.
The bottom line: the challenge isn’t simply whether AI should be used, it’s how to design assessments that remain meaningful, credible, and aligned with what we value, even when AI is part of the landscape.
What are the Basics of Effective AI Assessment Design?
Let’s get you started on designing better assessments with AI in mind. If AI has changed anything, it’s this: strong assessment is less about the final answer and more about how someone gets there. When designed well, AI-integrated assessments refocus on what is being measured.
- Integrate hybrid human–AI collaboration models (Kadel et al., 2024; Martin et al., 2025; Xia et al., 2024)
Assume AI will be part of the workflow. Sure, there are occasions when AI isn't part of the design, but for the most part, it will be part of human existence. So, thinking about how to measure human understanding and judgment while also building understanding of when, why, and how to use AI is a critical skill. The goal isn’t to avoid the tool but to evaluate how effectively it’s used alongside human thinking. - Focus on the process, not just the product (Kofinas et al., 2025; Martin et al., 2025)
A polished final submission tells you less than you think. Build in checkpoints: drafts, revisions, notes, and decision points. When you can see how thinking evolves, you get a far more accurate picture of learning. - Require artifacts (Martin et al., 2025)
Have students submit evidence of their work along the way, outlines, prompt iterations, feedback logs, reflections, revisions, or design notes. These artifacts make thinking visible and harder to outsource. These can be submitted in multiple formats such as documents, video demonstrations, images, and even live demonstrations. - Emphasize justification and evidence. (Martin et al., 2025; Salinas-Navarro et al., 2024)
Don’t just ask for answers; ask why and on what basis. Require students to explain their reasoning, cite sources, defend choices, and connect decisions to evidence. This shifts the task from production to argument. Again, justification and evidence can take many forms. - Include a reflection on AI use (Martin et al., 2025; Khlaif et al., 2025)
When using AI, make it part of the assessment. Ask students to document how they used it, what it helped with, where it fell short, and what they chose to accept or reject. This builds awareness and accountability. Have students submit their chat logs or demonstrate their interaction through a presentation or video. - Target higher-order thinking (Thanh et al., 2023)
AI is good at generating surface-level responses. So move up the ladder. Design tasks that require analysis, evaluation, synthesis, and critique, areas where judgment and context matter more than generation. - Frame tasks around complex, open-ended problems (Francis et al., 2025)
Lean into problems that don’t have a single correct answer. Use scenarios, case-based work, projects, and experiential learning that mirror real-world conditions. The messier the problem (within reason), the more it reveals about thinking. On the educator side, AI is great at helping create case studies and authentic assessments, using synthetic data and backstories.
What Does An Example of an AI Assessment Look Like?
Below is an example of a process-oriented assessment for an undergraduate special education teacher education course. The assessment required the students to reference the course content, focus on human-first thinking, then human-AI collaboration, critique, explanation, and reflection. It’s important to note that the case studies and associated materials were developed through human-AI collaboration. Finally, the students submit various artifacts from throughout the process.
Here’s a look at the step-by-step directions given to the students:
- Choose one case study.
- Analyze a case study and define the problem
- Identify and explain two or more appropriate research-based or evidence-based practices from the course materials for supporting the students in the case study.
- Collaborate with an AI: Craft an AI prompt to develop solutions to support the students.
- Critique and refine the AI response to create your final step-by-step solutions.
- Explain how your solutions will be applied based on the case study.
- Reflect on what you learned from the process, this case study, collaborating with AI, how you refined the responses, and how it might be applied in your future education career.
- Record a video to present and reflect on the entire process, including your collaboration with AI (see final submission checklist).
- Submit your work, at a minimum, including the video and AI transcript. You are encouraged to use the rubric to self-reflect on your work before submitting.
Conclusion
An effective AI assessment design doesn’t try to outsmart the technology. Rather than designing around the technology, assume it’s being used or purposefully integrate it into the process. Build students' human skills in a content area in authentic ways with a focus on higher-order thinking, while also supporting effective human-AI collaboration. An assessment design should support students in knowing what to use, when to use it, and how to use it effectively. Encourage human reflection and an audit of what AI did, said, or supported. It’s critical for humans to grow their own knowledge while also learning to effectively use this new technology. The assessment process should encourage both of these elements.
Join the conversation in our community!
CIDDL is committed to providing high-quality resources to support the growing knowledge, adoption, and use of educational technologies by educators, related services, and leadership preparation programs. For more resources, including videos and blogs, subscribe to our newsletter and follow us on YouTube, Facebook, and LinkedIn. The most important part of our CIDDL community is YOU. Join our community and share the innovative ways you are using technology, ask a question about technology integration, or participate in our bi-weekly live AI Community Chats. We look forward to seeing you in our community!
References
Evangelista, E. (2025). Ensuring academic integrity in the age of ChatGPT: Rethinking exam design, assessment strategies, and ethical AI policies in higher education. Contemporary Educational Technology. https://doi.org/10.30935/cedtech/15775.
Francis, N., Jones, S., & Smith, D. (2025). Generative AI in Higher Education: Balancing Innovation and Integrity. British Journal of Biomedical Science, 81. https://doi.org/10.3389/bjbs.2024.14048.
Gardner, J., O’Leary, M., & Yuan, L. (2021). Artificial intelligence in educational assessment: 'Breakthrough? Or buncombe and ballyhoo?'. J. Comput. Assist. Learn., 37, 1207-1216. https://doi.org/10.1111/jcal.12577.
Kadel, R., Mishra, B., Shailendra, S., Abid, S., Rani, M., & Mahato, S. (2024). Crafting Tomorrow’s Evaluations: Assessment Design Strategies in the Era of Generative AI. 2024 International Symposium on Educational Technology (ISET), 13-17. https://doi.org/10.1109/iset61814.2024.00012.
Khlaif, Z., Alkouk, W., Salama, N., & Eideh, B. (2025). Redesigning Assessments for AI-Enhanced Learning: A Framework for Educators in the Generative AI Era. Education Sciences. https://doi.org/10.3390/educsci15020174.
Kofinas, A., Tsay, C., & Pike, D. (2025). The impact of generative AI on academic integrity of authentic assessments within a higher education context. British Journal of Educational Technology. https://doi.org/10.1111/bjet.13585.
Kooli, C., & Yusuf, N. (2024). Transforming Educational Assessment: Insights Into the Use of ChatGPT and Large Language Models in Grading. International Journal of Human–Computer Interaction, 41, 3388 - 3399. https://doi.org/10.1080/10447318.2024.2338330.
Martin, A., Tubaltseva, S., Harrison, A., & Rubin, G. (2025). Participatory Co-Design and Evaluation of a Novel Approach to Generative AI-Integrated Coursework Assessment in Higher Education. Behavioral Sciences, 15. https://doi.org/10.3390/bs15060808.
Salinas-Navarro, D., Vilalta-Perdomo, E., Michel-Villarreal, R., & Montesinos, L. (2024). Using Generative Artificial Intelligence Tools to Explain and Enhance Experiential Learning for Authentic Assessment. Education Sciences. https://doi.org/10.3390/educsci14010083.
Thanh, B., Vo, D., Nhat, M., Pham, T., Trung, H., & Xuan, S. (2023). Race with the machines: Assessing the capability of generative AI in solving authentic assessments. Australasian Journal of Educational Technology. https://doi.org/10.14742/ajet.8902.
Topping, K., Gehringer, E., Khosravi, H., Gudipati, S., Jadhav, K., & Susarla, S. (2025). Enhancing peer assessment with artificial intelligence. International Journal of Educational Technology in Higher Education, 22. https://doi.org/10.1186/s41239-024-00501-1.
Xia, Q., Weng, X., Fan, O., Lin, T., & Chiu, T. (2024). A scoping review on how generative artificial intelligence transforms assessment in higher education. International Journal of Educational Technology in Higher Education, 21, 1-22. https://doi.org/10.1186/s41239-024-00468-z.