Beyond AI Adoption: Why Asking Better Questions About Privacy and Security Matters
Authors: Yerin Seung; info@ciddl.org
Artificial intelligence (AI) has quickly become part of conversations about teaching and learning. Much of the discussion has focused on what AI can do for education—personalizing instruction, increasing accessibility, reducing teachers' workload, or supporting student learning. As these opportunities continue to expand, schools increasingly face another challenge that receives far less attention: How do we know whether an AI tool is trustworthy enough to use with students?
This question was the focus of a recent CIDDL Office Hour featuring Dr. Jennifer Lohoefener, Associate Director of the Institute for Information Sciences (I2S) at the University of Kansas, and doctoral researcher Daniel Neugent. Drawing from their work in cybersecurity, trustworthy AI, and formal verification, they discussed why privacy and security have become increasingly important considerations as AI enters educational settings, and why educators do not need to become cybersecurity experts to make informed decisions.
AI changes how we think about privacy and security
Concerns about data privacy did not begin with AI. Schools have long considered how educational technologies collect, store, and protect student information. However, AI introduces new challenges because many of today's systems operate as "black boxes." Educators can see the prompts they enter and the responses they receive, but they often have little visibility into how the model reaches its conclusions or how information is processed behind the scenes.
This lack of transparency becomes particularly important in education, where AI systems increasingly support instructional decisions. Whether recommending the next practice problem, identifying students who may need additional support, or providing automated feedback, AI systems influence real educational experiences. As Dr. Lohoefener noted during the discussion, these decisions can have meaningful consequences for students, making transparency and trustworthiness especially important.
The challenge is not simply that AI collects data. Rather, educators often have limited information about what data are collected, who has access to them, how long they are retained, or how they may be used in the future. Because many AI tools are developed outside education, teachers and school leaders must often make adoption decisions without fully understanding these tools.
Educators do not need all the answers, but they should ask better questions
One of the strongest messages from the Office Hour was that educators are not expected to become experts in cybersecurity or machine learning. Instead, they can play an important role by asking thoughtful questions before adopting AI tools.
Dr. Lohoefener suggested beginning with four practical questions:
- What data does the AI system collect? Beyond obvious information such as student work, AI tools may also collect interaction data, including typing patterns, pauses, or other behavioral information.
- Who has access to the data, and how long is it retained? Understanding data retention policies is important because information may remain stored long after it has been collected.
- Is student data used to train future AI models? Using data to improve AI systems is not inherently problematic, but educators should understand whether data are anonymized, how they are used, and whether these practices align with district policies.
- What happens if there is a data breach? Rather than accepting general statements about security, schools should understand how vendors respond to breaches and what responsibilities educators and districts have if one occurs.
These questions do not require technical expertise. Instead, they encourage educators to become informed consumers of AI technologies by expecting greater transparency from vendors and developers.
Trustworthy AI requires collaboration
Another important idea that emerged from the discussion is that responsibility for trustworthy AI should not rest solely with educators. The researchers emphasized that many of the technical solutions for improving AI security and privacy already exist. However, these practices are not always consistently implemented across AI products. As a result, meaningful progress depends not only on technological innovation but also on schools, districts, and policymakers consistently asking for greater transparency and accountability from AI developers.
Importantly, the discussion also acknowledged that data collection is not inherently malicious. In many cases, developers use data to improve AI systems and create better educational experiences. The key issue is whether these practices are clearly communicated, responsibly implemented, and aligned with the expectations of educators, students, and families.
Looking ahead
AI is evolving at a remarkable pace, making it unrealistic for educators to keep up with every new tool that enters the market. Rather than feeling pressure to adopt every innovation, schools may benefit more from developing a thoughtful process for evaluating AI before implementation. Ultimately, privacy and security are not barriers to using AI in education. Instead, they are essential considerations that help educators make informed instructional decisions. By asking better questions about how AI systems collect, manage, and protect student information, educators can help shape a future in which AI supports learning while maintaining the trust of the students and families they serve.
Join the conversation in our community!
CIDDL is committed to providing high-quality resources to support the growing knowledge, adoption, and use of a range of educational technologies for educators, related services, or leadership preparation programs. For more resources, including videos and blogs, subscribe to our newsletter and follow us on YouTube, Facebook, and LinkedIn. The most important part of our CIDDL community is YOU. Join our community and share the innovative ways you are using technology, ask a question about technology integration, or participate in our bi-weekly live AI Community Chats. We look forward to seeing you in our community!