Improving Writing Outcomes for Students with Disabilities with Automated Scoring
Dr. Joshua Wilson
About Dr. Wilson
Dr. Joshua Wilson is an Associate Professor of Education at University of Delaware’s School of Education (SOE) and a faculty affiliate of University of Delaware’s AI Center of Excellence and Center for Research in Education and Social Policy.
As a foremost expert in the field of automated writing evaluation, Dr. Wilson’s research positions him at the forefront of integrating AI technologies in assessment and instruction to enhance educational outcomes. His research focuses specifically on the ways that automation and AI can be used to transform the teaching and learning of writing. A former special education teacher, Dr. Wilson remains focused on improving outcomes for students struggling to learn and on preparing teachers ready to make a difference in the lives of all of their students.
The problem highlighted in this brief
Students with disabilities (SWDs) struggle with writing, which is how teachers assess content-level understanding. To improve their writing outcomes, SWDs need timely feedback that is specific and individualized (Troia, 2014). However, many students are not receiving quality feedback because it takes considerable time to provide (Wilson et al., 2021), and teacher preparation programs often lack training on how to give effective writing feedback (Hodges et al., 2019). Thus, the purpose of this brief is to highlight how automated scoring can support teachers in providing feedback both in the classroom and as part of broader universal screening initiatives to identify struggling writers.
Why does this topic matter to teacher preparation?
As AI and automation become more prevalent in education, teachers must be trained to leverage automated scoring technologies to support writing outcomes for SWDs. Additionally, with the rise of generative AI tools that can assist or even replace parts of the writing process, teachers will need guidance on adapting their writing instruction.
About This Brief
This brief explores how automated writing evaluation and feedback tools can support SWDs, particularly by enabling universal screening and targeted interventions. Dr. Wilson shares his research on how automated feedback can improve the writing products of SWDs through productive revision.
Research and Practice Context
The Importance of Feedback
Effective feedback is essential in writing instruction, yet many educators struggle to provide targeted, actionable feedback, especially for students with learning difficulties (Troia, 2019). Research indicates that manual scoring is time-consuming, limiting teachers' ability to support struggling writers (Truckenmiller, 2020). AI-powered automated scoring and feedback systems offer promising solutions by enabling efficient assessments and personalized revision suggestions, which are important for improving student writing outcomes (Wilson et al., 2021). Understanding these tools' capabilities and limitations, including teacher perceptions, is vital for future writing pedagogy (Wilson, Delgado, et al., 2024).
The following are key insights shared by Dr. Wilson on this research. The interview focused on six questions around automated scoring in teacher preparation as well as recommendations for teacher educators to incorporate these ideas.
Conversation with Dr. Wilson
Q1: What issues are you trying to address through your work with automatic scoring?
Research shows that teachers are not prepared to teach writing and provide feedback on student writing (Brindle et al., 2016). Beyond this, teachers struggle to align assessment data to targeted instructional strategies (Troia, 2019). Dr. Wilson's work on automated writing evaluation and feedback aims to address these gaps by providing teachers with more efficient and informative assessment tools. This allows teachers to dedicate more time to targeted writing interventions while automated systems handle the time-consuming task of scoring and providing feedback on student essays.
WILSON: “I remembered the students that I had as a teacher who were fourth graders and writing at a first grade level, and I didn't have the tools that I needed to help them or the training.”
Q2: Can you walk us through how your work with automatic scoring supports students with disabilities?
Dr. Wilson's research highlights the potential of automated scoring tools to improve writing instruction for SWDs. Traditional writing assessments are time-consuming and often provide delayed results, hindering timely interventions (Wilson, Cruz Cordero, et al., 2024). Automated scoring systems, on the other hand, offer fast and consistent feedback, enabling educators to identify and address writing difficulties early on (Wilson, Cruz Cordero, et al., 2024). This allows for quicker implementation of targeted support, preventing students from falling behind academically.
WILSON: “We can get those scores back to the district immediately, as soon as they're done, that opens up a whole new possibility for getting these multi tiered systems of support in place where we can take assessment data and then connect it with intervention data to support the students who are showing signs of struggling before it's too late.”
Q3: How can we better prepare educators for using these technologies in the field?
There is a gap in how special educators are prepared to effectively teach writing and automated writing technologies. Dr. Wilson shared that this is an even more critical topic as generative AI tools like ChatGPT have the potential to dramatically transform the writing process.
WILSON: “There's definitely a gap there, and I think that gap is all the more alarming given the the age of AI that we're in now with generative AI, because we're in this spot now where it's possible that the future of the writing process is going to be forever changed with these generative AI tools. So how do we teach teachers, to teach students to use tools that they have never used or don't know how to use themselves”
Q4: What implications do you see for future research, and what are some questions we might be asking?
Dr. Wilson shared his excitement about the potential for future research to leverage generative AI to provide students with more sophisticated, tailored feedback. He sees the technology as an opportunity to better align automated feedback with best practices while also using it to advance research on effective feedback strategies. He suggests that by focusing on developing age and skill-appropriate AI feedback systems grounded in research on what works, there is immense potential to transform writing instruction and assessment in powerful ways.
WILSON: “What do we know works with feedback? How much feedback? What are we going to focus on? What are we going to prioritize in this feedback?”
Q5: What else should teacher preparation programs consider moving forward?
Research shows many teachers are not comfortable or skilled at providing effective feedback on student writing. Dr. Wilson noted that teachers often struggle with knowing what to prioritize in their feedback, how to communicate it effectively, and feeling discouraged when students don't act on the feedback. Using ChatGPT to practice feedback allows teachers to experiment and reflect without any risk, since they are just interacting with an AI system, not exposing their actual feedback to students. By embedding these kinds of structured AI-based feedback opportunities into teacher preparation programs, alongside instruction on feedback frameworks, Dr. Wilson believes it could be a powerful way to build teachers' competencies in writing instruction and feedback.
WILSON: “It would be interesting for teachers to have some opportunities, maybe structured, as they learn about best practices and feedback, to actually use AI, a chat bot, to to gain some sort of insight and reflection on their own in a very safe way, like the there's no there's no risk in revealing anything to an AI, you're not, you know, it's very safe space to say, like, here's some feedback I gave to my student. They're not applying it… How can I improve this?”
Q6: Are there any specific technologies you would suggest?
Dr. Wilson encourages teachers to dive in and explore the capabilities of these systems in a low-stakes, exploratory manner. One suggestion he had was to provide a generative AI with writing samples and then analyze the feedback it AI generates. These free, widely available language models are an ideal entry point for teachers to start building their understanding and comfort with automated writing technologies - a critical skill as these tools become more prevalent in educational settings.
Dr. Joshua Wilson emphasized the importance of preparing teachers to effectively leverage automated writing evaluation and feedback technologies to support all students. Drawing from his own experiences as a special education teacher, Dr. Wilson's research has demonstrated how these AI-powered tools can enable more efficient, timely assessment and targeted interventions for struggling writers. He advocates for explicitly incorporating training on automated writing tools into coursework, while cautioning against making the training too narrowly focused on specific platforms.
Ultimately, Dr. Wilson expressed excitement about the potential for AI and automation to transform writing instruction, but stressed the critical need to keep the human teacher at the center of the process. By preparing educators with the right knowledge, skills, and mindset, he believes schools can leverage these technologies to enhance learning outcomes for all students.
Resources and References
In the interview, Dr. Wilson provided resources and technology tools for teacher preparation programs starting with AI in Education.
Learning more about AI
- https://www.deeplearning.ai/the-batch/ - Newsletter about AI from Andrew Ng's "Deep Learning" organization
- "The AI Advantage" YouTube channel - provides up-to-date information on newest developments in AI, AI uses cases, and prompting methods
Learning about AI in Education
- https://aixeducation.substack.com/ - AI x Education substack
- https://iste.org/ai - ISTE (International Society for Technology in Education)
AI and Higher Education
- https://library.educause.edu/topics/infrastructure-and-research-technologies/artificial-intelligence-ai - Educause topic list of different AI-related resources
Troia, G. (2014). Evidence-based practices for writing instruction IC-5. In Collaboration for Effective Educator & A. Development, and Reform Center (Eds.), Innovations Configurations: University of Florida.
Troia G. A. (2019). Writing standards: Overcoming the implementation challenges. The State Education Standard: The Journal of the National Association of State Boards of Education, 19(2), 24–27, 46.
Truckenmiller, A. J., & Petscher, Y. (2020). The role of academic language in written composition in elementary and middle school. Reading and Writing, 33(1), 45-66.
Wilson, J., Ahrendt, C., Fudge, E. A., Raiche, A., Beard, G., & MacArthur, C. (2021). Elementary teachers’ perceptions of automated feedback and automated scoring: Transforming the teaching and learning of writing using automated writing evaluation. Computers & Education, 168, 104208.
Wilson, J., Cordero, T. C., Potter, A., Myers, M., MacArthur, C. A., Beard, G., ... & Ahrendt, C. (2024). Recommendations for Integrating Automated Writing Evaluation with Evidence-Based Instructional Practice. International Journal of Changes in Education.
Wilson, J., Delgado, A., Palermo, C., Cordero, T. M. C., Myers, M. C., Eacker, H., ... & Zhang, S. (2024). Middle school teachers’ implementation and perceptions of automated writing evaluation. Computers and Education Open, 7, 100231.
Suggested Citation
Goldman, S.R., & the CIDDL Team. (2025). Improving Writing Outcomes for Students with Disabilities with Automated Scoring. The Center for Innovation, Design, and Digital Learning.
This work is licensed under a Creative Commons Attribution 4.0 International License.