How Can Technology Change Assessment?

This United Nations Educational, Scientific and Cultural Organization (UNESCO) Institute for Information Technologies in Education (IITE) document examines information and communication technology (ICT) use in educational assessment. It asks:

• "First, how can technology more effectively enable assessments to be used as a tool to improve student learning in ICTs?
• Second, how can technology increase the range of learning outcomes we can assess and what will these assessments look like?"

The brief highlights the use of assessment for iterative improvement (progressively identified and improved) as a way to transform the design, evaluation, and delivery of ICTs. It describes assessing learning experiences versus assessing individuals. This kind of assessment can be built into adaptive learning systems to measure what the learner knows and, on the basis of results, direct the next assignment accordingly. These systems with built-in assessment can help identify problems and tailor instruction appropriately.

A second goal of learning assessment is to evaluate the learning experience - in short, to assess how well the instruction is doing rather than how well the student is doing. "Questions about the reliability and validity of assessment measures remain important, but equally important is the ability to make inferences about the quality of different specific learning experiences. If the goal of assessment is to improve instruction, then the measures need to yield actionable information at the level of the learning environment."

As stated here, summative assessments can be made on a large scale to test how groups of students are doing. However, they are not attuned to formative changes in instruction. However, technology does make possible large scale assessments that can provide formative information. For example, student learning can be measured on a large-scale for several variations of instruction. "Questions as broad as the best trajectory through the learning content or as narrow as the most effective way of visualizing information can be tested empirically, and the environment can be shaped to reflect the results. "

The document discusses A/B testing (two versions - A and B - are compared) and iterative design and testing. It elaborates possibilities of embedded assessments in which, during a course of study, related resources are provided for which students are then tested to see if they are able to learn from being provided with the additional resources or what types of resources provide them with added skills or knowledge. Iterative design of ICTs can provide measures of whether one experience (i.e., one version of the system) provided more learning than another.

This kind of information can be made available through data mining, an approach that leverages computers to discover patterns in large sets of data. Examples of ICT applications to reduce the complexity of data and increase the possibility of data mining results for iterative applications to learning include the crowd-sourcing of exam answers and the autoscoring of writing samples. Another ICT application is the use of natural language processing - computer algorithms to make sense of and interact with human language - to analyse writing samples in relation to a grading rubric for the content.

The document concludes that technology-aided assessments can help in the design, delivery, and improvement of learning environments. ICTs can be designed to collect data and selectively present different information to different students. "This can allow for the empirical testing of questions about the best design choices for learning, and the results can be incorporated into the learning environment for continual improvement. In terms of reach, technology allows for new kinds of assessments, including assessments of student’s learning and problem solving processes that are embedded in the learning context, and assessments of how well prepared students are for future learning."