An assessment to identify cognitive processes to improve

Do you know your students' cognitive strengths and weaknesses? A diagnostic evaluation could help you do this.

The last congress of theAssociation for the Development of Evaluation Methodologies in Education (ADMEE) having for theme the " Issues and challenges of assessment in the digital age in the classroom and at a distance »Took place in November 2016 in Sherbrooke. During this event, Nathalie Loye, associate professor at the Faculty of Education of the University of Montreal, and Dan Thanh Duong Thi, doctoral student, presented a paper on the diagnostic assessment of cognitive processes. Shortly thereafter, I spoke to Madame Loye about these reports.

École branchée: You recently carried out work on the cognitive diagnostic approach. What does this approach consist of in the school context?

Nathalie Loye: The cognitive diagnostic approach consists of highlighting cognitive processes that pose a problem for the student. Generally, these processes are linked to subject-specific content, strategies used by students or the characteristics of the questions asked of them. When a student scores low on an assessment, it may not always be possible to determine which cognitive process (s) caused the problem.

The particularity of the cognitive diagnostic approach lies in the fact that it is an evaluative test in which questions make it possible to measure these different processes. This approach requires identifying a priori the processes on which to base the diagnosis, then producing a test made up of questions aimed at measuring these processes. The test is then administered to the students.

Once the data has been collected from the students, psychometric models make it possible to estimate the probabilities of mastery of the attributes of each process by each student, which provides the diagnosis.

EB: What is the difference between a formative assessment and a diagnostic assessment? Is one the tool for the student to help him learn, while the other is the tool for the teacher to help him have a portrait of his class?

NL: Formative evaluation belongs to the teacher in his class. It is an integral part of its pedagogical approach and relates to the learning targeted by the teacher. The diagnostic evaluation in question here corresponds to an approach external to the class and often relates to prerequisites for learning. It makes it possible to have an individual portrait of each student and, by groupings, a portrait of the class, which can also be done by formative evaluation. However, these diagnostic portraits are based on the probabilities of mastering the targeted cognitive processes and not on the teacher's judgment.

EB: Can a teacher develop a diagnostic assessment on their own to measure cognitive processes and interpret relationships?

NL: The diagnostic evaluation of cognitive processes is a complex measurement and evaluation instrument to construct and the results of which are also complex to analyze. From this perspective, it would not be possible for a teacher to use this tool himself.

EB: So what can a teacher or a school manager interested in the diagnostic evaluation of cognitive processes do?

NL: In Europe, diagnostic assessments can be carried out by external bodies, such as ministries responsible for education. At the beginning of the year, for example, it is possible that a ministry administers these assessments to provide teachers with information to better understand the strengths and weaknesses of the students.

In Quebec, firms specializing in psychometrics and in measurement and evaluation could act as consultants for interested establishments. As such, the Quebec firm Brisson-Legris was one of my research partners as part of a grant from the Human Sciences Research Council.

EB: Exactly, what would these diagnostic reports contain?

NL: In order for the results of the assessment to be useful, it is important to communicate them to teachers, who are in the best position to put in place effective educational interventions to help students. Reports need to be understandable and useful so that recommendations are meaningful to teachers and school staff. These reports must describe and make accessible the intentions of the diagnostic evaluation as well as the results for each pupil and for the group according to the targeted cognitive processes. Ideally, these reports should also offer educational support avenues based on the difficulties identified for each student, or for the group.

EB: Can the teacher or school staff then draw inspiration from the report to implement an action plan?

NL: Yes, indeed. A teacher can obtain an overall portrait of his class as well as a portrait of each of the students. It could thus decide to adapt its interventions, its approaches or its tools according to these results. A school principal could, for his part, have an overview of the students in his establishment, of the students of a particular year or of a specific class. This could help it to put in place measures to support teachers or professionals.

More information to come

The preparation of diagnostic reports is currently the subject of a doctoral thesis, the results of which will be published shortly. Those interested in learning more about diagnostic reports can communicate with Madame Loye or Madame Duong Thi.