Evaluation
The success of the diversity and inclusion education in middle school program will depend on the implementation a leaner evaluation, a formative evaluation and a summative evaluation. Aspects of the learner evaluation will be detailed in the syllabus. It will measure performance against in a single point rubric. The formative evaluation will further measure effectiveness by validating learners’ knowledge outlined in the Dick, Carey & Carey approach as well as the final stage of ongoing evaluation. The summative evaluation will result from statistics gathered over the length of time the program will be running. It will be compiled quarterly and yearly to track the long-term impact of the program.
Formative Evaluation
The “formative evaluation is built into an instructional design project from the beginning stages… to gather data” (Brown & Green, 2020) on improving course content, The Dick, Carey & Carey evaluation model will be used to gather learner validation one to one, in small group and in field trial. This model will be the perfect fit for the diversity and inclusion education in middle school program because of how it will be structured.
One on one learner validation: a pre-instruction and post instruction questionnaire
In order to gage the familiarity of the middle school teachers vis-à-vis topics surrounding diversity and inclusion, bullying and middle school psychology, a short five-minute pre-instruction survey will define the learning audience’s knowledge gaps. The same survey will be distributed at the end of the instructional event to measure progress and whether or not the knowledge gap has receded.
The question that will be asked is:
On a scale of 1 to 5, 1 being the least expert and 5 being the most expert, please rate your expertise level with the following topics:
The survey will be translated into a learner ability chart with results in the 1 and 2 categories to signify low or limited experience, 3 as average experience, 4 and 5 as experienced learners. Criteria with a large count of learners with “low” or “limited experience” (1 and 2 on the scale) will have highest priorities in the content creation. Criteria with large numbers of learners with “average experience” (3 on the scale) will have the second highest priorities in the content creation. Criteria with large numbers of “experienced learners” (4 and 5 on the scale) will become topics that can be touched on without spending a significant amount of time in developing understanding.
The action from the data analysis of these surveys will be to develop in depth understanding for topics that are weaker in the learner’s knowledge from the course conception to the course completion. At the course completion, the identified topics that will still need to be developed in more depth will become topics to cover during the follow up workshops post instructional event.
Small group validation
The small group validation will take place as learners learn in teams and participate in group discussions. Questions will be incorporated as part of the activities conducted during the instructional event. They will be:
-
What fundamental elements have you learned from the wheel of identities exercise?
-
What conclusion can you draw from Bennett’s Developmental Model of Intercultural Sensitivity (DMIS)?
-
What impact can you have in teaching middle school students about diversity and inclusion?
As the learner’s evaluation will be benchmarked against a single rubric, their performance will be verified according to the below criteria:
The data that will be collected in the “concerns” column will be used to improve the course content. Observations in areas of struggles will be examined. If they are common areas for many learners, the action will be to review the coursework so that clarifications can be made to address these challenging areas for the learners. On the other hand, if the area of concern will be specific to a learner, it will be addressed individually with the leaner on one on one conversations and practice.
In field trial:
The diversity and inclusion education in middle school program will include a monthly one-hour follow up workshop after the learners complete the initial instructional event. By the time they will participate in the workshop, they will already have taught at least one class on diversity and inclusion to their middle school students. This workshop format will be for teachers to share with the facilitator and their peers successful teaching methods as well as criteria for continuous curriculum improvements as observed in the field. The questions that will be asked are:
-
What is the reaction from middle school students on this enrichment class?
-
How do the students’ parents feel about this initiative?
-
What format works well for your students?
-
What do you need help with? What support do you need?
-
Are there any recommendations for improvements or best practices?
The qualitative data gathered from these questions will be presented in bar graphs to represent the general feelings of students, their parents and the teachers. Based on the feedback, the content will be adapted to overcome challenges, whether it'd be to create a communication plan with the parents or an in-depth adjustment of the content to make it more relevant to the students. For this in trial field, it will be crucial to collaborate with the middle school teachers to improve processes and instructional design.
​
Summative Evaluation
​
In revisiting my instructional need analysis, I had indicated that the key performance indicators (KPI) for the program will be measured against changes in the following areas:
-
A decrease in instances of “Safe to Tell” reports in middle school
-
A lower to non-existent youth suicide rate
-
A lower to non-existent instances of school violence
-
An increase in student motivation to learn and excel in school i-e better school ratings and grade averages
-
An increase in school attendance
-
A decrease in student turnover in schools (transfers)
As these statistics will provide answers to the questions: “did the instruction bring about the desired changes?” and “ were the client’s goals met?”, the KPI are going to be used to measure the summative assessment for the diversity and inclusion education in middle school program. On a quarterly basis, the facilitator will collaborate with the school district to obtain reporting on each of the above-mentioned areas. The tracking will be totaled to a yearly progression to evaluate how sustainable the program will be long-term. Statistics that meet the KPI will translate into the success of the program. If no changes were to take place or the opposite results than expected werer to occur (such as an increase instances of “Safe to Tell reports instead of the expected decrease), then the whole program will need to be revisited from the instructional need analysis.
Conclusion
In the context of diversity and inclusion education in middle school, the evaluation plan will be comprehensive. It will include the learner evaluation, some of which will also serve as useful data for the formative evaluation. To track the long-term success of the program, it will be important to track specific reports on student trends quarterly and yearly in order to have a holistic approach to the success of the program.