Hello! My name is Elena Wirth and I am a Senior double majoring in Human Services & Social Justice and Women’s, Gender & Sexuality Studies. I am a member of the Team 4: Proposition and Evaluation Team, along with Adam and Chris. I worked with Adam and Chris in the past, and felt that our work ethics and personality traits would mesh well together. We were tasked to come up with an evaluation rubric. At first the task seemed daunting. I remember thinking to myself: “how are we going to come up with a scale that is not subjective?” However, having two other people to brainstorm ideas with really helped me approach the work.
Before diving in, we each conducted some preliminary research, and discovered it was important for us to leave our own comments in each section along with our scores. We created a template with a 32-point scale with five different categories for evaluation. The five categories were: Goals and Objectives of the Project, Plan for Implementation, Metrics and Plan for Evaluation, Budget, and Formatting. Applications could be awarded on a scale of 1 to 4 points. Originally, we weighted points for the Goals and Objectives of the Project, Plan for Implementation, and Metrics and Plan for Evaluation more heavily than the other categories. We felt that these were three areas that could be key indicators of the potential success of programs. Applications were ranked using qualifiers like compelling satisfactory, decent, or unsatisfactory. For example, if I saw the Goals and Objectives of a Project as compelling, I would award the application 4 points. However, if I saw the Goals and Objectives of a Project as unsatisfactory, I would award the application only 1 point.
During our in-class workshopping of our rubric, one important note we heard from our classmates was to make sure that the language was consistent across the categories. Our classmates were also worried about the subjectivity of the rubric and Team 4 as the readers. They felt that we might be biased by our experiences with certain organizations or by reading them as a team and hearing each other’s opinions. We decided that Team 4 will read through all applications, presenting the top three to the class. Team 4 will also write out our justifications for our scores in each category.
After we received feedback from our classmates, we first changed the qualifiers at the top of the rubric within the descriptions of each category. Thus, we changed the qualifiers to compelling/exceptional, satisfactory/adequate, decent/somewhat, and unsatisfactory/missing. We then decided that it was better to weight each of the categories equally, rather that arbitrarily deciding that some categories were worth more than others. Our final eight categories are: Goals and Objectives of the Project, Relevance of the Project, Indication of Need, Capacity of the Organization, Timeline for Project Completion, Metrics and Plan for Evaluation, Budget, and Formatting.
As we head into the decision-making process, I am excited to be a part of the team making the first round of selections, and look forward to reading about creative programs to address educational and professional opportunities for immigrants in the DMV area!
--Elena
Responses from our class
Thank you for sharing your insight as a member of Team 4. This was an interesting read in the process of creating the evaluation, as I do not know how to create a fair and accountable, scoring criteria sheet that have so many important elements! On a side note, this team was actually my third option when Dr. Kelso presented the team responsibilities because of my previous experience evaluating. However, given the fact that I am from the DMV area, I did not think it would be applicable, and I think you, Chris, and Adam, had a great start!
To follow up on the class discussion, I was glad that you all took into consideration and account of finding a way to mitigate any potential bias, by having the entire class assist with choosing the top three candidates. I believe this is a way to reinforce class collaboration and am forward to seeing the ending process from our recruitment side to y'alls Top 3 org's!
Wishing the best in Team 4's process
--Ashley