Inquiry Phase

Context Describe the course, class, or other teaching context in which you conducted your project. Both students and faculty from the Vanderbilt Center for Human Genetics Research participated in my project. They completed an online survey on their own time during the space of two weeks in March 2008.

Questions What questions about student learning and/or your teaching practice did you investigate in your project? Why are these questions of interest to you? What led you to focus on these questions? Why might these questions be of interest to others in your field or elsewhere? What does the literature on teaching and learning say about these questions? What conjectures did you have about the answers to these questions? Big Picture Questions --Do students and faculty differ in how they categorize questions according to Bloom's taxonomy? --If they don't, then does that imply that regardless of status on the novice-expert continuum, educated individuals agree on the type of knowledge required to answer questions? --If they do, then does that imply that inexperience in the field may affect how an individual views the type of knowledge required to answer questions? Does this decreased cognition affect how students respond to questions? I became interested in these questions after noticing a disconnect between what questions faculty were asking and what answers the students were giving for some exam questions. I wanted to know if this was due to poor wording of the questions, or whether there are fundamental differences in how experts and novices think about answering questions. This issue is probably universal across disciplines, and therefore may be of interest to those teaching in other fields. I did not find any literature specifically addressing this question, but as someone new to SoTL, it's possible that I am not doing a good search. However, I did find that Bloom's taxonomy does have good inter-rater reliability when applied to AP Biology, MCAT, and GRE questions (Zheng et al. 2008), which gave me confidence in using this instrument. My guess as to the outcome of this experiment is that the faculty will think they are asking questions at the higher levels of the taxonomy, but the students will see them as remember or understand questions at the lower levels.

Experimentation Phase

Gathering Evidence What sources of evidence of student learning did you collect in order to help answer your questions? How did you collect these sources of evidence? What did you hope to learn from these sources of evidence? I collected data through an online survey which asked both faculty and students: 1) to classify a question from the midterm or final exam of HGEN I or Fundamentals of Genetics according to Bloom's taxonomy, 2) to explain why they classified the question as they did, and 3) to rate their confidence in ability to answer the question. My main goal was to learn whether students and faculty classify questions in the same way and whether they give the same reasons for classifying questions. Bloom's Taxonomy Handout for Study Participants Screenshot of survey questions

Reflection Phase

Findings How did you analyze the evidence of student learning you collected? What answers to your questions emerged from your analysis? Support your answers with samples of student work, if possible. What findings not directly related to your questions of inquiry emerged from your analysis? Support your findings with samples of student work, if possible. Given your findings, what ideas do you have for future investigations along these lines? If you were to continue this project, what would your next steps be? I analyzed this data in a simple way, using pie and bar charts to compare the number of responses at each level of the taxonomy within the student group, within the faculty group, and between the students and faculty. I also considered the names of each level of the taxonomy as "keywords" and searched for them as I was reading through the text descriptions of why each person classified a question as they did. Lastly, I looked to see if confidence level, junior/senior student status, or statistical/molecular track status affected the classifications. Major Findings --There was poor agreement for most questions, regardless of student/faculty status. --For the most part, when there was excellent or good agreement for a question, students and faculty classified questions the same way. --Using a looser definition of agreement, there was more agreement for questions rated at the lower levels of the hierarchy (remember, understand, apply) than at the higher levels (analyze, evaluate, create). What accounts for differences? --There was not a strong relationship between confidence in ability to answer the question and agreement with peers or type of question. --As the overwhelming majority of participants referred to the handout often or very often, differences regarding handout referral could not be assessed. --Some differences in how students classify a small subset of questions may be explained by junior/senior status or statistical/molecular track status. If I were to continue this work, I would like to try the study again with a larger sample size, and perhaps in groups of students and faculty from other fields. I would also like to know if there is any relationship between high scores on exams and the student picking the same category of the taxonomy as the faculty member who wrote the question. PowerPoint Presentation with Detailed Description of Results The attached slide show provides the figures and graphs that led to the conclusions in this box.

Resources and Obstacles Who (faculty, graduate students, others) were resources and allies for your project? How were they helpful? In what ways, if any, did modes of thinking or methodologies in your discipline assist you in this project? What were some obstacles, challenges, or difficulties in conducting your project? How did you mitigate or overcome these obstacles? I would like to thank Derek Bruff, Laura Taylor, and my fellow cycle 3 participants for their insights and feedback as this project developed. Thanks to all the CHGR faculty and students who completed the survey. I appreciate your time! Lastly, thanks to my mentor Jonathan Haines, for encouraging me to pursue the Teaching Certificate. My background in statistics and data analysis proved useful during this project, as it has in nearly every research project on which I've worked. However, I am still adjusting to the way people doing SoTL talk about their projects. I am so used to Introduction, Methods, Results, Discussion that putting together this poster was disconcerting at first. After much revising, and input from the CFT and CHGR participants, I think the poster has improved.

Looking Ahead In what ways do you expect engaging in this project to affect how you approach your teaching, research, and other scholarly activity? In what ways do you expect engaging in this project to affect your career choices and success? In what ways do you expect engaging in this project to affect your department, school, or discipline? Before doing this project, I often thought about what did and didn't work in my teaching, but I had never systematically tackled a research question. Formalizing the process of inquiry, experimentation, and reflection has helped me take those ideas that I used to have and organize them so that next time I am teaching in a particular context it will be easier to go back to those ideas and implement changes. Based on these results, I plan to think more carefully about any exam questions I write. When I'm grading, I will check to see if the students responded at the level of the taxonomy I was asking for, or if there was confusion. I plan to do a mix of teaching and research in the future, and having completed the Teaching Certificate Program should help make me a more attractive candidate to less-research-intensive schools. If I had not sought out the CFT, I would never had taught while I was in graduate school or doing a postdoc. I think that having that experience will be invaluable when I am thrown into teaching my first year as faculty. I will be presenting the results of this project to faculty and students in the CHGR. If nothing else, I hope to raise their awareness that similar research methodologies can be applied just as easily to teaching and learning as genetics. I also view any opportunity to formally talk about teaching and learning as a positive experience for the CHGR, since opportunities for students and faculty to talk in the past have been rare.

This page was designed as part of the Vanderbilt Center for Teaching's Teaching Certificate program. Author: Kylee Spencer Last Updated: 4/8/2008