Describe the course, class, or other teaching context in which you conducted your project.
Both students and faculty from the Vanderbilt Center for Human Genetics
Research participated in my project. They completed an online survey on
their own time during the space of two weeks in March 2008.
What questions about student learning and/or your teaching practice did you investigate in your project?Why are these questions of interest to you? What led you to focus on these questions?Why might these questions be of interest to others in your field or elsewhere?What does the literature on teaching and learning say about these questions?What conjectures did you have about the answers to these questions?
Big Picture Questions
--Do students and faculty differ in how they categorize questions according to Bloom's taxonomy?
they don't, then does that imply that regardless of status on the
novice-expert continuum, educated individuals agree on the type of
knowledge required to answer questions?
they do, then does that imply that inexperience in the field may affect
how an individual views the type of knowledge required to answer
questions? Does this decreased cognition affect how students respond to
became interested in these questions after noticing a disconnect
between what questions faculty were asking and what answers the
students were giving for some exam questions. I wanted to know if this
was due to poor wording of the questions, or whether there are
fundamental differences in how experts and novices think about
answering questions. This issue is probably universal across
disciplines, and therefore may be of interest to those teaching in
other fields. I did not find any literature specifically addressing
this question, but as someone new to SoTL, it's possible that I am not
doing a good search. However, I did find that Bloom's taxonomy does
have good inter-rater reliability when applied to AP Biology, MCAT, and
GRE questions (Zheng et al. 2008), which gave me confidence in using
this instrument. My guess as to the outcome of this experiment is that
the faculty will think they are asking questions at the higher levels
of the taxonomy, but the students will see them as remember or
understand questions at the lower levels.
What sources of evidence of student learning did you collect in order to help answer your questions?How did you collect these sources of evidence?What did you hope to learn from these sources of evidence?
collected data through an online survey which asked both faculty and
students: 1) to classify a question from the midterm or final exam of
HGEN I or Fundamentals of Genetics according to Bloom's taxonomy, 2) to
explain why they classified the question as they did, and 3) to rate
their confidence in ability to answer the question. My main goal was to
learn whether students and faculty classify questions in the same way
and whether they give the same reasons for classifying questions.
Bloom's Taxonomy Handout for Study Participants
Screenshot of survey questions
How did you analyze the evidence of student learning you collected?What answers to your questions emerged from your analysis? Support your answers with samples of student work, if possible.What
findings not directly related to your questions of inquiry emerged from
your analysis? Support your findings with samples of student work, if
your findings, what ideas do you have for future investigations along
these lines? If you were to continue this project, what would your next
analyzed this data in a simple way, using pie and bar charts to compare
the number of responses at each level of the taxonomy within the
student group, within the faculty group, and between the students and
faculty. I also considered the names of each level of the taxonomy as
"keywords" and searched for them as I was reading through the text
descriptions of why each person classified a question as they did.
Lastly, I looked to see if confidence level, junior/senior student
status, or statistical/molecular track status affected the
--There was poor agreement for most questions, regardless of student/faculty status.
the most part, when there was excellent or good agreement for a
question, students and faculty classified questions the same way.
a looser definition of agreement, there was more agreement for
questions rated at the lower levels of the hierarchy (remember,
understand, apply) than at the higher levels (analyze, evaluate,
What accounts for differences?
was not a strong relationship between confidence in ability to answer
the question and agreement with peers or type of question.
the overwhelming majority of participants referred to the handout often
or very often, differences regarding handout referral could not be
differences in how students classify a small subset of questions may be
explained by junior/senior status or statistical/molecular track status.
I were to continue this work, I would like to try the study again with
a larger sample size, and perhaps in groups of students and faculty
from other fields. I would also like to know if there is any
relationship between high scores on exams and the student picking the
same category of the taxonomy as the faculty member who wrote the
PowerPoint Presentation with Detailed Description of Results
The attached slide show provides the figures and graphs that led to the conclusions in this box.
Resources and Obstacles
Who (faculty, graduate students, others) were resources and allies for your project? How were they helpful?In what ways, if any, did modes of thinking or methodologies in your discipline assist you in this project?What
were some obstacles, challenges, or difficulties in conducting your
project? How did you mitigate or overcome these obstacles?
would like to thank Derek Bruff, Laura Taylor, and my fellow cycle 3
participants for their insights and feedback as this project developed.
Thanks to all the CHGR faculty and students who completed the survey. I
appreciate your time! Lastly, thanks to my mentor Jonathan Haines, for
encouraging me to pursue the Teaching Certificate.
background in statistics and data analysis proved useful during this
project, as it has in nearly every research project on which I've
worked. However, I am still adjusting to the way people doing SoTL talk
about their projects. I am so used to Introduction, Methods, Results,
Discussion that putting together this poster was disconcerting at
first. After much revising, and input from the CFT and CHGR
participants, I think the poster has improved.
what ways do you expect engaging in this project to affect how you
approach your teaching, research, and other scholarly activity?In what ways do you expect engaging in this project to affect your career choices and success?In what ways do you expect engaging in this project to affect your department, school, or discipline?
doing this project, I often thought about what did and didn't work in
my teaching, but I had never systematically tackled a research
question. Formalizing the process of inquiry, experimentation, and
reflection has helped me take those ideas that I used to have and
organize them so that next time I am teaching in a particular context
it will be easier to go back to those ideas and implement changes.
Based on these results, I plan to think more carefully about any exam
questions I write. When I'm grading, I will check to see if the
students responded at the level of the taxonomy I was asking for, or if
there was confusion.
plan to do a mix of teaching and research in the future, and having
completed the Teaching Certificate Program should help make me a more
attractive candidate to less-research-intensive schools. If I had not
sought out the CFT, I would never had taught while I was in graduate
school or doing a postdoc. I think that having that experience will be
invaluable when I am thrown into teaching my first year as faculty.
will be presenting the results of this project to faculty and students
in the CHGR. If nothing else, I hope to raise their awareness that
similar research methodologies can be applied just as easily to
teaching and learning as genetics. I also view any opportunity to
formally talk about teaching and learning as a positive experience for
the CHGR, since opportunities for students and faculty to talk in the
past have been rare.
This page was designed as part of the Vanderbilt Center for Teaching's Teaching Certificate program.
Author: Kylee Spencer
Last Updated: 4/8/2008