Beyond exam basics

It’s that time of year when most academics on the semester system have just finished giving final exams. What I didn’t realize until I became a faculty member was how challenging writing (good) exams could be, as Doug Natelson noted on his blog a couple of weeks ago. When I first started at Carleton, I think I was much more nervous than my students on exam day… was the exam too hard, too easy, too long, too trivial? Did my exam effectively assess student learning? With experience, I’ve gotten better at gauging the appropriateness of an exam, although I still occasionally miss the mark.

Now that I’ve gotten past the worst of the exam-writing butterflies, I’ve been considering aspects of exams that go beyond the nuts and bolts. In particular, a PHYSLRNR list-serve discussion in October in response to this Boston Globe article coincided with the arrival of a Tomorrow’s Professor post on creating exams that aim to improve metacognitive skills.  (Stephanie Chasteen had a timely blog post in response to the PHYSLRNR discussion.) Together the list-serve discussion and the TP’s post provided some interesting food for thought on the topic of exams, serving as a reminder that exams aren’t just summative assessment tools but an integral part of the learning process.

All this brought to mind research reported in the July 2010 issue of the American Journal of Physics, “Do advanced physics students learn from their mistakes without explicit intervention?” by Andrew Mason and Chandralekha Singh, showing that students failed to use exams as learning experiences. Mason and Singh studied an upper-level honors quantum mechanics class where identical exam questions appeared on the midterm and final exams, and students were given the midterm exam solutions. While some students’ performances on the exam questions the second time around were as good or better than they had been the first time, indicating the concepts had been learned and retained, nearly as many students performed worse on questions that were repeats from the midterm. In introductory classes, I occasionally ask students to review their exam solutions, compare them to my exam solutions, and turn in a written evaluation of where they went wrong in their approach, but I generally assume that such explicit interventions are not needed for advanced students. However, Mason and Singh’s work made me question that assumption.

Thinking about all this — the testing effect and retention, testing approaches that enhance metacognition, summative versus formative assessment — is enough to cause me to redevelop exam-writing anxiety.

Outreach that works

Several months ago, the band director at my old high school sent me a message asking if the next time I returned to my hometown I would chat with a junior who was considering a physics career. Always excited to talk with a potential future physicist, I happily agreed, and I thoroughly enjoyed the hour-plus conversation I had yesterday with this enthusiastic young woman. I asked what had promoted her interest in physics. Among other things, she mentioned Saturday Morning Physics (SMP) at Fermilab. It struck a chord because SMP was one outreach program that played a key role in my decision to pursue physics, and I’ve met at least one or two other physicists originally from the Chicago area who were influenced by that program.

Although a variety of outreach programs impacted my choice of career, the SMP program at Fermilab was one of the most influential.* I’ve often thought about why it had such an influence on me. One obvious difference between SMP and other programs is that it was more than just an afternoon or a weekend of science. The program was 8 or 9 weeks long, and every week included both a lecture and a lab tour.  I remember that I would think about the topic presented on Saturday for the entire week following the lecture, and I would look forward with anticipation to the next Saturday when I would be introduced to another mind-boggling topic. That brings me to the second aspect of SMP that made it different — the choice and presentation of topics. SMP didn’t tell me about physics. Rather, it introduced me to the questions that kept physicists up at night, the strange paradoxes and unsolved problems that would engage physicists for decades to come. Physics was portrayed as a field that would welcome curious minds to tackle the big questions; what we knew appeared to be utterly fantastic and there were many unanswered questions waiting for future generations of physicists.

It seems as if instilling this sense of mystery and awe is one of the goals of the program. In his account of the history of SMP on the SMP website, Roger Dixon writes:

We preferred that [students] be given enough facts to mystify them and to make them want to learn more about the subject.   These are students who have seen the wonders of Star Trek on television, so they are not easy to impress.   It always gave me a good deal of satisfaction to find that I could amaze them with an account of the Twin Paradox and convince them that real science can be more fantastic than science fiction.

I’d be interested to hear if others had outreach experiences that influenced their choice to pursue a science career. What was the program, and what made it attractive?

 

 

* The ironic thing about the SMP program is that, although it did a fantastic job of getting me into physics, it made me aware that I absolutely did not want to be a cosmologist or particle physicist. Ah well!