Confused at a higher level

The view from Carleton College's physics department

  • Archives

  • Stats

    var sc_project=3293756; var sc_invisible=0; var sc_partition=21; var sc_security="d61881ba";
    free hit
counter
  • Subscribe

  • Recent Posts

  • Follow me on Twitter

Volunteer recognition, finally!

Posted by Arjendu on January 17, 2008

Gene Sprouse wrote me yesterday (as well as to many many many other physicists):

The Editors thank you for your help during the past year as a referee for the journals
of the American Physical Society. Your thoughtful and well-informed reviews are crucial
for our decisions, and your comments help authors improve their manuscripts. We sincerely
appreciate your assistance and look forward to your ongoing contribution to the physics
community.

To express our appreciation for the essential work that anonymous peer reviewers do for
our journals, we are starting an annual program to recognize approximately 130 of our
42,000 active referees each year as “Outstanding Referees”. Each awardee will receive a
certificate and pin, and we hope to thank each in person at one of our annual APS meetings.
The list of Outstanding Referees will be published online and in the journals although each
can choose not to be recognized publicly. We will select the referees based on many factors
that may include diligently returning reports in a timely manner, reviewing many papers
over many years, or providing especially insightful advice. The award will be one-time and
the referee must be living. Neither nationality, APS membership, nor field of scientific
expertise will be a factor. We hope that all referees will be pleased, whether or not they
are chosen, that we are recognizing some who have done an especially outstanding service to
the community.

Well, finally! Not because I’m expecting to be one of the Outstanding Referees, but some sort of recognition for this is a very good idea. It’s a little strange that the nominal ‘make or break’ activity for reward in academic physics — publications — is handled by unpaid anonymous refereeing. I don’t mean to say that the referee’s name should be published along with the authors (although in certain moments I have wondered if that might change things for the better considerably; transparency and acceptance of responsibility is almost always a good thing. But it would make it hard within the current power system for junior referees to do a good job of reviewing senior authors’ papers).

But some sort of lifting of the veil might help. Here’s why:

Physics has a relatively high publication rate (compared to Math, say), which I’ve heard attributed to a tendency to publish a lot more papers than are actually useful (or read, perhaps). There is a sense in which we believe, I think, that the good stuff out of all that will survive/rise to the fore, and so some sloppy papers getting through, or some inappropriate rejections don’t mean a lot. Great attitude in general, and probably responsible for the rise of arXiv (the free online unrefereed repository so many of us use).

I try my hardest to do a conscientious job when called to referee: Respond promptly, be honest about reading the paper carefully and understanding it, be clear and respectful about what I don’t understand, or about things that look wrong to me, and to be as constructive as possible in the comments/reports. In general, I try to act like a colleague down the hall from the authors — and I love it when I get referees who treat me the same way.

But the acceptance of slop in the system combined with the anonymity means that no one beats up on the referee if a particular paper that (s)he let through turns out to be flawed or incomplete and there is no payoff for being careful, compassionate, intelligent in your refereeing (except for serving the cause of physics itself, don’t get me wrong). What’s the external incentive then for doing a good job?

There was the usual reception hosted by the editors at Physical Review the last time I was at a DAMOP conference and I ducked into the tail end of it because I was feeling hungry and remembered that they always had good nibbles at the Phys Rev reception. I landed up chatting briefly with one of the Editors and in passing suggested to her that some sort of name recognition might be very valuable. It would give us an incentive — however minor it might seem, academics thrive on minor incentives 🙂 — to do a good job and it would give us some sort of responsibility for our product. All good things. The American Journal of Physics publishes the name of all their referees every year, and I think it’s worked really well for them.

I very seriously doubt that mine was the only such feedback they’ve gotten over the years, or that this was what was led to this new initiative, but about time!

5 Responses to “Volunteer recognition, finally!”

  1. Nick said

    I’m nowhere near being able to understand most of the papers for Physical Review but I’m perplexed by their volunteer referee system considering that they charge so much for APS membership and institutional subscriptions. I guess they’re trying to add incentive for their referees now but it seems… half-assed, maybe?

    I mean, the collective weighing-in of people like yourself either on your blog or someplace like arxiv.reddit.com would do the same trick but with more transparency and reach, don’t you think?

  2. arjendu said

    Nick, thanks for the comment.

    Well, let’s be honest — NO ONE understands more than a handful of papers in Phys. Rev. Which is why you need the Referee system to be trustworthy and efficient, since the someone needs to be the gatekeeper, given that consequences of publishing papers can be major professionally. Which is different from the consequences to science — that is about the result and not the publication, a subtle but important difference.

    As for the collective weighing-in … hmm, as I tell my students, science is not a democracy. It doesn’t matter what the majority of people think, the right answer is what it is. So we do value *expert* opinion somewhat more, particularly at the ‘cutting edge’ which is where we are trying to publish. Which means you’d have to appoint ‘weigher-ins’ and well, we are back to non-anonymous refereeing, which privileges those with the power over the lives of junior commentators.

    Hey, a thought, though: Perhaps making Referee reports available online (anonymously) would help, so people can see the kinds of junk Refereeing that we sometimes experience, examples of the good stuff, and also see some of the *process* of scientific debate. Linked to the journal subscription, of course.

  3. kavik said

    The system of anonymous referees is (in the big picture) is effective. However I think the authors and their institutions should be anonymous as well. This would eliminate (or at least reduce) the “star” factor and allow the referees to evaluate the work solely on its merits.

  4. Initial Thoughts on Refereeing

    I picked up Arjendu’s Confused on a higher level blog the other day from Chad and made a little comment on a post about refereeing for expensive journals. I’m still a little conflicted about the fact that high-profile journals seem…

  5. […] I picked up Arjendu's Confused on a higher level blog the other day from Chad and made a little comment on a post about refereeing for expensive journals. I'm still a little conflicted about the fact that high-profile journals seem to effectively do the same job as record companies; controlling their clients' work because it's in their own "best interest" whereas they'd get a lot more eyeballs on their work if it wasn't behind such an absurd paywall. That's not to say I don't recognize the positives behind having a random[1] selection of qualified people commenting on the merits of any particular paper – there's no way a small number of people could understand or even read all the contents of everything submitted to the most popular journals. Similarly, the referee and journal system should make it possible to have a completely unknown author break out with a really excellent paper since we're evaluating based on merit alone. Unfortunately I don't think this is the case for reasons I'm not completely clear about right now.I'm still trying to come up with specifications for a system that's effective where the current one fails. And this includes my first complaint[2] as well: referees are volunteering their time for free and journals still see fit to charge absurd amounts for subscriptions.[1] I'm actually unsure if the process is random, I'm just assuming that it would be to weed out any referee stacking in favor of a particular author with influence.[2] Posted as a comment on Arjendu's blog here […]

Leave a comment