Anonymous Feedback
At our university we are required to give a literature presentation. The presentation is 1 hour including time for questions. The goal of this exercise is to help students develop various skills that are valuable in a scientific career such as presenting, evaluating literature and answering questions. Good stuff.
I did my presentation and it really well except to one professor. According to my evaluation, I had confidently fabricated an answer and misled an entire audience. Fair enough, I make mistakes, but in this case I was right and had the literature to back it up. My scores were fine despite this one professor, but I wanted to discuss the issue. I was curious if the book and papers that supported my view were wrong. I wanted to learn.
I went to the head of session and asked who gave this evaluation. I was then told that the feedback was anonymous and they were not allowed to tell me. I then asked if he could ask the professor to contact me so that I could visit their office and discuss the evaluation. The head of session said that would be fine and sent out an email, but the professor never got in touch.
From this experience, I have also decided to no longer give anonymous feedback. I don’t want to hide. If the person who is getting my feedback disagrees, wants clarification, suggestions then great, let’s talk.
Scientific dialogue is invaluable and is reflected in our numerous publications and conferences.
I asked a faculty member of nearly 30 years why the policy existed and was told ‘this is the way we have always done it.’
Does your department give anonymous feedback? Do you find it helpful? Do you know why they keep that policy?
Artem
January 30th, 2010 at 3:16 PM #
This is good practice for later in your career when you have to deal with anonymous peer review – articles, grant applications, etc. Remember, there is always ‘reviewer #3′ (if you don’t know what this means you should look it up: http://www.youtube.com/watch?v=-VRBWLpYCPY )
Whether we like it or not (and I personally do not like it at all) anonymous review is an entrenched phenomenon and isn’t going to go away any time soon. It certainly has its issues but it also has benefits: consider if you had to review a lousy article from someone who is in position to harm your career – without anonymity you’d be exposing yourself to retaliation.
Perhaps there was never an irate professor at all – this could be a standard exercise to give you a taste of undeserved anonymous criticism and help you deal with it?
Sean
January 30th, 2010 at 5:49 PM #
I usually have an issue with ‘whether we like or not’ philosophy so that is why I am now against giving anonymous feedback.
I don’t believe the entire department is getting together to pull one over on the graduate students, but it is always hard to disprove conspiracy theories.
olchemist
January 30th, 2010 at 7:04 PM #
“Scientific dialogue is invaluable and is reflected in our numerous publications and conferences.”
)
Have you read or published any article of late? It is all about trying not to upset THE REFEREE like it was some ubermonster. Sadly, impact factor and the medieval peer-review process got the better out of the “scientific dialogue”. Most career oriented scientists are chicken… but in the end they may get a permanent position because they play well with others.
That reminds me of a present that I got for my 18th birthday
Artem
January 30th, 2010 at 10:31 PM #
Sean:
0. you may be against giving anonymous feedback – but rest assured that you *will* receive it. And therefore my central point is still valid – you need to be able to discern what parts of belligerent feedback are raving nonsense or personal vendetta, and what parts are actually useful to you in some way. Doing this without emotions getting in the way is the tricky bit.
1. If this is a game played with the students then the ‘entire department’ does not have to be involved – whoever is compiling feedback on you can simply insert a standard ‘angry reviewer’ entry into the pile of genuine responses. This is trivial stuff and in fact it has been done before – I have heard of this kind of ‘trial by fire’ done in law school. Naturally it only works for as long as the subjects are unaware. Ask your fellow students if any (all?) of them received 1-2 belligerent reviews on a background of generally nice ones.
2. I am not suggesting that you should not try and change the system – however the place to do so is *not* during the most vulnerable part of one’s career. In part my concern with the way academia is set up (in particular with respect to grant money) is what made me choose the industrial path.
3. There are almost no ‘permanent positions’ left. There won’t be any in the near future if the current trend continues. Scientific career system follows the flow of money and as the latter changes so will the former. For example, a lot of $$ now flows into academia from the private sector – much more so than ever in the past. If you believe the Ghostbusters – ‘they expect results’ (i.e. companies want to see specific results come out of their investment). Do you think the existing career system has a chance in a situation where most of the research money comes from industry and not from the government or endowments?
Sean
January 30th, 2010 at 11:30 PM #
@olchemist: I am actually on a paper being reviewed at the moment, so it will be interesting to see the feedback.
@Artem:
0. True, I will have to be able to correctly evaluate anonymous feedback.
1. The other students also had a number of horrible reviews so you may be on to something.
2. The interesting part was that when I asked the tenured professor about the situation he didn’t seem to care. I realize that this a datum point, but once you are no longer vulnerable perhaps there is no reason to make things better. Why change the system if you are on top? If the system is wrong then it should be changed whether you are vulnerable or not. What would you consider the most vulnerable part of a career?
3. I bank on change. I think we will see less professors having a long career at a single institution.
Artem
January 31st, 2010 at 1:23 AM #
If you are a student or a postdoc you’re much more vulnerable to harm simply due to the absence of established reputation. Your advisor’s ‘aura’ extends to you – but only to an extent. Once you get a reputation and also once reputations of others get tied with yours (by virtue of collaboration, co-authoring of papers and/or grant applications, presenting at meetings etc.) – your position becomes much more defensible.
When one is a student one’s career is protected by obscurity – for the most time students are just not important enough to spare the time and squash. It’s during late graduate school and postdoctoral studies when one’s career is probably the most vulnerable because at this time you’re expected to stick out from the background of young researchers and start making a name for yourself – which also means increased chances of stepping on toes, rattling the trash cans, and generally upsetting the status quo.
This doesn’t in any way imply that science is full of blood-crazy megalomaniacs who are waiting to destroy anyone challenging their dominance. Nevertheless it helps to keep in mind that in science (like in any other endeavor) careers tend to be built around ‘claimed’ areas of influence – and these areas are often zealously defended against incursions.
Dima
February 2nd, 2010 at 8:11 PM #
Anonymous peer review is positively evil. A quote I like:
“Peer review is just another popularity contest, inducing familiar political games; savvy players criticize outsiders, praise insiders, follow the fashions insiders indicate, and avoid subjects between or outside the familiar subjects.” And anonymity only makes it all much worse.
It’s from a section “PROBLEMS WITH ACADEMIA” found here. IMHO, the most succinct and to the point criticism of academic system ever written. This was written 20 years ago and yet not a single thing has changed.
Dima
February 2nd, 2010 at 8:15 PM #
The linked URL in my previous post was eaten by the system… Here it is in plain form: http://hanson.gmu.edu/gamble.html
DrSNO
February 2nd, 2010 at 11:24 PM #
@Artem, There are almost no ‘permanent positions’ left.: There are unfortunately too may tenure-track positions available. I have no respect for PIs that have studied their favorite protein/system for the last 20-30 years. There shouldn’t be more point-mutations to be done. Sadly NIH supports that. Sure the 200+ crystal structures of lysozyme by Brian Matthew’s lab was also supported by Howard Hugh money. But I can with great confidence say that 90%+ of that money could have been put to better use. There are many reasons for this system, however, my tax money isn’t used well by the current system. NIH’s budged should be dramatically cut and there would finally be a greater focus on research that will benefit human health. Sure I can and have provided evident for this and that, and publish articles. The money could have been better used for other things, however, I am limited by the techniques I know.
Artem
February 3rd, 2010 at 3:29 PM #
I am glad that I am not in academia