A while ago now (the 28th June, 2011, to be exact) I gave a talk at the “How’s My Feedback?” (HMF) conference at the Saïd Business School @ University of Oxford. It was a great conference and I thoroughly enjoyed the presentations and discussions. My talk was a bit peculiar in that I was being ‘wormed’ during it. Putting aside the obvious jokes, I shall say simply that this involved use of a bit of software developed by ‘SocialPoll.tv‘ to enable people to rate my talk live. A representation of those ratings was then displayed as a moving line on the screen behind me. You can see James Munro explaining this here, and Steve Woolgar leading a discussion on the technology here.
The talk itself was about a very small bit of research I did for the purposes of giving the talk, having been kindly invited by Malte Ziewitz and Steve to speak about rating systems and reflect on the prototype website they’d developed, http://www.howsmyfeedback.org/, a rating site for rating sites. Being a unfamiliar with many of the rating sites with which HMF was concerned, I decided to take on one that would be at least a little interesting and fun, adopting the principle that if you don’t have anything much to say, at least say it entertainingly. You can see the talk here, and the thoughtful reflections provided by Sally Wyatt, who acted as a discussant. Overall, I think the site piqued my interest, although- from the limited peaks I took at the worm poll – I can’t speak for the audience.
I think, and this is the point of this entry (aside from publicising the various excellent people involved), that a challenge with live feedback is that it breaks down the power relationship between the audience and the speaker. Ordinarily, I suppose, there is the opportunity for someone in the audience to shout out during your talk and disagree with you, however, collective measures of exasperation, or indeed enjoyment, have historically been far more regimented, confined as they are to the polite applause and question period. Our historical mode of measuring negative feedback during our talks has been to spot people sleeping, yawning, stretching, doodling, chatting, gazing listlessly out the window, or – occasionally – looking repulsed, offended, terrified or completely baffled. Positive feedback comes in the form of smiles, a straight back or engaged seating position, a thoughtful expression or a tentatively probing question mid-flow. Paul Rabinow has commented that such expressions of boredom and suchlike are characteristic of academic habitus, and so we might wonder if the adoption of novel forms of feedback might disrupt that habitus.
In part, this issue of feedback has been circling around my mind of late because the Unit Evaluation Questionnaire results recently dropped into my email box and I’ve spent a little while reading through the quantitative and qualitative scores. This is serendipity, of sorts, since I’ve also just been lecturing first year UG students on questionnaires and interviews. One of the things we tend to teach at this level is that there’s a difference between quantitative and qualitative research methods and that they’re perhaps contradictory, perhaps complementary. I pointed out, for instance, that one is tied to a particular history of positivism that imagines the survey to be something like a laboratory, where the social context, the outside world, can be slowly but surely operationalised out. The other, our qualitative methods, try and embrace context and – at their best – visibly inscribe the mode of production into the object of knowledge. But how, then, do we go about interpreting the vagaries of student attitudes using both measures? How do I interpret their feedback when it doesn’t seem to relate to their behaviour: one week they’re asleep, the next they’re alert. Feedback, in other words, comes in multiple forms that are not necessarily coherent and perhaps have more political value than they do pragmatic. My UEQ feedback isn’t terribly useful except in that it helps me argue a case for my continued employment.
In my hotornot talk I discussed the ways in which the power to contest your score were limited and I think this applies here. Though my UEQ scores are perfectly wonderful (thanks principally to bribery and an endearing inability to stop swearing) there’s very little I can do to understand them. This is similarly the case in the hotornot.com example. What does a 6.1 hotness rating mean if I can’t contextualise it and reconcile it with the more qualitative comments made on my profile (you’re fit! – you’re fugly!). How do I use the knowledge produced through the UEQ to engage in ethical action regarding my teaching and to take care of my self/identity/students/reputation? This difficulty of making use and making sense of the feedback within complex contexts is an important one that goes largely unexplored in bureaucratic systems of student organisation, not least in these and other satisfaction polls. One specific difficulty with these representations to which we might seek to relate is that we can’t get much access to means of producing them- who generated the feedback, using what strategy and with what purpose? Rating systems, then, impose an identity that one has to negotiate with, but afford little in the way of mechanisms for challenge or interpretation. In this respect, we must be more attentive to the ethics and care that are implicated in ratings and their social management.
In closing, I should point out what properly started this rather rambling detour: I received an email about a new smart mobile phone-based system being trialled at Manchester for live interactive voting for lectures (this is to replace an older, more inconvenient system), which I’m now signed-up to try. So perhaps everything is just getting more and more ‘live’. The temporality of feedback, how your ratings change over time, the phenomenal experience of disappointment or jubilation, is difficult to contextualise whether your ratings are live or post hoc, but the challenge of making them sensible during the activity being rated is surely more complex. Whatever the outcome of this addition to my lectures, I don’t doubt it will suffice only to further complicate the experience, to add one more rating to a plethora or (in)formal measures. No doubt also, in due course, you’ll find some ‘live rating’ statistics in my curriculum vitae. Look how much the worm moved this year!
@AndyBalmer 