Academic Roadkill

October 15, 2018October 15, 2018 Lily Cho

Grind Image via

We are so lucky to have a guest post from the inimitable Linda Morra. Here she is, thinking hard about how the “grievance studies” hoax hurts:

I’ve thought quite a lot about roadkill recently, probably because of the long stretch of drive, from Montreal (where I live) to Sherbrooke (where I teach). Today, it was a deer; a couple of days ago, it was a raccoon and a cat. I consistently think about how these animals were simply foraging for food, innocent of their imminent violent end—one that was just veering around the corner and bearing down on them, as the result of a vehicle on an entirely different trajectory. I feel for these animals – I sometimes even foolishly weep for them.

And today, this week in particular, I identified with them.

I am not actual roadkill, very obviously. But, when I read about the latest version of the Sokal hoax, which produced and found publication for sham essays that spouted left-wing ideology, I felt like I was thrown to a curb. They were out to prove that the social sciences and humanities are not undergirded by proper research, but rather by left-wing ideology. Apparently, seven of twenty of the latest Sokal hoaxes were accepted. Seven. That’s one third. It saddens me to think that either the editors or peer reviewers weren’t doing their job, and that it has, as a result, cast suspicion on the entire field.

On some levels, I think the hoax serves as an important reminder of how some of us failed in our responsibility by allowing such articles to find publication. How different are we from, say, some very right-wing American news station, if we simply allow left-wing ideology to stand, rather than allowing real and meaningful scholarship to undergird the ideology? Than well-articulated, defensible points of view? Aren’t we just producing fake news too?

Perhaps some of us are. But here is why I felt like roadkill: many, if not most of us, aren’t.

As Lily Cho pointed out, we invest substantial voluntary time—and often unrecognized, at that—in peer-reviewing. We are the ones who invest countless hours in reading and vetting papers for scholarly journals and manuscripts; we also train students, both graduate and undergraduate about what real scholarship means and what it looks like; we try to build meaningful connections and address injustices, locally and globally. I and others often do this work with very little recognition or reward—neither public, nor institutional. We do not take up time in a spotlight as a means of advancing ourselves or our careers, and, for many of us, there is no other reason to make such an investment. (For a moment, at least, I’d like us to reflect upon how, conversely, these three academics used their time, as a point of contrast. Seriously—didn’t they have better things to do?)

There is no reward, particularly, for vetting a manuscript or an essay for a journal—not at my institution, anyway. But I still do it, at regular intervals, and try to provide feedback with care—looking up sources to make sure what has been presented is accurate and to check claims that are being made. Some of the claims are not properly historically grounded—the writer might try to apply recent theoretical or ideological trends anachronistically. I check that tendency. Others simply pretend that no other academic has done the research before—and “disappear” other critics in the process in order to elevate their own scholarly ego (more of that in another blog post). Usually, I and other peer reviewers do our best to catch these kinds of errors, because we know we have a standard to maintain. We may not always succeed—but we try.

Why do we do this? Because peer-reviewers are, in fact, gate-keepers. There’s no point in claiming we are not, because we ultimately determine what passes muster and what does not.

And this is not an issue of control, as some of my own colleagues have suggested—because that would render us no different than any major news outlet that lays claim to ideology as news rather than factually-based research. We do it out of a sense of personal and communal responsibility. And responsibility means accountability. And accountability works on multiple levels: from the individuals who write the articles, to those who serve as peer reviewers, to the editors themselves.

That’s why I feel like roadkill: I and others have been inadvertently injured too. The recent Sokal hoax may have shown us where the weakest links in our chain are. The perpetrators themselves engaged in an unethical intellectual exercise to prove that ideological politics have supplanted scholarship, or at least that very little scholarship undergirds the ideological politics being championed—in the very realm where it shouldn’t do that.

But, in the process, they harmed the credibility of many good academics, who are committed to their work and to maintaining scholarly standards—which may now seem like no standard at all.

Linda Morra is a Professor of English at Bishop’s University. She has tried to research and publish meticulously about archives, especially those related to women writers in Canada, including Sheila Watson, Jane Rule, and M. NourbeSe Philip (in Unarrested Archives, UTP 2014). She is extremely grateful to her peer reviewers and editors, who have invested time in providing critical feedback that has helped to shape and improve her scholarly work.

academic publishing · emotional labour · feminist communities · peer review

Sokal Spare Me: Hoaxes and Anti-feminism

October 6, 2018October 6, 2018 Lily Cho

So here we are again. Somebody (three of them this time) thinks that they are so smart and clever. They are going to show the academy, and especially the feminist, queer, and racialized academy, that it doesn’t know anything.

All it shows me is that these three people — and I am purposely working on forgetting their names because I am a professional and I hope to give their real work a fair shake if I ever came across it, which is more than they deserve, but I am nothing if not an actual professional even though I feel a lot of rage at this particular moment — have shown a profound disdain for the gendered labour of academic journal editing and peer reviewing.

When I say that peer review and journal editing is gendered work, I mean that it is largely (if not wholly) invisible, underpaid or not paid at all, and almost entirely thankless.

As I wrote in my Love Letter to Peer Reviewers Everywhere, peer reviewers rock my world. I see their work everywhere: “in that book that changed the course of my dissertation, in that first article of mine that saw the light of printed day, in that other article that I taught in my grad seminar that re-oriented the entire discussion for the better, in all these journals that I read when I get a chance, marveling at all this marvelous work out there.”

They make all that happen. They make my world smarter, brighter, and just plain better.

I peer review anywhere between five and ten articles a year. It takes weeks of time that I never have. No one will know that I did it except the editor of the journal who can’t reveal my identity anyways. I do it because someone else did it for me and because I know that this completely invisible and thankless labour is a crucial part of sustaining our work as scholars.

Like most scholars at my age and stage, I serve on the editorial boards for three or four major journals both in Canada and internationally. This also takes a lot of time that I never have.

I was the co-editor of an academic journal for three years. It just about killed me. I just agreed to step back into that role because the current editor asked and I respect him so much and he is overwhelmed by the work and I know exactly how overwhelmig it is. It is the main cultural studies journal in Canada. If we don’t do this, there wouldn’t be a publication venue for a lot of amazing cultural studies work. As editors, we do not get paid for this work. I don’t get course release or a stipend. There may be the tiniest crumb of prestige but it is frankly outweighed by the reality of the work — hundreds of hours fielding angry or nudging emails while shepherding manuscripts through the peer review process while the authors are anxious and mad at me for not getting their work turned around more quickly.

Others have pointed out that academic journal publishing is a good faith system. This hoax takes advantage of the deep generosity of a community of scholars in order to score a stupidly cheap point. It does not show that the journals, fields, and disciplines that it targets are fraudulent. It shows that these three people had so much time on their hands, and so little regard for their peers, that they are happy to waste the time of people who are trying to make space and give a platform for new scholarship.

This hoax comes across as anti-feminist not because of the content of the articles but because its very form is premised on scorn and derision for deeply gendered labour. It misunderstands power. It mistakes peer reviewing for gate keeping. It mistakes the journal editor as a disciplinary figurehead. I don’t know of anyone who agrees to peer review something because they want to keep someone or some thinking out of the field. And journal editing is, honestly, an extravagant convergence of caretaking and traffic control.

So, spare me. Spare me this thinly disguised contempt for gendered labour. Spare me this willingness to waste time we never had.

peer review · writing

Learning to Love Being Edited

April 6, 2017April 18, 2017 melissadalgleish

I had this professor in graduate school who was notorious for being a brutal grader. You would submit a paper for her class, and know that you would get it back with nearly every word marked up, plus a page of razor-sharp, painful comments at the end. We all dreaded the day we knew she was returning our work. We all learned to get our papers back, quickly check our final grade (they were often quite good, despite the reams of criticism), and then tuck the paper away until we could come back to the edits with the ability to actually process them, not just with tears and adrenaline.

Later in graduate school, I had a supervisor who didn’t want to see work until it was as close to polish perfect as I could get it. I was stuck on the first big section of my research project, and would have loved to share the draft work I had in hand so that we would work together to figure out why I was stuck and where to go next. But that wasn’t an option, because I was expected to figure it out on my own, and so I stayed stuck for a long time. It was a moment when I really could have used a generous editor to move things along.

My PhD supervisor gives her graduate students an article she’s submitted for publication, along with the peer review reports, and has them read and summarize the most useful feedback for her. It means that she doesn’t have to read the more scathing (and infuriating–we all hate the reviewer whose point is just “you didn’t do it how I would”) reviews, and her students learn how peer review works and how to respond to requests for revision. It also means that she never has to cry over reviewer reports so critical and uncollegial that their writers would never say those things in person.

Given my experience, it’s no wonder that I hated getting feedback and being edited when I was still in academia. As teaching assistants and professors, so many of us aren’t trained in anything like the way that substantive and copyeditors are–to give useful, kind feedback that the writers we teach will actually respond to and act on, not just shut down about. Some of our ranks hide behind the anonymity of peer review to provide feedback that is unnecessarily harsh and potentially damaging. We’re also (for those of us in traditional humanities and arts programs) trained in a system that privileges originality and sole authorship over so much else. We don’t learn how to see writing and publication as a collaborative act.

So it’s been a surprise to me how much I love being edited these days. Many of my current writing projects–selected poetry editions, articles for major online publications, ghostwriting–require me to work with professional editors, and to have my work go through at least one, sometimes many, rounds of substantive, line, and copy editing. Sometimes my editors are really happy with what I send in, and it gets a minor tweaking. Sometimes I’m told to go back to the drawing board and try again. In either case, I genuinely appreciate it, and that’s a huge change from my experience in academia.

Why is that? Here’s what I identify as the key differences from my past experience that make being edited such a generative process now:

By virtue of our working relationship, my editors and I are in this together. We’re both working toward the common goal producing high quality content for the publication or press that they ultimately for, so they’re invested in (and work to promote) my success.

Working with an editor takes the pressure of sole-authorship off my shoulders. While my name is (most of the time) still the one that appears on the cover or byline, I’m no longer solely responsible for how my writing turns out. It becomes a collaborative endeavour, and one that benefits from multiple perspectives and sets of eyeballs.

I’m working with consummate professionals who know and are good at their jobs. Academia doesn’t always do a great job of training us for key parts of faculty jobs (most parts of faculty jobs, some would argue), and editing (which is a part of grading, peer review, and providing feedback on thesis and dissertation writing) is certainly one of those things. I trust my editors quite a lot, and I know that they’ve been trained–both academically and on the job–to elicit from me the best writing I’m capable of giving them, in the most productive (which often means kind and generous) way possible.

My editors make my work so much better. It is such a surprising pleasure to write something that I’m already happy with, then have it come back to me tighter, more elegant, more on the nose, better structured. Often my editors are only suggesting minor tweaks, but they’re changes that I couldn’t see my way to on my own and that make a world of difference to the effectiveness and style my work.

What about you, dear readers? How are you feeling about the editors in your lives, good and bad? How can we do better at teaching people to give and receive useful criticism in academia?

free time · peer review · perpetual crush · you're awesome

A Love Letter to Peer Reviewers Everywhere

February 2, 2015October 6, 2018 Lily Cho

Dearest, loveliest, most gorgeous,

Are you surprised that I am beginning with terms of endearment even though we hardly know each other? But you and I know that you really are gorgeous. Oh yes, you. Don’t blush. This is not the time for bashfulness. It is true that I hardly know you. Indeed, chances are, if I walked past you in the street, I would not even know to say hi. But you should know that I am tipping my hat to you, even if I don’t really know you. Indeed, the crazy thing about our relationship is that it is almost completely dependent upon not really knowing you. Our relationship is at its best when I admire you from something of a distance – or at least from arm’s length. Anyways, enough was enough and it is high time I tell you how I feel.

I need you. Sorry to sound a bit clingy. But I really need you. I’m not even sure that ours is one of those healthy co-dependent relationships. Where would I be without you? Where would any of us (the tenured prof whose book is out with a university press, the precariat worker whose article is now happily out with that sweet little journal you never say on to, the graduate student with that first pub under her belt, the mid-career academic whose grant just bought her a little space and time to get that project together, the mildly totally desperate academic journal editor who is trying to usher through that one last piece so that the next issue can come out) be without you?

Did you come back from the winter break to an inbox full of “gentle reminders” for things that you had promised in a haze of exhaustion and a rush of nobility? Did you scramble to get all those grant apps assessed, those articles reviewed, that book manuscript evaluated? All while teaching your courses, writing reference letters, maybe pulling together a job app, or (in a slightly different version of peer review) reading lots of job apps files and so on, and so on, and so on. You did it, didn’t you? I know, you were pretty late with some of those. I know you felt bad. But the point is, you got that report in. You totally came through.

I know, sometimes it hurts.

Like when your words are referred to as fecal matter and perhaps taken out of context.

Or when you are depicted as a vicious sharp-toothed sea monster.

I get it. Our relationship would be nowhere without your brutal and unflinching honesty. Indeed, along with that arm’s length business, this brutal honesty is foundational to our relationship.

And I know, sometimes you just don’t feel seen, as though you are totally being taken for granted.

Like, when no one, not even the editor (you’ve given up expecting anything from the actual author because you realize, having been there, that it does feel a bit weird to acknowledge “the anonymous reviewers whose comments were so helpful yaddy, yaddy, ya”) who asked you for this thing in the first place, remembers to thank you.

Or when you write a ten-page, single-spaced review of a manuscript with detailed notes for revision, and then you see the thing come out in print and the author seems to have ignored everything you said.

Or when you told the editors that the ms was truly awful and should not be published only to see it out, with nary a comma moved, months later.

At times like these, you wonder why you bother with this relationship. It’s not for the money (hullo, unpaid and invisible labour? Sign you up!). You wonder if it makes you happy. You wonder why you can’t say no more. You wonder why the relationship feels so one-sided. You put out all this brilliance, and, at best, you get a pre-scripted thank you spit out from some OJS robot. You wonder how it is possible to feel so under-appreciated and so unloved. It’s not like anyone asks you how many pages of peer review reports you wrote this year as part of how they assess your “performance.” Certainly, your dean doesn’t pat you on the back and say, Those peer reviews you did were really great! Good job!

So maybe you wonder if you should say yes the next time. You wonder if they could at least treat you to a milk shake for once.

And then you remember that arm’s length thing. I couldn’t hug you even if I wanted to.

And okay, I know that we are not “exclusive.” Yes, I admit, there are a lot of you in my life. And, even weirder, sometimes I am you. (Woah, mind twister!) You know that this relationship wouldn’t work at all if you were the only one. Think of the pressure!

And you remember that other people have sometimes done this for you. Or they might. You never know. And mostly, you remember that the profession would be nowhere, really and truly nowhere, without you.

If you walked out on me tomorrow, the world as I know it would pretty much collapse. I’m not exaggerating. There would be no publications, no grants, no academic books.

But there’s more. I want you to know that I think about you all the time. Sorry if that sounds a bit stalkerish. Don’t worry. Remember that part about not knowing you if I passed you on the street? But I do think about you. I think about how great you are. I think about how you always come through even though you must brace yourself for potential ingratitude and disregard every time. Not to mention the nagging (hello, “gentle reminders”). I think about how you are the one who says yes after that poor, desperate journal editor (sometimes me) has been fully rejected by a string of others. You can’t imagine how grateful I am when you say yes. When you say yes, I do a little happy dance. I know you can’t see it. But it’s pretty cute. Trust me.

So, dear anonymous peer reviewer, I just wanted you to know that I would bring you flowers and buy you chocolates if I could.

You are marvelous. You step up. You come through. You shine quietly, brilliantly, in that space of anonymity that is the condition of our relationship. Some people might think that I should simply be writing a letter of gratitude. A love letter can be a bit weird given the nature of our relationship (yes, yes, arm’s length!). But I’m not just thankful and grateful. I am, but there’s more.

I sit at my desk and look around and I see you everywhere – in that book that changed the course of my dissertation, in that first article of mine that saw the light of printed day, in that other article that I taught in my grad seminar that re-oriented the entire discussion for the better, in all these journals that I read when I get a chance, marveling at all this marvelous work out there. You make all that happen.

You make my world smarter, brighter, and just plain better.

You rock.

Love,

Lily

emotional labour · ideas for change · peer review · you're awesome

Doing Peer Review Better

November 21, 2012April 19, 2017 aimeemorrison2 Comments

Last week I wrote about how to improve a conference paper proposal, to make it more likely to impress the peer reviewer assigned to assess it. This week, I’m thinking about how those of us who do peer reviews might do our part of the job better.

I’ve drawn all my inspiration for this post from the program committee of DH2013, the umbrella conference of the Allied Digital Humanities Organizations, which comprises the Canadian Digital Humanities Association, the Association for Computers in the Humanities, the Association for Literary and Linguistic Computing, and others. The conference is global and it is interdisciplinary. It is also highly selective, sometimes prone to controversy about who is in and who is out (and what is DH and what is not). For example, and in retrospect hilariously, even though I regularly review proposals for the conference, all three submissions I have myself made have been rejected. The first time, one peer review gave a one sentence assessment of my work (I paraphrase): “This work is not even interesting and I don’t know why this author would propose to consider it for this group.” That was in 2001 or 2002, and I still remember it as the most dismissive, disrespectful review I have ever received for a conference paper.

So imagine my pleasure this year when I visited the conference website to review the CFP as I prepared to assess my five assigned proposals. This year, the organization has put together not only a guide to writing proposals for its authors, but also, magnificently, a guide to peer reviewing this proposals for its assessors.

Go see it. Then come back.

Aren’t these just the very model of transparency? All the implicit rules by which proposals will be assessed are explicitly outlined. Even better, peer reviewers are reminded that their work is not simply to assess in summative fashion (accept / do not accept) but to mentor in formative fashion (How might this be improved? What are the strengths and weaknesses of this proposal?). Even better, peer reviewers are reminded of … well … the affective dimension of this part of academic work. Nasty peer reviews work to exclude people from the field. Harsh rejections are bad for morale generally. Community is built upon mutual kindness. The “big tent” model is not supported by vicious kicks to the support poles at the edges of the structure. This bit, to me, is the most incredible, and the most awesome: instead of training would-be participants to develop a thick skin and “try harder” when faced with what looks sometimes like gleeful rejection, peer reviewers instead are being asked to consider what might help the not-accepted scholarship fit within the fold next time. It manifests a kind of humility about the field and our expertise as gatekeepers within it, even as we are still very much called upon to review the intellectual merits of each proposal.

I imagine the acceptance rate will still be low. But maybe we all won’t feel so rejected by that.

I loved this so much I sent a mash note to Bethany Nowviskie about it.

The lessons of DH2013 and the wisdom of its program committee extend to all our peer review work. I know it’s given me pause. I get asked to do peer review all the time. And some of the papers I read are, not to mince words, terrible. And sometimes it feels like I’m wasting my time to read all 30 pages. I’m mad at the editors for even forwarding this to two hapless reviewers. I write incredibly sarcastic notes in my printouts, to blow off steam.

But I try, and will try harder, to make my reviews, even the ones about papers that purport to be about user-generated review sites but are actually thinly-veiled screeds about how evolution is just a theory and Richard Dawkins is going to hell (really), constructive. To keep them focused on the intellectual. To not descend into recriminations against the author’s not knowing what the journal is for, or not caring to take the time to copy-edit let alone proofread, or for stuffing an unreconstructed coursework paper into a digital envelope in total cluelessness about how that is not what an article is. That’s all judgy and personal. Try to be constructive.

I’m using the classic “shit sandwich” approach now. You are probably familiar with it.

Here’s how it works. Start by saying something nice (and true): maybe the topic is important. Maybe the approach is worthy. Maybe the primary texts have never really been considered before. Maybe the author has a great knack for fluid and engaging prose. Maybe the reference list is superb. Acknowledge the merits that you find. Next is the “however”: this varies from paper to paper, but might be courseworkitis (all lit review, no argument), or it might be something more complex and unique about the approach to the topic or the methodology or the sample set or whatever. Be specific but try to confine the criticism to the words on the page, not the character of the author. This can be sometimes very difficult to do. Do say: “The paper takes what it describes as “genre analysis” as its theme, but does not reference the major works in that field.” Do not say: “The author has no right to perform as so-called ‘genre-analysis’ when it is obvious that s/he hasn’t read anything at all in this well-developed field.” Finally, end with some suggestions for improvement. This is hopeful. Having perhaps said that it’s unworkable to try to prove point A with reference to text F and L, maybe suggest a set of texts that might be more useful. Or if the author seems unfamiliar with an important subfield he or she generalizes about, suggest one or two texts and one or two major ideas from that subfield the author might consult to better support his or her contentions.

If you feel this all doesn’t convey enough the depth of your rejection, then by all means make use of the field that is labelled “for the editors only”–you can let rip in that section, if you really need to.

Putting stuff out to review is very hard: this is part of the reason we all hold onto our drafts for too long. We are afraid of what the reviewers will say. And yet, as reviewers, we very often lash out at the poor shmucks who’ve let their precious drafts out into the world. Yeah, maybe they’re totally not ready for the big time, but we don’t have to be mean about it.

Do you have any tips and tricks for doing or receiving peer review? Funny stories? Terrible tales?