THE PIPELINE SLIP AND GOOGLE FORMS
WHAT’S CHANGED IN 2019
Note: This post is meant as an online addendum to my book Write More, Grade Less: The Pipeline System of Essay Scoring (c) 2008, 2018. Some of what I say in this post may not make much sense but if you have a copy of the book, and it still doesn’t make sense, ask me questions here.
In the book I am referring to page 3.9.
Also, what I’m doing now with Google Forms replaces: Teach your Trusty T.A. How to Record Results, p. 4.18 and T.A. Records the Results on the Scoring Tracking Forms, p. 4.32).
THE JOURNEY
When I began this system of peer review early in my career, I designed a long pipeline slip that attaches to the front of a packet that includes the rubric, then the student’s handwritten draft, then the student’s plan—unless of course the teen apostate defies Commandments 1 and 2 of Mr. Stanford’s writing workshop: Thou Shalt Plan, and Thou Shalt Not Argue About Planning.
The reason for the long slip was that I wanted to provide the scorer with feedback. Nothing needed to be detached or un-stapled—all I needed was a TA to cut the pipeline slip in half on the dotted line. The bottom half would go into the scorer’s mailbox, while the packet, with the top half of the pipeline slip still attached, ended up in the writer’s mailbox.
It was a good idea but there were problems.
The scorer could never remember anything about the draft they’d scored, particularly when they received six or ten score slips in the mailbox; as a result, the feedback was minimal, because the scorer would only see the score they’d issued, plus whether the writer marked “agree” or “challenge,” along with the result issued by the teacher. So, the only valuable feedback the scorer could see over time were the trends: Did writers tend to agree or challenge? Did the teacher tend to agree with the scorer? Sure, there’s value in that, but I always wondered how I could make it more valuable.
I never liked that I was dependent on the kids to keep track of all their score slips, because inevitably, even on Reward Day, when I issued grade points or raffle tickets in exchange for saved score slips, somewhere between a fifth and a quarter of the class reported misplacing multiple score slips, if not all of them, and I was left with no good record of the results.
So, I tried training a student T.A. to record all the results. Even with one talented T.A. working almost daily, the task proved too large, too tedious. Mistakes were made, especially recording the results on paper.
Next, I tried ditching the slips. Writers marked “Agree” or “Challenge” directly on the rubrics. Again I tried having the T.A. record all the results, for all the writers and all the scorers. I tried asking the T.A. to use specialized forms in a big binder. My poor T.A. —Katherine was her name—one semester she did this, day after bloody day: she would pick up a draft, and turn through the binder to the writer’s page, and record the result, and then turn through the binder to the scorer’s page, and record the result. Day after day. She should have been awarded a purple heart for her time in my prison.
So, second semester I created a spreadsheet for her. That was better; all she had to do was press CTRL+F and search for the writer’s code name, record the result, search for the scorer’s code name, then record that result. (Am I driving you crazy yet? I’m sorry!)
She’s human and she made mistakes, and besides, I didn’t have time to proof her work. And what was this spreadsheet for, anyway? The scorers certainly weren’t getting better feedback this way.
TECHNOLOGY TO THE RESCUE
Meanwhile, technology continued to improve. When I started this system in the early 2000s, kids owned phones, but not smartphones. It was paper, or nothing. By 2015 I no longer needed an overhead projector, and no longer walked through life with black and blue and red hands from messing around with transparencies and dry-erase pens every day. By 2017, for the first time, our school site moved to 24-7 one-to-one Chromebook rollouts. Each kid would have a Chromebook. What a change—before, most kids had a smart phone, and about half the kids had Internet access and a computer at home. And most kids would understand how things worked in the Church of the Holy Google.
By the start of second semester in January 2018, epiphany: I created two Google Forms. I called them Pipeline Surveys, one for scorers and one for writers. I created spreadsheets into which these forms would deposit data. The first one scorers are now required to fill out after scoring each draft.
THE DRAFT CODE
Into the form they must enter a draft code: six digits comprised as follows: draft number (first digit zero if the draft number is less than 10), writer’s mailbox number, then scorer’s mailbox number.
I thought that in the spreadsheet, each draft written by a student and scored by another student would have a unique identifier, but I forgot that students share mailboxes, and so there are some rare occasions in which multiple drafts share the same draft number. Example: student A scores student B’s draft #10. Student C scores student D’s draft #10. Students A and C share the same mailbox number; students C and D share the same mailbox number. But, this is infrequent and I can live with it.
I worried the students would not be able to figure this out. I worried needlessly. They’re fine—they almost always rise to expectations.
I LET THE STUDENTS SEE THE RECORDS
The forms do not ask for students’ real names—only code names and mailbox numbers—so I allowed students to view, but not edit, the results in the forms. The moment I allowed them to earn a little extra credit to tell me about mistakes they found, a handful in each class (you know, the ones who don’t actually need any extra credit) set about combing through the records and submitting edits. I would make the changes quickly and mark them in red so that everyone could see that something had changed. Now, the students entering the information into the forms tend to work more carefully, knowing that their work will be reviewed. As a result the records are highly accurate.
OTHER BENEFITS
Finally, writers and scorers are getting the kind of feedback I always envisioned, but could never actualize before the Church of the Holy Google.
Scorers can see the results of their efforts—easily—and so can I. After students enter data into the scorer’s pipeline form, I sort by the scorers’ code names so it’s easy for them to see it all in one place. On Reward Day this spring I will have them add up the results of their efforts.
Writers can also see results more easily. After students enter data into the writer’s pipeline form, I sort by the writers’ code names. Almost all of my writers are showing positive trendlines in terms of essay scores issued by me. And I can more quickly and easily see which writers are spinning their wheels in the mud, and use what limited conferencing time I have to focus on those writers.
In conclusion, I realize that this level of tech and accounting adds up to sheer misery for many of my colleagues. I would say this: first, learn how Google Forms work. Learn how to share the results in a spreadsheet in Google Drive, with your students, in a way that prevents direct editing. If you can do that, you can use the tool to accomplish all this.
I encourage you to ask me questions and I will do what I can to help you. I will also try to find ways to create some tutorial videos, because the system as it works now is benefiting my kids like never before.