Writing skills are critical across disciplines for academic and professional success. For students to develop effective writing skills, they need to write papers, get feedback on them, revise them, and get feedback on revised papers. Through these processes, students can improve writing skills and domain-specific knowledge. However, students are losing writing opportunities because of instructor’s workload in reading and giving feedback.

To remedy this adverse situation, SWoRD, a web-based system, has been developed to manage all of these related facets. SWoRD is a reciprocal peer review system that considers that all students do not generate accurate evaluation and feedback, especially because they are novices in their disciplines. SWoRD has developed its unique computational algorithm to estimate each individual reviewer's accuracy. Along with this measure, back-reviews by authors who get peer reviewers’ feedback are used.

What am I going to do with SWoRD?

In SWoRD, each student plays two roles anonymously, one as writer and one as reviewer. You do not know who wrote the papers that you review and who reviews your papers. Individual students will participate in the following SWoRD process for each assignment. You will start your activities by choosing topics for writing and reviewing.

For your writing assignment, you act as an author and turn in the first draft. After your reviewers evaluate the first draft, you receive their feedback. Then you turn in the final/revised draft. Then, you back-review or evaluate the reviewer’s feedback on how much it was helpful in revising your draft. The same reviewers review your final draft. After receiving the 2^nd round of peer reviews, you back-review the 2^nd feedback in terms of how it would be helpful if you revise the draft again.

For your reviewing assignment, you review a number of authors’ first and final drafts. Your instructor decides the number of peer drafts you review. As a rule of thumb, instructors set 5 drafts per version per reviewers. When reviewing, you provide written comments on how to improve drafts and then rate the quality of each draft. SWoRD examines the accuracy of your rating, and gives you reviewing grades. In addition, your authors evaluate your written comments, which also influence your final SWoRD grades.

How does SWoRD grade my assignments?

First, SWoRD examines and weighs each reviewer's evaluations through its own unique algorithm. This is used for grading each review activity. Also, this measure is used for computing each writer's writing quality. It helps writers from being penalized by unqualified reviewers.

Reviewing Grades. SWoRD evaluates the reliabilities of your ratings in terms of systemic differences, consistency, spread, and helpfulness. The reliability scores are summed, explaining 50% of review grades in each draft. The other 50% is helpfulness from the authors' evaluation.

Writing grades. SWoRD weighs each peer rating based on the reviewer's grading reliability. First and second drafts are weighed equally.

All SWoRD grades are curved, with a mean of 80 and a standard deviation of 10. Instructors are allowed to set their own curving criteria.

THIS SEEMS FISHY

Are my peers qualified to review and grade papers?

You may be concerned about how appropriate it is for your peers to evaluate your assignments. Our research has shown that undergraduate peer review and grading is highly reliable and as valid as instructor’s. Given that, SWoRD takes a few additional steps to ensure that your SWoRD evaluations are accurate, while simultaneously providing you with help in improving your writing and reviewing skills.

Why do I receive peer reviews from smaller number of peers than I reviewed?

Although each student was assigned to review the same number of papers, not all students complete the review assignments. However, it does not affect your writing scores.

Will I be penalized for having a bad reviewer?

Don't be panic. SWoRD diagnoses individual reviewer's review accuracy which is used to weigh your writing scores and also to grade reviewer grades. In addition, you can use the back-evaluation to give feedback to the reviewer.

Various studies show that peer reviews are more effective than instructor/expert reviews. For example, Cho and Schunn (in press) found that in SWoRD students receiving feedback from multiple peers improved the writing quality significantly better than those receiving feedback from a single expert. Also they found that six peer evaluators are most beneficial for their performance improvement (Cho, Schunn, & Wilson, in press).

HOW TO…

For Technical Support contact the SWoRD Support Team at sword@pitt.edu.

How do I create My SWoRD Account?

Go to the main SWoRD web page (http://ladybug.lrdc.pitt.edu/sword3).

At the main page, click on “Create New Account”.

Step 1. Policy Statement:

This will take you to the policy page that describes the agreement to participate and use SWoRD.

Read the policy carefully.

Check the box that says “Check only if you agree,” and click the Agree/Register button.

Step 2. SWoRD Technical Support Policy Statement:

This will take you to the tech support policy page that describes what help tech support can and cannot provide.

Read the policy carefully.

Check the box that says “Check only if you agree”.

Click the “Agree/Register” button.

Step 3. New Account:

The registration page will appear.

Fill in your information that will register you in the system. This page allows you to create a password that you will use each time you log onto sword.

Fill in your information that will register you in the system.

Click the arrow to finish with registration.

Step 4. Creating a pseudonym:

Choose a pseudonym to be identified with and type in the space provided.

Then click “Available?”.

The system will inform you if the name is already taken, so you may have to try another name.

If your pseudonym is available, click the arrow to proceed.

Step 5. Confirmation:

After you received confirmation that your account has been successfully completed, you will need to enroll in a course.

Click on the arrow to enroll in a course.

How do I enroll in SWoRD classes?

After logging into this system, click on the tab labeled 'Enroll Courses' where you pick a course.

Search for your course in the pull down box.

Once you have chosen your course, you will need to enter the CourseKey. This will be provided by your instructor.

Click ENROLL to finish.

How do I submit my writing?

From the “Writer’s Work” menu, click the “Submit Now” button for the appropriate assignment (first draft or final draft).

The following page is the submission page. Here you will be entering information about your paper, evaluating your own paper, and attaching the document.

Type the title of the paper.

Read the plagiarism statement. If you agree, click in the box next to “Check here if you agree”. (This step is MANDATORY to continue.)

Rate your draft on the three dimensions (Flow, Logic, and Insight).

Attach your document following the instructions online.

Click “Submit” once you’ve attached the document.

Once your draft has been successfully submitted, you should see a confirmation page. You should also receive an e-mail confirmation. Make sure to save your e-mail.

I can't login! What should I do?

First make sure that you are using the right account you used to create a SWoRD account and if you are using a full email address. Also make sure that you are using the right password. If you are still having trouble, click the Find ID or Password button on the main SWoRD homepage.

I can't upload my writing! What should I do?

Check your file size.

Check if your file is open when you submit the file. If it is open, close the file.

I am a Mac user. Is there any problem?

No.

I am a Netscape user. Is there any problem?

No.

I am a Safari user. Is there any problem?

No.

How do I know if my submission was successful?

There are at least three ways to know whether your submission was successful.

First, you should see a confirmation message on the site right after your submission.

Second, you should see your writing or reviewing status icon check-marked, which indicates your successful submission.

Third, you receive an email confirmation about your submission. Note that sometimes you do not receive the message for various reasons. If you do not receive confirmation messages, first check if your email inbox is over quota at the email account that you are using for SWoRD. Otherwise, report your problem using the Help Web Form to contact the SWoRD Support Team.

How do I submit my reviews?

Select the assignment tab and click on “Reviewer’s Work”.

Click “Review this paper”.

Enter your feedback for each of the three dimensions and then click the next arrow.

Select the rating for each of the three dimensions and then click on the next arrow.

*If a window pops out with several options for macros, choose Enable Marcos.

How do I review peer papers offline?

Access the reviewer’s work menu, click on the “Reviewer’s Work” button on the assignment page sorted by type OR click on the reviewing assignment listed on the assignment page sorted by date.

Download the Papers to Review

Select the assignment tab and click on “Reviewer’s Work”.

Click on the title of the paper to save a copy.

Review the Papers

Download the dimension guidelines. See instructor for more information.

Create your ratings and comments in a separate document (or use the dimension guidelines form). Pay attention to the specifics addressed in the dimension guidelines.

Please read your peer writing carefully with reference to the criteria given. 50 % of your reviewing grades are from your authors who will receive your feedback. She or he will decide how helpful your feedback would be in revising their writing.

How do I get to see feedback for my paper?

Please visit your writing assignment page where you submitted your manuscript. There you may read peer feedback. Note that at the same page, you evaluate their feedback back to the reviewers in terms of how helpful their feedback was/would be.

For Technical Support contact the SWoRD Support Team at sword@pitt.edu.