Please use our GitHub Issues page for all new support inquiries. If you need to contact us directly, please e-mail [email protected]

I'm trying to write a script for inter-rater reliability that would allow me to go from 100% inter rater to then a lower percentage (such as 25% inter-rater). Does anyone have any advice on how to write a script for this manipulation?

asked 04 Nov '13, 14:39

ugraksw's gravatar image

accept rate: 0%

There's actually a function to do this already programmed in, so depending on the structure of your data you may just need to write a script that uses that function. It will create a new column, inserting a cell for however many of the primary cells you set it to (for example, if you want 25% reliability, it will insert a rel cell for every fourth primary cell). It can also copy over any arguments you don't need the reliability coder to fill in.

make_rel(RelColumn, PrimaryColumn, 4, "onset", "offset")

In this example, it will create a new column called "RelColumn" and insert a cell in RelColumn for every fourth cell in the primary column. It will copy over the onset and offset that primary coder set to the new cell, but leave all the other arguments blank for the rel coder to fill in. If you want the rel coder to set the offset, simply remove that from the list of arguments at the end. Or if you want other arguments to be copied over, e.g., trial number, just add them to the list.

If you have a more complicated set up (or if you want the rel coder to not be given either of the times the primary coder set), you might need a script that's a little more complex. If that's the case just let us know and we can help you with a more detailed script.

permanent link

answered 04 Nov '13, 15:39

WhitneyCole's gravatar image

WhitneyCole ♦
accept rate: 21%

Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here



Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text]( "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:


question asked: 04 Nov '13, 14:39

question was seen: 4,416 times

last updated: 04 Nov '13, 15:39

powered by OSQA