Please use our GitHub Issues page for all new support inquiries. If you need to contact us directly, please e-mail [email protected]

Our study is interested in the time that a mom talks to a child in a one hour video. We have two coders coding 100%, and each coder codes whenever the mom starts and stops talking. However, a coder could have a discrepancy in onset or offset time or miss some of the talking resulting in a difference in ordinal number.

We have tried the reliability script given on the website, but this didn’t work because there may be a different number of cells. We tried the mutually exclusive script, but do not think this is giving the information we need. We are wondering if there is a script that will compare cells between two columns based on a more flexible time window- that would compare two cells if, for example, the onset time is within 100 milliseconds of each other.

There was a similar question titled “more complex interrater reliability question” but this did not quite answer our question.

asked 09 Jun '15, 10:03

Dani's gravatar image

accept rate: 0%

Here is an example of a script that we use. Feel free to edit it to make it work with your data.

require 'Datavyu_API.rb'


agree = 0 total = 0 disagree = 0 dir = File.expand_path("~/Desktop") out_file = + "/CheckRel.txt", "w")

  #id = getVariable("id")
  mutex = create_mutually_exclusive("MutexGaze", "look", "rellook")

  #mutex = getVariable("MutexGaze")
  rblock = getVariable("relblock")
    touch = getVariable("touch")

  for rcell in rblock.cells
    for mutexcell in mutex.cells
      if mutexcell.onset >= rcell.onset and mutexcell.offset <= rcell.offset
        if (mutexcell.look_loc != mutexcell.rellook_loc) or (mutexcell.look_rlo != mutexcell.rellook_rlo)
          disagree = disagree + (mutexcell.offset - mutexcell.onset)
          if (mutexcell.offset - mutexcell.onset) >= 100
            out_file.syswrite ("Disagreement at look ordinal " + mutexcell.look_ordinal + " and rellook ordinal " + mutexcell.rellook_ordinal + " of " + (mutexcell.offset - mutexcell.onset).to_s + " ms" "\n")
            puts "Disagreement at look ordinal " + mutexcell.look_ordinal + " and rellook ordinal " + mutexcell.rellook_ordinal + " of " + (mutexcell.offset - mutexcell.onset).to_s + " ms" "\n"
            # out_file.syswrite ("Disagreement at mutex ordinal " + mutexcell.ordinal.to_s + " of " + (mutexcell.offset - mutexcell.onset).to_s + " ms" "\n")
            # puts "Disagreement at mutex ordinal " + mutexcell.ordinal.to_s + " of " + (mutexcell.offset - mutexcell.onset).to_s + " ms" "\n"
        #elsif (mutexcell.gaze_loc == mutexcell.relgaze_loc) 
        #  agree = agree + (mutexcell.offset - mutexcell.onset)
    total = total + (rcell.offset-rcell.onset)

mutex = getVariable("MutexGaze")

percentagree = (total.to_f-disagree.to_f) / total.to_f out_file.syswrite ("Percent frame match = " + percentagree.to_s) out_file.syswrite ("\nTotal time coded = " + (total.to_f / 1000).to_s) puts "\nPercent frame match = " + percentagree.to_s + "\n" puts "FINISHED PRINTING DISAGREEMENTS" setVariable("MutexGaze", mutex) end

permanent link

answered 09 Jun '15, 17:04

Vicky%20Foo's gravatar image

Vicky Foo ♦♦
accept rate: 1%

Thank you very much for your response. We just have a couple of questions regarding the script you've given us.

We’ve tried editing the script in order to fit our situation but ran into a few issues. Would you mind telling us the Datavyu setup that you used this script for? For example, the names of the columns and codes in the Datavyu file. This would be very helpful as we are having issues understanding, for example, what things like “touch” and "relblock" in the script refer to. In any case, we appreciate your help a lot.

permanent link

answered 11 Jun '15, 10:19

Dani's gravatar image

accept rate: 0%

Absolutely! Can I please have your email so I can send you the datavyu file that would correspond to the script?

(11 Jun '15, 12:46) Vicky Foo ♦♦ Vicky%20Foo's gravatar image

You can send it to [email protected] Thank you so much!

(11 Jun '15, 12:54) Dani Dani's gravatar image
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here



Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text]( "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:


question asked: 09 Jun '15, 10:03

question was seen: 11,699 times

last updated: 17 Jun '15, 00:17

powered by OSQA