This is a sortable version of Table3 from The timing mega-study: comparing a range of experiment generators, both lab-based and online
Timing summaries of web-based software by package, platform, and browser. The Var(iability) measures are the inter-trial standard deviations of the various latencies for that configuration. The table is sorted by the mean of those variabilities (Mean Var). The Lag/Bias measures are the mean latency values, for that configuration. In the case of audiovisual sync, a negative bias indicates the audio lead the visual stimulus, a positive bias means the visual lead the audio. Each of the values with a hyperlink will lead to a plot of the distribution of values leading to that summary value.
Click the headers to sort the table.
The first time you click, the sorting direction is ascending (A to Z).
Click again, and the sorting direction will be descending (Z to A):
| Package | Platform | Browser | Mean Precision | RT var | RT lag | Visual duration var | Visual duration bias | Audiovisual var | Audiovisual bias |
|---|---|---|---|---|---|---|---|---|---|
| PsychoPy | Win10 | Chrome | 1.36 | 0.39 | 43.95 | 0.67 | -2.08 | 3.01 | 65.32 |
| Gorilla | Win10 | Firefox | 1.84 | 1.11 | 24.83 | 2.67 | 1.35 | 1.73 | 88.27 |
| Gorilla | MacOS | Firefox | 2.18 | 4.47 | 30.34 | 0.94 | 1.16 | 1.12 | 38.43 |
| PsychoPy | Win10 | Edge (Standard) | 2.22 | 2.03 | 42.00 | 0.93 | -2.28 | 3.69 | 56.19 |
| PsychoPy | MacOS | Firefox | 2.65 | 1.17 | 67.01 | 3.38 | 0.24 | 3.40 | -10.21 |
| PsychoPy | MacOS | Safari | 2.66 | 1.05 | 33.50 | 4.26 | 0.49 | nan | nan |
| PsychoPy | Win10 | Firefox | 2.76 | 1.96 | 40.97 | 2.42 | -2.61 | 3.90 | 58.93 |
| Gorilla | Ubuntu | Firefox | 2.76 | 4.71 | 24.71 | 2.35 | 2.05 | 1.23 | -30.61 |
| jsPsych | MacOS | Safari | 3.39 | 0.66 | 31.31 | 4.39 | 3.09 | 5.11 | -23.48 |
| jsPsych | Win10 | Edge (Chromium-based) | 3.85 | 1.74 | 15.19 | 4.21 | 2.97 | 5.60 | 44.51 |
| Testable.org | Win10 | Firefox | 3.92 | 3.87 | 31.36 | 2.94 | 1.91 | 4.95 | 76.32 |
| PsychoPy | Ubuntu | Firefox | 3.97 | 1.57 | 42.50 | 4.97 | 1.02 | 5.36 | 190.45 |
| Testable.org | Ubuntu | Firefox | 4.05 | 3.97 | 31.57 | 3.25 | 1.84 | 4.92 | -44.36 |
| PsychoPy | Ubuntu | Chrome | 4.14 | 0.20 | 66.98 | 1.77 | 1.77 | 10.45 | 187.19 |
| Lab.js | MacOS | Firefox | 4.20 | 0.97 | 16.38 | 8.61 | 19.51 | 3.01 | 4.26 |
| PsychoPy | Win10 | Edge (Chromium-based) | 4.24 | 1.04 | 46.01 | 3.03 | -3.36 | 8.66 | 63.30 |
| jsPsych | Ubuntu | Chrome | 4.63 | 3.23 | 48.29 | 4.33 | 4.05 | 6.34 | 27.73 |
| PsychoPy | MacOS | Chrome | 4.84 | 3.22 | 35.29 | 6.47 | -0.30 | 4.82 | -6.86 |
| Lab.js | Ubuntu | Chrome | 5.12 | 8.27 | 31.79 | 1.19 | 2.34 | 5.91 | 198.05 |
| jsPsych | Ubuntu | Firefox | 5.12 | 4.11 | 31.38 | 4.84 | 4.05 | 6.43 | 10.55 |
| jsPsych | MacOS | Firefox | 5.16 | 6.85 | 53.70 | 2.57 | 3.27 | 6.05 | -15.29 |
| Testable.org | Ubuntu | Chrome | 5.46 | 4.23 | 47.09 | 7.06 | -11.92 | 5.09 | 94.18 |
| Testable.org | MacOS | Chrome | 5.52 | 3.99 | 41.94 | 7.43 | 6.34 | 5.13 | 67.20 |
| jsPsych | MacOS | Chrome | 5.62 | 5.68 | 41.42 | 5.74 | 3.25 | 5.45 | -9.71 |
| Lab.js | Win10 | Firefox | 5.78 | 7.88 | 8.22 | 4.21 | 14.25 | 5.25 | 70.93 |
| Lab.js | MacOS | Chrome | 5.79 | 3.26 | 20.30 | 8.78 | 6.51 | 5.31 | -0.21 |
| Gorilla | Win10 | Edge (Standard) | 5.96 | 4.95 | 40.23 | 7.15 | 4.95 | 5.79 | 70.93 |
| Testable.org | MacOS | Firefox | 6.10 | 5.20 | 57.66 | 7.63 | 22.83 | 5.48 | 40.73 |
| Lab.js | Ubuntu | Firefox | 6.19 | 2.91 | 25.54 | 10.22 | 13.66 | 5.44 | 185.35 |
| jsPsych | Win10 | Chrome | 6.23 | 7.85 | 23.27 | 5.10 | 3.60 | 5.73 | 43.57 |
| Testable.org | Win10 | Edge (Chromium-based) | 6.80 | 4.11 | 15.99 | 8.34 | -5.39 | 7.94 | 73.79 |
| jsPsych | Win10 | Firefox | 7.38 | 8.37 | 25.70 | 7.04 | 15.32 | 6.74 | 32.32 |
| Gorilla | Win10 | Chrome | 7.89 | 3.58 | 25.60 | 5.03 | 4.24 | 15.06 | 98.84 |
| Testable.org | Win10 | Chrome | 8.08 | 7.88 | 23.96 | 8.38 | -5.90 | 7.98 | 72.57 |
| Lab.js | Win10 | Edge (Chromium-based) | 8.57 | 4.22 | 17.14 | 8.03 | -4.90 | 13.45 | 82.45 |
| Lab.js | Win10 | Chrome | 9.48 | 8.44 | 19.20 | 7.76 | -1.73 | 12.25 | 86.03 |
| Gorilla | MacOS | Chrome | 9.76 | 5.30 | 35.31 | 5.36 | 3.40 | 18.61 | 32.03 |
| Gorilla | Win10 | Edge (Chromium-based) | 11.34 | 4.89 | 23.40 | 3.01 | 1.56 | 26.13 | 121.57 |
| Gorilla | Ubuntu | Chrome | 14.17 | 0.43 | 40.85 | 1.66 | 3.36 | 40.42 | 200.55 |
| Gorilla | MacOS | Safari | 19.16 | 1.53 | 29.65 | 30.11 | 22.25 | 25.83 | 285.81 |
