So, I am working with a dataset that contains questions from a form filled by two different users. Now I want to compare the results of these two fills based on the 'answer value' field - meaning what did user 1 select and what did user 2 select. My goal is to find the bias or variance between these two users in these questions and how they scored.
Here's an example - The interaction ID is my anchor point which tells me that it's the same evaluation these two evaluators looked at.
The output (or the Precision score) is what I am trying to get from my dataset.
Note: Orange is my expert evaluator and is evaluating if Apple is filling the forms correctly.
Using pivot tables I can construct a result like a table I want, but how to get the matched field?
My other thought is in beast mode do a calculation for Precision score like
Case when
Interaction id is the same
question id is the same
if the answer value is different - count distinct question id
then divide by
total number of distinct question id
Now if I select an evaluator can I see the precision score? - Like evaluator Apple's precision score is 60%. Note: Orange is my expert evaluator and is evaluating if Apple is filling the forms correctly.
