How do I treat an ‘Incomplete’ in a Usability study?
I conducted a usability test where a user is given a set of tasks and we score them based on task completion (completed with ease, completed with minor difficulty, completed with major difficulty, and did not complete task).
However, recently I have encountered a problem where in some cases users never complete the task, so I have been marking them as incomplete. This can be due to running out of time so they never even were presented the task or they were taking so long that I had to move on from the task. In the case where they are taking long, one can argue that it should be marked as a fail since they failed to complete the task however another might argue that if given more time they would have figured it out and that we are testing the material and not the person.
A reason this is becoming an issue is because when I report out data it can be viewed not representative of the actual results. For example if we had a total of 8 users and for Task 1, 6/8 users passed but for Task 6, 4/6 users passed because 2 users never completed the task (due to having to move on but they had difficulty with it) how should I treat this data? What are the best practices for this scenario.