Which layer to end tree test on?
I'm doing a Tree Test on a product that runs embedded software. I am faced with a dilemma on how to design the tree and where to have the designated "correct response" be. I am using Optimal Workshop (Tree Jack).
We have constructed this information architecture based on the results of a previous card sorting study. I should note that users of the tree test may or not be familiar with the product and may not be familiar with the newly added features in this tree test. Additionally, users typically receive training prior to using the product and will receive no training on the tree test.
What is the recommended best-practice?
Choices are as follows:
1) Have the "correct response" in the tree be the actual item in question.
Example: Find where you can change your alert volume.
Device Settings > Volume > Alerts
During normal system use the user would then be able to make adjustments to this setting after opening the menu.
The pros of this choice seem to be that it will be able to simulate how a user may search through the navigation on the embedded product in that if they access the wrong menu they would then go back. Through the results we would be able to see if they navigate down a wrong menu before eventually finding the correct answer.
2) Have the "correct response" be the menu which contains the feature/item.
Example: Find you where you can change your alert volume.
Device Settings > Volume
The pro of this seems to be that it will put more emphasis on testing if the label clearly communicates what it contains, choosing the correct label, and if it contains what user's believe it contains.