Tree testing during a redesign – which answers are correct?
I'm working with another UX person on reorganizing the information architecture for a large website. The IA work may eventually lead to a complete redesign of the website and will certainly impact its content.
I am the researcher running the studies on the IA. We have run a series of tree tests, including benchmarking, but the one we just ran has a mostly new set of tasks for a different audience. Some tasks have had multiple people nominate the same answers that we had originally thought were incorrect. Just like in the Atlassian wiki's guide, we've needed to add some new correct answers that we had missed initially.
We are still undecided about some other answers that we have frequently seen.
For example: There's a page called Contact Us. Let's say it has only a generic contact form for the entire organization in question. The task in the tree test involves finding someone in a specific department, which participants outside our organization would recognize (at least very generally). The correct answers initially dealt with that department, but about 20% of users nominated Contact Us. (This affects several other tasks too, but these are more complex, and changing the content on the pages would require more significant discussions with stakeholders.)
I've been saying that Contact Us is not the right answer for this task, based on its current purpose. Users won't find what the task is asking for on that page, and there is no cross link to pages where they can contact more specific people.
The other person (who designed the IA) is saying that Contact Us should be a right answer because the site is in a redesign and, if users are saying the content should be in that place, we should change the site to put it there. I'm concerned this may be a slippery slope, but I'm not sure.
In studying tree testing, I haven't seen anything on how to handle defining correct answers when the site's content itself is open to change. Are there any previous studies or best practices surrounding this?