Messaging A Site Redesign To Users: Does This Taint A/B Testing Data
The company I work for (a deal roundup site) has redesigned the look of our deal information tiles. (Changing the layout of how the item's image, price, and other information is arranged.) We're going to A/B test to see if the new design is more engaging to our users... however, there's some debate as to whether or not we should message our users about this change, before they see it. The anti-messaging people fear that priming an audience before such a change will yield tainted test data. Everything else I've read online so far says it's dangerous not to prime an audience before a UI change.
So, my question is this: Does messaging a UI change before it's implemented sully the results of an A/B test in any way? Are there any pros to pulling a surprise UI change on users?
Thanks, in advance, for any help or advice. I'm a newly-minted UX Researcher and I'm glad to have found your community, as I'll need all the help I can get!