A/B-test interpret goals down the funnel
How should you interpret goals set further down the line? One part of the hypothesis is by changing the UI-controls for navigation and visual weight of secondary elements so users are able to easily navigate to target product. Well setting goals on control and variant does have the expected results. Items with less visual-weight will trigger less clicks, changing navigation-controls to more familiar patterns will left navigation. All these goals are significant and fine (G1 to G4).
As you can see G6 and G7 are down the funnel. Both control and variant follow the same path. Conversions on both control land variant are pretty low; Onestepcheckout control: 65/1155 (change to beat all 83%) and variant: 61/1190, Checkout control: 105/1155 (change to beat all 71%) and variant 95/1190. Based on G1, G2, G3 and G4 with 100% significance I would advise to adopt variant. Conversions on G6 and G7 are rather low. Test started about 1 month ago. It will take probably couple of weeks before significance on G6 and G7.
I've got 2 questions;
- How would you approach Goals set further from the actual changes?
- Are goals set further downstream shared by both control and variant any good?