Summative user tests that allows you to compare against competitors?

I want to understand when it would be appropriate to conduct summative users tests with the focus on evaluating the performance of our product against our competitors?

We have a finished live product which we know can be improved in many ways and want to think of a way to evaluate ours against others in the market. We've done some competitor analysis but want to see how people use our product and what they like/dislike. Would this kind of user testing work?