Selecting the right user testing method for mobile app

We have a mobile app for different platforms. And now we want to improve the UX and UI. Due to the security policy, we cannot install analytic tools. I'm thinking of options we have to measure our design improvements impact: One-on-one A/B testing with 20 users per design to measure speed, number of errors, satisfaction rate etc. That's about 40 user tests per design update, which seems to be not realistically for our company. Any more variants or thoughts? Once in a few months we have 1 negative feedback about the design like "I don't like it" and some more feedback like "I like it".