I need your opinion : "Previous category" button

I'm working on an e-commerce website and i'm faced to this issue : I need to use a "back to category" buttons on my products pages but some of these products are linked to 2 or 3 differents categories. So I don't know if I just have to chose the most relevant category or if I have to create a button that just carry the user to the previous page. Because in the first case, if for example the user comes from the category 1 and the button is linked to the category 2, he will be carried to the category 2. In the second case, the problem is that if the user comes from an external page, he will be carried out of the website.

Thank's for your help ! (By the way, I'm working on wordpress, especially with Elementor)