Should tabs changes in a web app be part of the browser history?

I've implemented a JavaScript plugin that tracks tab changes in through the browser history. Here's a brief overview of how it works:

• Once tab is clicked, the URL hashtag (the part that looks like #hashtag) changes and the tab's content appears.
• Clicking the back button brings up the previous hashtag and the previous tab is given the focus.

Now that I have it working, I'm rethinking whether or not I should use it from a usability perspective. On the one hand, I have a legitimate use case for it: clicking on in-page links will both scroll to the appropriate section and open the appropriate tab. This is a very useful feature for my app and I would have a hard time abandoning it.

On the other hand, if one has looked through a lot of tabs (some pages could have a lot of sections, each with its own tab menu) on a page and then tried to use the browser's back button, I could imagine a user growing frustrated pretty quickly if they have to navigate through 20+ tab changes just to get to the previous page.

I'm leaning toward keeping the tab history functionality as it would be useful across my app. Also, there are only a few pages for my app and they are easily accessible from the menu, so that mitigate the problem to a degree. Before going down that path, however, I'd like to know if there is a better way of doing this, what my trade-offs may be, or if it's even a real problem.

Are there any usability studies on this particular issue? Any experiences or best practices that you could share?