Slideout menu: touch enabled only on close

I'm building a mobile first webapp. I want to add a slideout menu for the main navigation.

The main page of the app is an interactive map, as the user will be moving around on the map a lot, I must deactivate touch for opening the slideout menu, otherwise the user will be opening it by mistake quite often.

Using touch to slide out the menu is a nice feature for mobile users. So, I was considering enabling touch for the rest of the site, and enabling it only for closing it on the map page. Would this seem confusing for the user ? Should I just stick to a button for toggling the slide out menu ?