Add another item – Update existing item

I'm creating a form which will allows users to filter job applications using screening questions. The user will be able to add up to 5 yes/no questions. The issue I'm having is deciding what most appropriate behaviour should be when a user wants to edit an existing question.

This is what the user will see if they add 3 questions.

If the user wants to edit question 2, the view changes to this:

Whilst in edit mode, I've decided to disable the other options (edit, add new) to prevent the user from clicking elsewhere. To exit this state, the user will have to click on cancel. Although, this will allow me to skip building a mechanism to hold half edited questions in a temporary state, it feels like a bit restrictive. Any guidance would be greatly appreciated.