When should the error state on an input field disappear?

I've seen two types of behaviour circulating; one where the error state disappears as soon as you start typing a new input (so when the input field is in focus), and one where it stays until the input is 'accepted' (for example, a valid email format is typed).

I can imagine it is beneficial to keep the error state's context until the user has fixed their input, but my team has built it the other way around.

Which interaction is preferable and why?

Edit: Here's the error state I currently use:

Currently the built version removes the error state (and returns it to its normal state) as soon as it is in focus. I intended for the error state to stay until the input of the field had been 'corrected'.