Inline validation on smaller forms

I'm trying to redesign the forms on our web app and run into an issue with inline validation messages. I started with putting the error message in the top right corner above the input and this worked initially, but then I realised on a smaller screen this was causing it to overlap with the label and then jump onto a new line, so I'm now looking at alternatives.

In terms of the UX of inline validation, is it imperative that the error message itself is always shown? When I first started looking at this part of the system, I tried a few options such as putting the validation under the input, in popups etc but they all had their issues:

• Under the input caused other inputs to move down the form when the error was displayed. This didn't feel very nice?
• Popovers covered content or again caused things to move
• Message inline with the label breaks when too small

So what I'm considering now, is to have the input show that its in an error state with the usual red border, and then place an error icon somewhere, but don't show the error message itself all the time. Instead, this would be shown when the user either focuses on the input in question, or hovers over the error icon.

Would this work from a UX stand point, or is it better to have the error visible all the time?