What is best practice for designing form error messages?

I've seen quite a lot of research on form design, but so far, I haven't come across any studies on error message design best practices. The only piece of advice seems to be that each error should clearly be associated with the invalid form field that it aims to correct. Has there been any research on this?

I have some specific issues I'm trying to work through and would appreciate any insight or resources.

Error message location

When you have live form validation, where should an error (or success) message appear in relation to the form element?

Some options:


To the right of the input

mockup

download bmml source – Wireframes created with Balsamiq Mockups


To the right of the label

mockup

download bmml source


Above the label

mockup

download bmml source


Below the label

mockup

download bmml source


Below the input

mockup

download bmml source


There are some thorny issues that also need to be taken into account regarding location:

  • How are errors displayed for radio or checkbox groups?
  • How wide is the error message?
  • Is the error message multi-line?

Error message visibility

Besides location, I'm also interested in how the visibility of the error message affects its usability. Does it matter or is it strictly a aesthetic decision? Consider the following 3 options:

mockup

download bmml source

Edit: There have been some excellent points on accessibility in some of the answers so far, as well as some interesting examples. However, I really would like to see some data from actual studies, if possible. Also, none of the answers so far have really addressed the two questions I had above:

  • Where should the error be located in relation to the input and label?
  • How does the design and visibility of the error message itself affect its usability?

Regarding the accessibility of the forms, it seems to me as if there might be two potentially conflicting use cases, live validation and server-side validation (which would require reloading the page). If the page is reloaded and the user again starts at the top of the form, an error summary makes sense, especially from an accessibility perspective. However, in the case of live validation, the error summary at the top makes much less sense, both for form design and for accessibility purposes.

Consider how a user would actually use a form with live validation:

The validation occurs on the blur event. Usually, the focus of will be on the next form element, not the list at the top. As I understand screen readers, there wouldn't be any reason to return to the top of the form to read the error list. For users not using screen readers, the error list at the top pushes the entire form down, doubly so if the error takes up vertical space near the form field as well. In longer forms, this is a usability issue as the error summary may not be visible. In shorter forms that have per-field errors, like a login, the error summary just looks like overkill. For all forms, the interface would be pushed down, making a poor user experience.

For live validation, if you really want to make the form errors accessible, I think using ARIA Live Regions might be the preferred solution.