Should users be allowed to use any special character they want when creating a password?

I came across a number of login configuration settings where there is a list of allowable special characters and was wondering:

Does this limitation cater for a specific security or usability need?

Example: A list of special characters supported by Oracle Identity Manager and Microsoft Active Directory for password field :

Update:

Thanks everyone for the generous response!

Every time I have asked a question that involves security and usability there seems to be a clear divide between proponents on each side. However this need not be as this is one area that requires a lot of compromises and trade-offs… UX depends on it!