Alphabetical grouping of accented words

We have a list of countries in our UI where the design prototype groups the countries by initial letter. The country names change by language, so the grouping also changes. This also means that the groups themselves change.

In some languages, country names start with accented characters, e.g Austria in English, and Österreich in German. How is it more common to group these words: should Österreich belong to the "O" group or to the "Ö" group? Are there different traditions in different countries (I often hear Germans refer to "a" and "ä" as different letters)?

Another example in Portuguese: Alemanha and Áustria both start with A and no one would expect them to be in separate groups.