How many UX designers does it take to NOT spoil a design?

Is there a rule of thumb for how many UX designers should be working together on a project? Factoring in things like the size of the project (e.g. number of functions/screens), timeline of the project, complexity of the user interaction.

I would like to think that there should be at least one UX designer per project team, but two would be even better. It might also depend on where the strengths/weaknesses of each UX designer is. Theoretically one would think that multiple UX designers produce better results than a single UX designer, yet there are also issues such as too many people double handling things, or having different opinions about style and design directions (see Apollo syndrome.

I would like to know if there has been studies to identify problems when UX designers work by themselves or with other people?

UPDATE: Now that I have actually worked in a few different projects with other UX designers, I feel like I should provide an update to this question (with my own answer) now that I have a different perspective on this topic.