How many visually distinct colors can accurately be associated with a separated legend in a figure?

For charts and graphs it's a relatively common question by programmers to want to procedurally generate an arbitrary number of colors that are visually distinct.

Various research (or educated guesses) exist that say that the number of colors humans can differentiate is somewhere between 200,000 and 20 million. Obviously this varies to some degree by the person (and the presence or degree of any color blindness). However, this is based on whether or not a human can distinguish two colors side-by-side; it is separate from the concern of placing two boxes on the screen with very similar colors, repeating these colors in a legend somewhere, and determining whether or not a human can determine which color goes with which box.

I have done my own investigations into this and come up with a page for generating a set of colors with user-specifiable visual separation.

However, my question is: is there any research showing a "safe" and "reasonable" number of distinct colors that can be properly associated back to a chart legend?

Can You Tell Which Yellow Is Represented by the Yellow Line?