Best practices for designing layouts of highly configurable/customizable web applications

As the title says, I'm looking for best practices or guidelines on how to design highly configurable web applications.

We're currently doing a complete redesign of our HR application and there isn't a single screen without discussion about the issue of configurations possibilities. Customers love that they can tailor our platform to their needs; in fact it's one of our main selling points. But how can you ensure an aesthetically pleasing layout when you don't know the number of buttons that will be displayed, or how many options will be available. Or how complex the search masks will be...you get the idea.

Examples for configuration possibilites are:

  • QuickActions (one button per action)
  • QuickFilters (checkboxes that activate a set of filters tailored to your needs)
  • labels and images
  • visibility of certain functions or contents
  • etc.

Our current strategy is to make sure that the default configuration looks and feels good and we even validate our designs through user testing. However, since the possibilities of configuration are nearly endless, we can't possibly test for all cases. We try to take extremes or corner cases into account but for "badly" or overly configured pages, we can't ensure that the layout will still be alright.

Are there any best practices or design guidelines about how to ensure a minimum amount of usability and aesthetic? How are you coping with that kind of "fuzzyness" in your layout? Any experiences or ressources are highly appreciated!