User Permissions Management Screen

I am working on a challenging user management UI. The manage permissions screen here needs to do all of the following:

a. Allow the user to search for organizations and select any # of organizations

b. Allow the user to search for users and select any # of users

c. Bring up all orgs associated with the selected users

d. Bring up all users associated with selected orgs

e. Allow users to manage permissions listed below in the table

I am worried that c and d could easily create a snowball effect if they start populating all orgs associated with these users and one of those newly populated users has 10 more orgs ... and so on and so forth.

Are there any existing patterns or recommendations on how to handle this kind of situation?

Screen design

Update: Would it be a bad idea to add checkboxes that say 'show associated orgs' and 'show associated users' to prevent automatic population of associations? enter image description here