How to avoid the "local elevator" problem?

I live in a high-rise apartment building in New York. Our building has four resident elevators which each serve every residential floor.

However, "elevator #1" is special - it is the only one of the four that can reach the rooftop, as well as the only one of the four that can reach the parking garage in the basement. It is also slightly larger in size than the other 3 elevators, to accommodate wheelchair-using residents and strollers.

As such, the elevator banks on each floor have been outfitted with two elevator call panels - the one in the center of the bank (panel "B") can call any of the four elevators, while the buttons on the panel next to elevator 1 (panel "A") specifically call only elevator 1:

          1               2                 3               4      
    +-----+-----+   +-----+-----+     +-----+-----+   +-----+-----+
 A  |     |     |   |     |     |  B  |     |     |   |     |     |
+-+ |     |     |   |     |     | +-+ |     |     |   |     |     |
| | |     |     |   |     |     | | | |     |     |   |     |     |
| | |     |     |   |     |     | | | |     |     |   |     |     |
+-+ |     |     |   |     |     | +-+ |     |     |   |     |     |
    |     |     |   |     |     |     |     |     |   |     |     |
    |     |     |   |     |     |     |     |     |   |     |     |
    +-----+-----+   +-----+-----+     +-----+-----+   +-----+-----+

A problem commonly arises (especially during peak times) that residents will walk by both panels and select their desired direction of travel on both, even if they are intending to go to a floor which is serviced by all 4 elevators. This becomes a challenge when those riding in elevator #1 are now stuck stopping at every floor in which another resident has pressed a button on the leftmost panel, whether or not the resident has already boarded one of the other three elevators headed in their direction. In these peak periods, this can easily add 2-3 additional minutes to a resident's travel time, effectively becoming the "local" elevator (as opposed to the "express" - for anyone who's ridden the subway, you'll certainly know the difference).

How can residents be guided to use the special panel ("A") as intended (only when intending to ride elevator #1)?