Visualize an object as non movable

I am currently creating a slider for numerical values. The idea of the slider is, that values are changed by moving the ribbon. Some tests showed that users mostly tried to move the indicator itself and not the ribbon.

Current state of the slider:

How would I visualize that the indicator is not movable?