It comes down to "who had position first?" If you get into position to field the ball, then you are allowed to maintain it fairly.
Obviously, the referee's interpretation of the contact will not always be perfect. But generally, a player who gets into position to play the ball first can maintain the position so long as he does not do anything dangerous in the challenge.
In fact, you arre allowed to impede when the ball is within playing distance. If you look through interpretations of Law 12, it's spelled out:
Impeding the progress of an opponent means moving into the path of the opponent to obstruct, block, slow down or force a change of direction by an opponent when the ball is not within playing distance of either player.
All players have a right to their position on the field of play, being in the way of an opponent is not the same as moving into the way of an opponent.
So, again, the whistle in this case is often not blown because of impeding play due to an arm bar, but rather due to charging or pushiing which is considered dangerous contact.