Those movements are treated the same as a shift prior to the play or a tight end going into motion. From page 98 of the college football rule book (emphasis mine):
a. If a snap is preceded by a huddle or shift, all players of the
offensive team must come to an absolute stop and remain stationary in their
positions, without movement of the feet, body, head or arms, for at least one
full second before the ball is snapped (A.R. 7-1-4-I) [S20].
b. It is not intended that Rule 7-1-4-a should prohibit smooth, rhythmical
shifts if properly executed. A smooth cadence shift or unhurried motion
is not an infraction. However, it is the responsibility of an offensive
player who moves before the snap to do so in a manner that in no way
simulates the beginning of a play. After the ball is ready for play and
all players are in scrimmage formation, no offensive player shall make
a quick, jerky movement before the snap, including but not limited to
- A lineman moving his foot, shoulder, arm, body or head in a quick,
jerky motion in any direction [S19].
So, it appears that the spirit of the rule is not to prevent movement before the snap of the ball, but to prevent the offensive players from simulating the snap. This, to me, largely seems like a judgement call on behalf of the officials but it is pretty obvious that when the whole offensive line turns around to look at the sideline that they are not attempting to simulate a snap. I'm assuming that the spirit of the rule is the same in the NFL as referenced by corsiKa's answer.