Seconds before is normally acceptable, as it's done before server and receiver are ready for the play. Otherwise, it would be called hindrance and probably be a foul. The referee call will be based upon the intention of the player. You may move slightly or adjust your position but many steps that will disturb the players are not acceptable sportsmanship.
From United States Tennis Association
-- The Players' Guide to Fair Play and the Unwritten Rules
35. Body movement. A player may feint with the body while a ball is in play. A player may change position at any time, including while the server is tossing a ball. Any other movement or any sound that is made solely to distract an opponent, including, but not limited to, waving arms or racket or stamping feet, is not allowed.
According to what you describe, this is clearly out of bounds. Foul. The move has to be done before the server makes any move.