I've seen this happen quite a few times in the 2014 FIFA World Cup - the referee accidentally trips or bumps into a player but nothing is called.
An example of this was USA vs Belgium. The referee actually trips number 13 when he gets open for a shot at net. The player is unharmed but he is very disappointed. Later on, he is making a solo run when he bumps into the referee, losing possession.
What should happen?
In football, the referee and assistant referees are considered to be part of the field, just like the corner flags, crossbars and goalposts. In the case of the ball hitting the referee, there are two passages that support play continuing.
On page 33 of the 2014/15 FIFA Laws of The Game:
The ball is in play at all other times, including when:
- it rebounds off a goalpost, crossbar or corner flagpost and remains in the field of play
- it rebounds off either the referee or an assistant referee when they are on the field of play
Also, on page 106:
If, when the ball is in play, it touches the referee or an assistant referee who is temporarily on the field of play, play continues because the referee and the assistant referees are part of the match.
As for a referee contacting a player, there is nothing explicitly written in the Laws about this. However, the only way a free kick or penalty kick can be awarded to a team in football, is if a player, substitute or substituted player on the other team commits an infringement, so this is out of the question.
As for a potential dropped ball restart, this would only occur if the player and/or referee was injured as a result of the collision. Page 31 covers when a dropped ball is appropriate:
A dropped ball is a method of restarting play when, while the ball is still in play, the referee is required to stop play temporarily for any reason not mentioned elsewhere in the Laws of the Game.
As the referee is not required to stop play, it is not stopped and hence play continues.
In football (you should ask a separate question for the hockey) the game plays on as if the ball had been touched by a player. It happens quite a lot when, for example, in crowded areas of the pitch the referee can bump into players or be hit by the ball. In all cases, the team that has a disadvantage will feel aggrieved but it's just part of the game.
