If in a penalty shoot out the ball hits the goalpost and lobs outside the penalty area and then returns back into the goal without touching any player, will it be considered a goal or not? Please provide a reference to FIFA rules if possible.
On page 55 of the 2014/15 FIFA Laws of the Game, in the Kicks from the Penalty Mark procedure, it says
Unless otherwise stated, the relevant Laws of the Game and International F.A. Board Decisions apply when kicks from the penalty mark are being taken.
This means that, since no other description is given, Law 14 (governing Penalty Kicks during the game) applies to this situation, with the exception of the placement of extra players.
On page 45, Law 14 (The Penalty Kick) says, in regard to a goal being scored:
A goal is awarded if, before passing between the goalposts and under the crossbar the ball touches either or both of the goalposts and/or the crossbar and/or the goalkeeper.
The referee decides when a penalty kick has been completed.
So, in the situation you describe the ball hits the goalpost/crossbar, hits the ground and then goes back into the goal. This would be a goal unless the referee had decided that the kick was completed. If the ball had landed all the way outside the penalty area (very unlikely), it's possible the referee would have blown the ball dead since it was so far from the goal. But if he hadn't, it's a goal.