Why does the game stop after the kicking team catches a punt?

Question

I am a occasional football watcher from Europe, so please excuse my stupid question.

My question is the following: The standard outcome of a punt is a catch by the defending team and some running before he goes down or out of bounds. Sometimes, however, a member of the punting team happens to catch the punt, in which case the defending team gets possession where the catch happened. I don't really understand why the play does not continue in this case. What does such a catch distinguish to a 'normal catch'? I have the following thoughts:

• Kicking forward is generally allowed and could theoretically be done in 'normal play'. [EDIT: This assumption turned out to be wrong.]
• Sometimes, kick-offs are done in a manner to maintain possession (I guess this is called on-side kick?). If it's allowed in this case to catch the ball, why not after a punt?
• Kicking the ball wide forward (and usually giving up possession) is often seen in Rugby as a means to open up the game. If a member of the kicking team gains possession, the game just goes on.

Based on my understanding, I'd hence assume the kicking team would at least be granted a new first down. I don't see a reason why the possession would change in this case.

Answer 1

This is covered in Rule 9, Section 3, Article 2, Item 2 of the NFL Rules.

Illegal Catch or Recovery. If the kickers catch or recover a kick beyond the line that has not been touched beyond the line by the receiving team, the ball is dead, and it is first-and-10 for the receivers at the spot of catch or recovery.

In your scenario, the ball is dead because the punting team recovered a kick beyond the line of scrimmage without the receiving team touching the ball. Note that this is different from a kick-off, which can be recovered by the kicking team after the ball has travelled 10 yards.

Answer 2

If I understand your question correctly, you're asking why play stops if the punting team touches the ball. This is because it is technically a penalty for illegal touching for the punting team to touch the ball on a punt first.

Now, because it is a penalty, the receiving team is in fact allowed to advance the ball if it merely touched and not "downed", and anything bad that happens is null (similar to interceptions and fumbles being wiped out because of a defensive penalty). Most of the time, there isn't an opportunity to do so and the play merely ends. Sometimes, the punting team doesn't pay attention and a savvy player can scoop up the touched ball and run with it.