Rule 4.4f of Rugby Union states that "a player may not wear shorts with padding sewn into them". What is the rationale behind this rule?
Padding was traditionally not permitted at all, except for scrum caps. Soft, thin shoulder pads covering only the shoulder and collar bone are a relatively new addition to the game, coming along shortly after the transition to professionalism. In World Rugby Regulation 12, which specifies the technical requirements for permitted padding, the introduction reads:
The intent of this standard is to specify requirements ... without compromising the form or appeal of the game.
Basically the concern is that bringing padding into the game would turn it into something else, for example the way American football players tackle by just launching themselves as a missile into an opposing player and hoping they hit them hard enough they fall over without having to be brought down in a controlled manner.
TO add to MattW it also has to do with the physics of how rugby union tackles happen.
A lot of rugby tackles have do with position your shoulders at the ball carriers hips and driving his feet from under him. There is quite a large amount of force that goes trough your shoulder to stop these rampaging giants from moving forward.
I would think that the rugby lawmakers thought that any sort of stiff padding in the lower half of the body would adversely effect players shoulders who always had to take a fair amount of punishment anyway.
In regards to the padding that is allowed generally it is considered poor tackling technique to tackle as high as shoulders. These padding protect players against the occasional high tackle or shoulder charge. While not effecting the vast majority of technically correct tackles.
Here is a few pictures to demonstrate my point.