The major reason players pitch on four days rest is that they are better pitchers when they do so. They are able to throw harder, throw more pitches, and throw more difficult pitches (with more spin/etc.) than they would if they had less rest. They are also less prone to injury.
Some evidence exists that pitching on three days rest in the postseason (where it's somewhat common); for example, this article at BleacherReport finds pitchers pitching on three days rest (who are usually the ace or #2 pitcher of a team) have a 4.48 ERA and win less than 40% of the games they pitch. Some of this may be pitchers being unused to pitching in those circumstances (and pitching against fairly good teams, it being the postseason); but it does seem logical that pitching on short rest is somewhat harder.
The other major factor is the fact that for most teams, the fifth starter isn't all that much worse than the fourth starter. While a few teams might have four great starters and then a poor fifth starters (the 2014 Tigers post-Price trade for example), the large amount of starting talent plus the ability to use multiple relievers means you wouldn't lose much by starting that fifth starter, particularly if it allows you to have better performances on the other four days.
If the question is what's the difference between then and now, you'll probably find that the major difference is in modern strength techniques allowing pitchers to throw that much harder; this both increases the number of pitchers who can competently pitch, and increases the benefit from rest - I'd rather have my starter hit 94 on most of his pitches every fifth day, rather than throw at 92 every fourth day.
The other issue with this is the premise of the question - namely, that the four man rotation was very common 20-30 years ago. This is simply untrue.
Reading SABR's Origins of the Pitching Rotation by Frank Vaccaro, the 5 man rotation existed in the 1930s, and even in the 1930s was the most common kind of rotation. See the chart (unfortunately very low DPI) in that article; by 1935 more teams pitched with a five man rotation than a four man rotation. If I have some time (maybe on the train home) I'll download the retrosheet data he used and make a better chart.
In the 1970s, you typically had 20-30 pitchers who started at least 36 games (a true four man rotation would start 40). Given every pitcher on a team would start that many games in a true four man rotation, that means a whopping 8 to 10 teams may have had four man rotations; the rest likely had five man rotations. If you up the number of games started to 40 (again, the true four man rotation), you are in the teens or less in the 1970s - so maybe a handful of teams had a true four man rotation.
Many teams had modified 4 man - 4 day rest rotations during this period, where a team had four starters and used a fifth periodically when needed to keep them on four day rest. This often led to #1 pitchers having 36-37 starts, so 3 or 4 more starts than the true 5 man rotation (33). To some extent this fell out of style finally in the late 1980s and 1990s, where Bobby Cox and Joe Torre used true 5 man rotations with great success; while it seems like you might as well keep your aces on 5 days rest, it may be that an occasional sixth day is helpful for their rest, and/or keeping the 5th man on a regular rotation is better than having him only pitch every 10-20 days. (It also may be largely irrelevant, who knows.)
Either way, anyone arguing that "most teams had four man rotations in the 1970s" is misremembering, probably thanks to folks in the 1970s and 1980s misremembering themselves.