Because Sundays.
Traditionally in England, Test Matches started on a Thursday. This was so that Day 3 of the match, often the pivotal day, could be on a Saturday to maximise attendance when workers had the day off. Given that Sunday is the Christian holy day, sports would not be played on a Sunday - therefore a rest day. Other countries followed along with the tradition, to some extent because all the Test playing countries are heavily English influenced one way or another.
The reasons for the change are the normal ones: the increasing secularisation of society, meaning that Sunday is less special than it was, and money: a second day of big takings is good for the organisers. The first Test match in England which included Sunday play was the Trent Bridge Test in 1981, and rest days drifted away after that; the last rest days were on the Sunday of the Wimbledon tennis final but even that went in 1997.
Note that it is still not mandatory for a Test match to be played on 5 consecutive days - organisers could still have a rest day if they wish for one, although I don't believe one has been used since Dhaka in 2008.
For more details, see The rest is history at Cricinfo.