According to Wikipedia's article on the record's progression:
The drop in world record times in 2008/2009 coincided with the introduction of polyurethane suits from Speedo in 2008 and by Arena, Adidas, and Italian swimming suit manufacturer Jaked in 2009. FINA's ban on non-textile suits came into effect in January 2010.
Essentially, brand-new suit technology was introduced that allowed swimmers to go faster, and while that technology has now been outlawed, the records remain as they were legal at the time they were set.
I can't find any information about what caused the massive jump in 1980, when the record was broken 6 times in as many months, including thrice in one day (April 10th). However, the men's 50m freestyle was still a fairly new discipline at that time - FINA only began keeping records for it in 1976 - so it's plausible that 1980 simply marked the point where swimmers figured out the optimal techniques to let them go faster, or started taking the emerging discipline more seriously and pushing the performance envelope harder.
The following chart (larger view here) compares the world record progressions for various different swimming disciplines, and several of them show similar drastic improvements within the first few years of FINA's record-keeping. The world record for the mens' 100m breaststroke fell by four seconds in less than four months after FINA started keeping track!