There are several things, including a major one that you mentioned before which is the break of the curveball/slider.
Generally speaking, hitters can handle pitches on the inside portion of the plate better than the outside. Also, hitters can track and see a ball that is moving towards them better than away. So for a lefty on lefty (or righty on righty) a 4 seam fastball tends to move slightly away (advantage pitcher) and both curveballs and sliders will break much further away (advantage pitcher).
You'll notice that many of today's best left handed pitchers have developed excellent Change Ups (Johan Santana, Cole Hamels, and popularized by Tom Glavine). Since there are far more right handed batters than left, they needed a pitch that would break away instead of towards the right handed hitters.
For pitchers who use primarily a fastball change-up it still makes a difference but it is not as exaggerated. The 4 seem fastball should still be a tougher pitch for the left on left matchup.
The last thing is: everyone, both left and right handed hitters, see far fewer left handed pitchers throughout their careers. Just being unfamiliar and unique is an advantage to the lefty pitcher.