There are two ways to deliver a pitch from the mound:
- The windup
- The stretch
Most pitchers will pitch out of the windup when there are no runners on base. It usually helps with the pitchers rhythm. They are able to use a high leg kick and deliver a pitch.
Some relievers will pitch out of the stretch whether or not there is a runner on base. This is because they often come in with runners on base and their rhythm is to pitch out of the stretch.
As for the stretch, it is used to deter the other teams runners from stealing a base. If the pitcher was in the windup, as soon as he started his windup the runners can steal. If the pitchers stops his windup that would be a balk with runners on base. He can still pick from the windup by stepping off with his foot that is on the rubber. (RHP would step off with his right foot and vice versa for a LHP.)
In the stretch pitchers will slide step to deliver the ball quickly to home. This is when they step to home without bringing their lead leg up very far from the ground. Many pitchers can be around 1.2 seconds to home plate with the slide step, giving the catcher a good chance to throw a stealing runner out.
Basically the difference is in the time to the plate, which is why all pitchers pitch out of the stretch with runners on 1st and or 2nd.
One side note is that with a runner on third some pitchers may choose to pitch out of the windup because in order to get the runner out stealing home, they simply have to get the ball to the catcher. This is a per pitcher and situation type of thing and while not common, it is certainly not rare.
Usually there is a slight to mild drop off in the effectiveness of pitchers pitches when they are in the stretch, especially using a slide step.
Here is a brief Wiki article that may help understanding