tl;dr:
- Track distance, tightness of curves, and banked curves
- Event distances
- Different implement weights
- Lack of wind
- Differences in runways (length and responsiveness)
For the track events, you've nailed it: the 200m standard indoor track means races of the same distance (e.g. 800m) can be notably different from races on a standard 400m outdoor track. In the sprints, the tight banked turns can significantly change the athlete's approach to the race, and some events (e.g. 60m) are only held indoors.
For the field events, the reasoning for different records is more tricky. In the throws, e.g. shot put, the implements are different (size and weight). For the jumps, though, the differences are entirely environmental: lack of wind (either supporting or impeding) and different runway surfaces, mostly. In these events, competing indoors can actually be easier (although long jump and triple jump may be limited in the length of their run-up - I'm not sure if that would help or hurt an athlete).