@Kevin summarised it nicely, I just can add something from my own experience. The lowest temperature I ran for training was -10C. For temperatures around 0C, I use long, stretchy pants, underpants and simple running t-shirt, coupled with wind-breaker jacket and pants, which are very thin and impregnable. These are enough, since when you run, you generate a lot of heat.
For lower temperatures, I add gloves, and a hat and technical shirt with long-sleeves. If you run more than 1 hour under -10C, you start losing heat in the legs, but I never searched for solutions, because my training runs rarely exceed 1 hour.
This winter we had temperatures as low as -25C. Then definitely balaclava is needed, since if you try to run and breath through the nose, you soon start to choke (because the nose cannot warm the air fast enough), and breathing through the mouth may invite cold. On this question though I am not sure. Under cold weather I tried to minimise breathing through mouth just in case.
Concerning footwear, if there is no ice or snow, usual footwear suffices. I ran under -10C with simple running socks and usual shoes with no problems. If there is ice or snow, then special shoes should be used, or you can put screws in your old running shoes.
Finally another advice which I found useful. You must feel cold when you start to run. If you do not feel cold, you are overdressed. This is true, although it is annoying when I wait for my Garmin to catch GPS signal.