It seems to me that every figure skater (in championship levels) specializes in just one of pair or single skating. Is that true? If yes, why? Is there such a great difference between them that no one can master both?
Indeed skaters focus on one at a time. And there notable differences between single skating and pair skating that require a lot of time to practice. Often skaters switch from one discipline to another. Usually singles switch to either pair skating (if they cannot nail all triples for instance and in pairs they can focus on one or two) or dance (if they cannot jump or have very good skating skills). Of course these reasons are a bit stereotypical. Sometimes it is only about being able to position yourself better competitively. Famously Jane Torvill switched from singles to pairs to dance.
I think it is possible to do well in both. There are two problems though: First, training time is limited by injury, so skaters who practice only one discipline have more training in that discipline than skaters who do two disciplines. Second, pairs skating develops upper body muscle (especially for men, but also see Meagan Duhamel). Lots of upper body muscle makes the skater top heavy, which means that jumps must be more precise. It is also dead weight during a jump. This is a disadvantage in freeskating.