The major reason for using white ball in ODI and T20 is that many of those matches were happening as Day & Night matches and it was very difficult to spot the red ball under floodlights. Cricket balls were traditionally red and test matches still use that(Compared to red ball, white balls deteriorates more quickly and hence red balls are more suitable for test matches).
From Cricket ball Wiki:
Cricket balls are traditionally dyed red, and red balls are used in
Test cricket and First-class cricket. White balls were introduced when
one-day matches began being played at night under floodlights, as they
are more visible at night. Professional one-day matches are now played
with white balls, even when they are not played at night.
This was one of the reasons why the player's dress color was changed from white to colors so that the ball can be spotted very easily in ODIs and T20s.
There was also an attempt to experiment with some other colors as well as mentioned in the wiki:
Other colours have occasionally been experimented with, such as yellow
and orange for improved night visibility, but the colouring process
has so far rendered such balls unsuitable for professional play
because they wear differently from standard balls. A pink ball was
used for the first time in an international match in July 2009 when
the England Woman's team defeated Australia at Wormsley.
As per the Wiki page:
The white ball has been found to swing a lot more during the first
half of the innings than the red ball.
So you are correct, white balls swings more than the red balls during the first half of innings.