As per FIFA's "Equipment Regulations", Section 22, only the goalkeeper is permitted to wear a cap.
22.1 All goalkeepers may, irrespective of the prevailing conditions, wear a
goalkeeper cap of any Colour. The goalkeeper cap must be produced by the
Manufacturer of a Playing Equipment item. The goalkeepers of the same team
may wear different goalkeeper caps.
The rule regarding caps was brought in to afford goalkeepers better visibility in sunny conditions, with the cap shading the eyes from the glare of the sun. The rule includes the line "irrespective of the prevailing conditions" so that goalkeepers can also where a cap even if there is no sun. Some goalkeepers will wear a cap in the rain so that their vision is not impaired by rainwater in their eyes.
Regarding headbands (and wristbands), section 24 states:
24.2 The Players may wear a headband or wristbands. Neither the name, or
any abbreviation thereof, nor the number of a Player may be displayed on a
headband or wristband.
Headbands and wristbands reduce the problem of sweat and rain and remain a personal choice as to whether the player wears them.