Basically, the golf ball that is best for you depends on what you need out of a golf ball, and that depends on the rest of your game including the course you play on. Without going into a doctoral dissertation on the subject, the two most important factors, IMO, are compression and spin characteristics.
Compression is the "squishiness" of the ball. A low-compression ball is softer, requiring less force to compress. This will keep the ball on the clubface longer, imparting more "impulse" from the clubhead to the ball, and it will also induce more spin as the clubhead has more time in contact with the ball to get it spinning. They also feel softer when hit, which is preferable to many players. But, they will waste some energy of a particularly powerful swing because the softer material won't completely return to its normal roundness until after it's left the clubhead (like bouncing on a trampoline but not straightening your legs fully until you're already back in the air). High-compression balls, almost the exact opposite; less time on the clubhead, less spin, harder feel, faster rebound.
There's considerable debate about the true importance of compression on raw distance; there's not nearly as much in it, all other things being equal, in matching compression rating to swing speed as there is in matching your shaft flex. Most experts say that for the average player it's a question of feel; if you want your swing to feel like there was nothing in its way, go softer. If you want the ball to give you more feedback on ball striking, go harder.
Spin characteristics are a function of practically every aspect of the ball's design, including, as mentioned, compression. A low-compression ball will get more contact with higher-lofted clubs that are swung at slower speeds than the driver, increasing their backspin, which increases their height and reduces their rollout, so they'll drop down onto the fairway or green and stay where you put them. Most harder balls will spin less, and so they won't climb as much or drop as sharply, so they'll roll longer. Great for distance, bad for control.
Higher-quality (and more expensive) golf balls are made with multiple core layers to help give the best of both worlds. Outer layers are softer and compress more easily, inner layers are harder. The net effect is a plateau in the amount the ball can be compressed as swing speeds increase. Off the driver, total compression of the ball is limited by the inner core, so the ball rebounds faster, wasting less energy and spinning less. However, the outer layer will still compress easily enough against the clubface of a higher-lofted club, giving those clubs the additional spin they need for the shot shape golfers want when approaching the green. Mid-grade balls often have three-layer construction (two core layers and outer cover) while the top-grade balls commonly have four layers. These balls also often trade the more durable and cheaper but slicker ionomer plastic covers of two-piece balls for softer covers like urethane, which grabs both the clubhead at launch and the ground on landing to magnify spin at the cost of durability (hit a Pro V1 onto the cart path, or even out of a bunker, and odds are you'll need to retire it after that hole due to the damage the grit has caused to the cover layer).
Dimple pattern can also affect spin, though in a more minor way. The exact number, size, depth and arrangement of dimples on the ball all affect the "boundary layer" of air passing around the ball in flight. Deeper dimples create a thicker boundary layer, reducing friction (the maximum depth of dimples is regulated by the Rules of Golf for this reason). Manufacturers have experimented with varying dimple patterns, with dimples of varying sizes, to create different flight characteristics, some more forgiving, others more consistent.
