Actually, yes, it would be your point if it hits your opponent before touching the ground - but only if the ball has not passed the end of the table already.
if, after he or she has made a service or a return, the ball touches
anything other than the net assembly before being struck by an
and Rule 2.10.01.04:
if the ball passes over his or her court or beyond his or her end line
without touching his or her court, after being struck by an opponent;
This means after the ball passes beyond the end line (end of the table) - the point is over and whoever hit the ball beyond the end line loses the point.
Obviously if it were legal to win a point just by hitting your opponent with the ball, some crazy strategies would come into play - for the worse of the game.