On your first point: Fangraphs certainly agrees with you that old metrics seem to dominate. There are no hard and fast rules about which stats are presentable, though; it's solely due to who is hearing the case - and there is no doubt that stats (both old and new) have made a huge difference (see Baseball Prospectus'** article on Arbitration, for example).
A list of what evidence you may present:
The following evidence is admissable:
- The quality of the player's contribution to his club during the past season (including but not limited to his overall performance, special qualities of leadership and public appeal).
- The length and consistency of his career contribution.
- The record of the player's past compensation.
- Comparative baseball salaries (the arbitration panel is provided with a table of confidential baseball salaries for all players broken down by years of service).
- The existence of any physical or mental defects on the part of the player.
- The recent performance of the club, including but not limited to his league standing and attendance.
Second, as to why 'younger' players get less: Players and their agents are only allowed to compare themselves to other players in their service year or players with one year more service time. As much of the success of the arbitration is due to finding other players with the same stats, this significantly limits the argument that players may have compared to the market-inflated salaries. The arbitrator basically can only compare the player to other players in arbitration, until they are in their last year of arbitration; so Mike Trout cannot be compared to Albert Pujols, for example, despite the fact that any idiot would sign Trout for more than Pujols at this point. This can be argued to be waived in the case of 'special' players (see: Trout), but typically it does not yield nearly the 'market' salary nonetheless.
From BP:
In other words, Super Twos can be compared to Super Twos and three-year players, but not four-year players. Threes can be compared to other threes and fours, but not fives. The exception is that fives can compare themselves to anyone, just like free agents.
Further, this likely leads to aggressive signing of young superstar players to pre-arb contracts, like Mike Trout ($24M/year for 6 years); this is because owners want to avoid setting overly large arbitration precedents and schedule when money will be delivered such that it doesn't inflate other salaries. Trout will earn $24MM a year, but it is back loaded: 5MM in (service) year 3, 15MM in year 4, 19MM in year 5, then 33MM in the last 3 years (service years 6,7,8). This helps avoid large arbitration comps for other players somewhat.
They certainly do not reflect market salaries would be, even in the final year of arbitration (though they do get closer). The market salary is effectively the highest amount any one GM thinks a player is worth, and bakes into it not only the past performance of the player, but the likely future performance as well - some years of control over the player, expected growth, and such. Arbitration salaries do not consider this 'future performance', except as it may be realized in other players' comparative salaries - which, remember, are also arbitration derived until the final year. In addition, the fact that no teams are bidding 'against' the team and player means that the arbitrators' decision does not include the additional money a player likely would receive from other owners bidding on his talents.
Finally, your third point is correct: the arbitrator must choose one of the two options. In theory, the arbitrator would compare the midpoint of the two proposed salaries to the determined 'correct' salary and choose the one closest, but as BP notes in the earlier-linked article, that is not set in stone, and they may make their decision based on whatever they choose - which probably depresses extreme salaries some.
As far as your questions at the end: the system works largely because it's a compromise between the players (who would like earlier free agency) and the owners (who would prefer six years of complete control). It probably benefits the clubs slightly, particularly as it's easy to identify certain players who are likely to receive 'bad' awards (closers in particular seem to get overly high awards - hence trades like the Addison Reed trade in the 2013/14 offseason) and cut them without cost; but it certainly is helpful to players as well compared to what it would be otherwise, and the owners are unlikely to agree to completely open free agency after 3 or 4 years.
Why did it exist? Read the Hardball Times article linked in the other answer; it explains how it came to pass in the 1970s. Basically, arbitration existed before free agency, after the 1974 strike/lockout; it was the compromise that allowed salaries to have at least some tie to performance (before that, owners set the salary for a player, and a player could play for that salary or not play baseball at all due to the reserve clause).
The reason it is Final Offer Arbitration is because that is common in labor negotiations. See the wiki on Arbitration:
It is designed to encourage the parties to moderate their initial positions so as to make it more likely they receive a favourable decision.
Without it, both sides likely would make outraegous offers. Still happens sometimes (see: Eric Gagne) but in general clubs and players are within a million or so in most cases.
** The BP article I linked is from 2012. In 2013, a major change was made that removed the 'free agent' arbitration option, so that no longer applies. Clubs now may tender a qualifying contract to FAs (a bit north of $14MM in 2014), which retains their rights to draft pick compensation, but there is no risk of an arbitrator awarding the player more.
