I want to make a display using arduino that shows "who's on first" etc at yankee stadium. So, I'm looking for a real-time feed with that info. What are my options?
Not sure you're going to find a "free" solution that is legitimately usable in the fashion you require and official/guaranteed to persist.
That said, MLB does publish an API, and has for years - it's just not something they guarantee. It's their GamedayAPI. I'm not sure how frequently it's updated, but I just peeked at the folder for 9/24/2015 ARIvsLAN and found that they were at least on the current inning compared to the live site, so it's obviously pretty up to date.
The most common commercial product is from Baseball Info Solutions, which provides data to sites like FanGraphs.