The rookie status varies between different sports.
In the MLB:
Determining rookie status:
A player shall be considered a rookie unless, during a previous season or seasons, he has (a) exceeded 130
at-bats or 50 innings pitched in the Major Leagues; or (b) accumulated
more than 45 days on the active roster of a Major League club or clubs
during the period of 25-player limit (excluding time in the military
service and time on the disabled list).
So in Baseball it is possible to be considered a rookie in your 2nd season or even later, if injuries or other things prevent you from being on the active roster long enough.
The NFL uses the term just as you said, but has no definition in the rules. Here is a small statement about it.
What is Rookie?
This is a term used in most every sport at every level of competition,
NFL football included, to refer to a first year player. In the
National Football League, a first year player may be acquired through
the NFL Draft or as a free-agent if they went undrafted. (...) A rookie is not necessarily young,
though this is usually the case. If a player has never played NFL
football before, they are considered a rookie, despite their age. (...)
The NBA does the same:
In the National Basketball Association, a rookie is any player who has
never played a game in the NBA until that year.
The MLS is different again and a rookie is someone with no prior professional experience (Wiki).
MLS, as opposed to other American sports leagues like Major League
Baseball, only designates a player as a rookie if the player has had
no prior professional experience. So, for example, although Taylor
Twellman finished second in MVP voting in 2002, he was ineligible for
the Rookie of the Year Award because he had played on the reserve team
of 1860 Munich.
(I don't know why he was ineligible, because 1860 Munich II played in the Regionalliga at this time, which is not a professional league (3rd division at this time) and he never played for the Bundesliga team of 1860 Munich)
In the NHL the rules are pretty strict, because they take any major professional league into account and not just Ice Hockey.
To be eligible for the award, a player cannot have played any more
than 25 regular season games previously in any single season, nor have
played in more than six regular season games in each of two separate
preceding seasons in any major professional league.