You've got it right. Payoff pitch basically means that there's going to be a result. It does preclude a foul ball, which can extend the at bat, but foul balls can extend other at bats that are not in "pay-off pitch" position, for example an 0-2, 1-2 or 2-2 count.
The term refers more to the situation (3-2) than the pitch itself. 3-2 is a "payoff" situation because there is no other count to change to. At this point, if the pitch is not fouled it will definitely result in an out or another runner.
It may be used more often when there are two outs, bases loaded, because when the at bat ends it will either end the inning or score at least one run, but different sources are not consistent in this.
It's not an official baseball term, but the Merriam Webster dictionary defines "payoff" as:
3 : the climax of an incident or enterprise; specifically : the denouement of a narrative
4 : a decisive fact or factor resolving a situation or bringing about a definitive conclusion
So both of those fit with the idea that the "payoff pitch" will end the at bat (even if it won't necessarily end the at bat).
Sources
Sporting charts - http://www.sportingcharts.com/dictionary/mlb/payoff-pitch.aspx
Wikipedia - http://en.wikipedia.org/wiki/Glossary_of_baseball_(P)
Merriam-Webster - http://www.merriam-webster.com/dictionary/payoff