First, players on the field may switch positions while the ball is dead during an inning. A pitcher could switch with an outfielder and then switch back for the next batter. The only difference between switching between a pitcher and a position player is that a pitcher must face a batter (unless he becomes injured during the at bat).
In your scenario, the outfielder could come in to pitch and the pitcher could go to the outfield. The next batter, or next inning they could switch back. There is no required substitution in this situation. It often happens late in games with limited bench players renaming. A left handed pitcher may be brought in to face one batter, and instead of burning an extra pitcher in the process, the current pitcher is placed in the outfield for a batter.
This scenario happened to the Atlanta Braves. You can see in the box score, Chris Resop (a pitcher) comes in to pitch, then moves to left field, and then back to pitcher.
As to your question about "swing men", the basic answer is no. There are many position players with good arms. There are many problems with this though. First of all it takes a lot of time and practice to even make it to the big leagues. This is spent on hitting or pitching, and not both. A position player may be able to throw as hard or harder than some pitchers, but it does not mean he can throw strikes, locate pitches, hold runners, etc. Some players that struggle hitting in the big leagues will go back to the minor leagues and convert to a pitcher. This again takes time to perfect. The other issue with "swing men" is that there is an organization of 100+ players, and to have a position player that is better than those pitchers is pretty rare.
The "swing players" you refer to would be called a two way player in baseball. In college baseball it is much more common. The players may also DH for them selves as they pitch (a quirky NCAA rule). The award for that specialty player is called the John Olerud Award.
In most sports, the lower (or younger) levels are dominated on mostly by talent. Once the highest levels are reached, most players are very talented and the players that work the hardest and train the hardest will be the most successful. If is hard to out work and out train two different groups of players at the same time.
One last comment, position players will come in to pitch in blow out games some times. This is usually just to save the innings and pitch count number for the relief pitchers in the bullpen over the course of the season. These guys are not seen as two way players, even if they pitched in college or high school. They are simply position players pitching in blow outs.
A position player getting the win in extra innings