We are a Minor's Division team (9 & 10 year olds).
I wanted to rotate between my two pitchers by inning. I pitched my first kid for an inning, switched kids and pitched the second one for an inning. On the third inning I tried to have my first kid pitch again, but the other team manager said that was against the rules.
The rule states (regulation VI, part B): A pitcher once removed from the mound cannot return as a pitcher.
Did I 'remove the pitcher from the mound' when I tried to rotate them?
Yes.
Removing a pitcher from the mound is any action where the pitcher was the pitcher, and is now no longer the pitcher.
It's worth noting that this is a place (among many), where Little League departs from MLB rules. In MLB rules you can take a pitcher off the mound, make him a field player, and then bring him back to the mound. However, because Little League has free substition, this could (and would) cause managers to abuse pitching changes in a way that isn't keeping with the spirit of the game.
So yes, once a kid has pitched in a little league game, they can't pitch again in that same game, even if they stay out.
