In Pokemon Mystery Dungeon Explorers of Sky why does my fourth party Pokemon stop following me?

The setting his for tactics is on "Follow Me" and when I walk right next to him, or use a roll call orb, he still won't follow me (just stands there.) Is this some sort of glitch? It's really annoying, because I have to send him back to the guild all the time.