Based on a quick test I just did, it appears that the program state effectively changes from paused to stopped when moving from Local to Remote mode. Therefore, when the start signal is sent, the robot restarts the entire program because it is restarting from a stopped state.
Are you not able to pause the program with our pause input while it is already running in remote mode?