My son does the same thing. He moves to the middle of my tummy when people aside from his dad and i touch my belly. I use a flashlight to get him to move since the light is one of the few things he responds to.
Hm. that stinks. Now, my baby started moving around and would soon change positions so maybe yours will soon. You can always do the orange juice trick where you drink it and the sugar causes them to move and maybe they'd move back over to the side. good luck