Hi there. Gosh, I think you always have to air on the side of safety. A three year old isn't exposed to things to make it up usually. It has to come from some where for them to know about it. I'd pursue it with your doctor and add on a psychologist to investigate being careful that no one is 'feeding' thoughts to him. goodluck and I'm sorry you have to deal with this.
You haven't said anything further about this. Did you take your son to a doctor? Did you ask him WHO has touched him. Where did he say he was touched? What has been done?
I would talk to his doctor