Along with the idea of selective mutism, I would also look up Sensory Integration Disorder as this can also be associated with selective mutism.
My son has an autistic spectrum disorder and sensory integration disorder and in some environmental situations he is unable to multi-function from a sensory point of view. So, for example, he may not be able to speak if he is looking at something or doing/making something at the same time. If this is also going on then it will rank up the anxiety and overstimulation of your child. An OT can assess for this and give therapy. Try to get an OT with experience in this area.
Good Luck
Your son is not "refusing" to talk; he is "unable" to talk in perceived, unsafe environments. I strongly suspect your son is suffering from a severe anxiety disorder called "selective mutism". The best site on the internet is "selectivemutism.org" which will give you reams of information re this disorder. It is very treatable but will take time (and I mean months or years) for your son to learn how to manage his fears. The first step is to talk to your family doctor about this disorder - if he/she has no knowledge, then ask for a referral to a specialist who has experience in anxiety disorders as a child psychiatrist or child psychologist or child neurologists.
By the way, this year it might be best to retain your son one more year before starting kindergarten. Your son will have "enough on his plate" while he learns how to "manage his fears". By the way, your son will not outgrow this anxiety nor will it go away so please see your doctor as soon as possible - the earlier the diagnosis, the better the prognosis. I have a lot of experience with this disorder so if you have any questions, please contact me by e-mail - done by clicking on my blue name at the top of this post. I wish you the best ....