This is quite common and its almost as if children only want to give their parents individual time, I wouldnt be concerned by it, get his Dad to play games, old fashioned board games are fun, and they have some for younger children, read him stories and generally have fun, it has to come from the adult get Dad to make him laugh,,,that'll do it.