I hope you check out scouts, too. Swimming lessons are great, and fun, but he's very likely to run into boys with very similar personalities (whose parents are also trying to help them find friends) in Cub Scouts.
Best wishes.
Thanks for the comment, it really helped. My son likes similar activities, I just enrolled him to a holiday swimming class. I hope it works well for him.
Thanks anyway.
My youngest son as like him. He's now 13, and has lots of guy friends - they don't play competitive sports with him, but they swim, hike, bike around, etc.
Have you tried scouts? Your son would probably find a lot of like minded boys there.