His problem has nothing to do with you. It isn't your fault atoll. I know how frustrating it can be. The best thing for him to do is to see an urologist that specializes in sexual disorder. You can ask your family doctor for a recommendation.
It's porn induced ED?, or other conditions such as diabetes, cardiovascular disease.
But since its with the stimulation as in penetration does not provide much stimulation (soft vagina walls) as compared to the hand (firm grip) or mouth (u have control on pressure) so it could be more related to nerves .. so onset of diabetes (which damagesblood vessels and nerves), or neural problems could be at the root of it. Please check with doctor...