Oh thats valid, you should try some medications or try to communicate with some professional about these problems but you should not worry about this. You can also try pills to cure impotence but you should be prescribed by your doctors or medicare.
How is your emotional state? Being stressed, anxious, depressed all can impact this. Do you get nervous sometimes? Do you masturbate often? Do you need to spice up your sex life where you kind of drift mentally during intercourse so you lose the erection? You do know you can still pleasure here just as well than intercourse, right? At those times, switch to those methods to take the pressure off of you. Do you take any medication?