If someone gets symptoms from gonorrhea, especially a man, they will usually appear within days of the infection. It would be very, very, very unlikely that he'd have it for 4 years and the both of you show no symptoms.
How was he tested? What were his symptoms?
What is he saying about how he got it?
Gonorrhea is only transmitted by oral, vaginal or anal sex. It is not transmitted by injecting drugs. HIV and hep B and C are, though.
Have you been tested? You need to be.