So first - do you know what type of genital herpes you have? You can have hsv1 or hsv2 genitally. If you don't know, you can get a blood test to find out. (It won't make a difference for oral sex, which I'll explain in a minute, but it can for genital or anal sex.)
So herpes easily goes from the mouth to the genitals (which is why you could have genital herpes type 1 - the number indicates the strain of the virus, not the location), but rarely goes from the genitals to the mouth. It's possible, but not likely.
If you have genital hsv1, it rarely sheds, rarely recurs. This means you are not at all likely to ever transmit it to someone, either to someone's mouth or genitals via oral or genital or anal sex. Also, about half of the adult population already has hsv1 (and 90% of them will never get symptoms, so they don't know they have it), and they can't get what they already have.
If you have genital hsv2, it really doesn't like the oral area. While genital hsv2 does recur and shed more frequently than genital hsv1, oral hsv2 is considered rare. If someone does get oral hsv2, that sheds and recurs less frequently than genital hsv1.
If you have ghsv2, then suppressive therapy (daily valtrex or acyclovir) may be something you consider.
Ask your doctor for a type specific IgG blood test to find out what type you have. How were you diagnosed with herpes? If your results show only hsv1, then you have that genitally (assuming you were diagnosed properly). If you have both types, type 1 is oral, type 2 is genital.
Your partner should also test. He could have either/both types and not know it.
You can read more on all of this in the Herpes Handbook. It's written by Terri Warren, who is one of the leading experts on this. It's also free. :) https://westoverheights.com/herpes/the-updated-herpes-handbook/