Yes. You have genital herpes, which actually means you are infectious from the entire area, including the anus.
Herpes infects nerve groups, and genital herpes infects the sacral ganglia, which means it affects the area that boxer shorts cover. Herpes is infectious, or contagious, from mucus membranes - the vagina, vulva, anus, penis, etc - the moist, thinner skin areas.
Sometimes, the virus is active on the skin without symptoms. This is called asymptomatic (without symptoms) viral shedding. How often this happens depends on what type of herpes you have genitally. You can have type 1 or type 2 genitally (the number indicates the strain of the virus, not the location).
Shedding rates: (and you can find all this in the herpes handbook - https://westoverheights.com/herpes/the-updated-herpes-handbook/)
HSV 2 genital 15-30% (55-110) of days evaluated per year
HSV 1 genital 3-5% (10-18) of days evaluated per year
HSV 1 oral 25% (91) of days evaluated per year
HSV 2 oral 1% (~4) of days evaluated per year
Do you know what type you have?
Also, anal sex can make it more likely to transmit because it often causes tearing in the tissues. Make sure to use condoms (always, with anal sex, especially if you don't know your partner's STD status), and lots and lots of lube.
Read the herpes handbook I linked above. It's free and written by one of the world's leading experts on herpes. You don't have to send for it or anything, just read it on your computer or phone. :)