So a few things -
The IgM is terribly unreliable, and shouldn't have been done on you in the first place. The CDC and all herpes experts recommend not using this test. Ignore it entirely. It's as likely to give you a false result as it is an accurate one.
Did anyone culture your sores? The best way to diagnose herpes is with a type specific culture, or preferably, a PCR swab, which is more sensitive.
Have you been taking the antiviral since you first got symptoms? If you have, it could be interfering with your testing. The best thing you can do is to stop the antiviral, wait about 12 weeks and test again. The valcyclovir could have sort of taken over for your immune system, preventing antibodies from developing.
If you get another sore, get it cultured. Insist on a type specific culture or PCR swab.
When you say you were exposed orally - did that person give you oral sex? If they didn't, you can't get genital hsv1 from them.
Herpes doesn't bleed, so I can't say that what you had was herpes, or if you have it. Herpes doesn't usually cause swollen testicles, either. Gonorrhea, chlamydia and syphilis can, though. Have your testicles gone down in size?
A primary outbreak of herpes - where you have neither hsv1 or 2 - wouldn't likely clear up in 2-3 days, even with antivirals.
I don't want to give you false hope - you could have herpes. I don't know. I'm saying that the symptoms don't sound like it. You could have herpes and something else, as well. What other STDs were you tested for?