It is rare for herpes to go from the genitals to the mouth. This isn't something I'd worry about at all. Definitely get your symptoms checked, though. It sounds as if you have something going on, but I'd be surprised if it's herpes.
Additionally even if you have had a blood test igg and have been negative you can only be roughly 70% sure of that as the igg test misses 30% of hsv1 infections
While not impossible, the little bit of contact you describe would typically not be enough to transmit herpes. You may want to see a doctor and have your sores swabbed and pcr tested or cultured.
You may not realize this but at least 50% of adults are infected with hsv1 and most do not know it because they rarely if ever get outbreaks. So, you may be infected and not know it, and because of this 50% prevalence, most uninfected people have had numerous sexual encounters with someone that is hsv1 positive. I would also add that the 50% figure is a conservative estimate. Due to cumulative exposure, it's likely that in the demographic of people over 40 years old, at least 65% of them are infected with hsv1.