Okay, so get comfy because it's going to get a bit complicated.
First, let me say that I see no reason to think he was unfaithful if this is the only reason to think he was. I'll explain why, but let me put that out there so you can read the rest calmly.
You have genital hsv1. That's been established by your PCR swab. (And congrats to your doctor for doing the best available test there is on sores. Really. Not many docs know to do that.)
You got this genitally from oral sex, almost certainly. It is almost never transmitted genitally to genitally.
Now, for the blood testing. Our blood tests suck. That's about as nicely as I can put it here. Our hsv2 tests are slightly better than the hsv1 tests, and our hsv1 tests really, really suck.
The hsv1 IgG test (and yes, they are type specific) miss a full 30% of infections. Your partner could have a well-established infection and the blood test could easily miss it. He could have gotten it as a child and the blood test might not pick it up.
The other lousy part of this - as far as figuring this out - is that about 90% of people with oral hsv1 never get a single symptom, so unless they test and get an accurate test result, they don't even know they have it.
They are still infectious and can transmit it, even without symptoms, but they won't know they have it until they transmit it.
Did you get an hsv1 IgG blood test? Was it positive? It's also possible that you've had it for a long time, and just didn't know it, too. There are many people who get an outbreak, don't know it's herpes, and think it's something else, and then don't get another one for years and years. Some, especially with genital hsv1, never get another one.
If you and your partner decide to get back together, and if it's an otherwise good relationship, I hope you do, there will likely be little impact on your relationship from this.
Since it's ghsv1, you are unlikely to ever transmit this. If you got it from him, he has antibodies that give him strong protection against getting it in another location.
If you got it before him, that means you've had it for more than 2 years. We know that ghsv1 sheds about 4 days a year after having it for 2 years outside of outbreaks. Again, this means you are unlikely to ever transmit it, and even if he doesn't have hsv1, he's not at any real risk of getting it from you.
You can read more about shedding in the Herpes Handbook at https://westoverheights.com/herpes/the-updated-herpes-handbook/. It's free and you can read it on any device. It's geared mostly to hsv2, but the shedding info is good for anyone.
I hope I didn't confuse you too much, but let me know what questions you have.