I'm sorry you were blindsided. That's a terrible way to practice medicine.
A swab has a very low chance of being a false positive, but I've seen it before. You can always get it confirmed with a type specific IgG blood test. That would be fair, since you haven't really had obvious symptoms.
The numbers in tests that you keep reading about are the ones from the blood tests. The cut off for a positive, officially, is 1.10, but anything under a 3.5 needs to be confirmed because it could be a false positive.
You should ask your doctor for the blood test if you aren't fully sure about the culture results, or if you need it for peace of mind - that's fair, too. You need to do what it takes to accept any diagnosis. If you're in the US, you can also go to STDCheck.com or LetsGetChecked.com and get it on your own. They are the same tests, slightly different prices, so just use whichever is more convenient/cheaper (whichever is more important to you).
It is possible to have herpes and not get blindingly painful sores. Having hsv1 already can help keep hsv2 mild, so that could explain things, too. But absolutely do what you need to do to prove it/disprove it. That's important.
If you do have hsv2, I can help walk you through that. I just explained some things to a woman who's had herpes for a long time, but is newly dating - some of it may help you - https://
www.medhelp.org/posts/Herpes/Oral-and-beyond-with-HSV2/show/3066069
Let me know what you need. That's what we're here for.