So do you know if your doctor did a regular culture or a PCR swab? I ask because the PCR swab is more sensitive and can pick up the herpes virus more easily than a regular culture. A false positive for either is far less likely than a false negative, but I've seen it happen on both.
However, you have a (technically) positive IgG test, though it's low.
I wonder if you had a pimple or infected hair follicle or what have you that triggered viral shedding, and that's what the swab picked up on.
If you want to get a Western Blot, then do that. If you need to do that to accept it or disprove it, that's okay. There's no right or wrong answer here. If your IgG was much higher, I'd tell you it's a waste of time.
I don't want to give you false hope - it's more likely than not that you have it, but you need to do what you need to do. If you have it, you could have had it for years, and just not known it. Do you also have hsv1? That is known to help keep hsv2 milder.
If you do have it, unless you have all of your former partners tested, you may not ever know. It doesn't sound like you could have gotten it from the threesome. I'm assuming she didn't have an outbreak at the time, and even if he did oral on her and then went right to you and gave you oral, without an outbreak, it's not a likely effective mode of transmission. Hands and mutual masturbation don't transmit STDs.
I'm sorry you're going through this. Let me know what you decide to do.