The IgM is worthless. My own were negative when it should have been positive, and positive when it should have been negative.
You gave oral sex. Oral hsv2 is very rare, and even if he had ghsv1, that rarely transmits, as it sheds and recurs infrequently. If your only exposure was giving oral sex, I see no reason to worry at all. I understand your concern with the sore throat, but that could have been so many things, and herpes wouldn't be near the top of the list if you had no sores or blisters.
https://www.statnews.com/2017/01/26/flawed-herpes-testing-leads-to-false-positives/
https://www.kevinmd.com/blog/2013/07/order-herpes-igm-blood-test.html
https://drjengunter.com/2013/06/17/igm-blood-test-for-herpes-just-say-no/
https://westoverheights.com/forum/question/diagnosed-soley-on-igm/
Act as if you never had the IgM, and know that your risk was very, very low.