Nothing you describe sounds like herpes. While a recent herpes infection can cause the symptoms you describe - swollen lymph nodes, lower back pain, etc - these symptoms would not be present without obvious blisters, lesions, and sores.
In addition, a blood test 5 days after your encounter, while not completely worthless, would be way too early to tell you if you were infected from this encounter. If your blood work came back positive for hsv1 or hsv2, it would indicate you have been infected for a lot longer than 5 days and not from this recent encounter.
Also, most people that are infected with oral hsv1 have no idea that they're infected. Unlike your brother, who I am assuming must get frequent outbreaks, most hsv1 infected people rarely if ever get recurring outbreaks. In fact, most people when they receive a positive hsv1 result on an antibody blood test are very surprised by the results because they never get outbreaks.
More importantly, if you were experiencing a primary oral hsv1 outbreak, e.g. sores inside your mouth, it typically would be quite severe. In other words, you would know because it would be quite obvious and painful.
The symptoms you describe sound like they could be a urinary tract infection, but I am NOT saying that it is. All I am saying is that your symptoms are far more consistent with a UTI than an Hsv genital infection.
One more thing, congratulations on using condoms and practicing safe sex. Good luck.
Nothing mentioned here points to hsv. Armpit bumps sounds like irritation from shaving.