No, that wouldn't be the only symptom. You probably just have a backache. If you both tested negative at least 3 months after your last risk event with other people, neither of you has HIV. If you're going to continue to be concerned about having unprotected intercourse with him, then either wear a condom or get tested together again.
Symptoms, or lack thereof, are not predictive of a person's HIV status. What was your risk?