If Dr. Handsfield says it's rare, then I'd certainly believe that. What I've read about it being common is based on animal models, not reality.
You've been married for 20 years. You and your wife have been tested numerous times, all negative, and have healthy children. I think it's safe to say you are free of chlamydia. You also probably have much bigger things to worry about, being the parents of (presumably) 4 kids.
Let it go now, and enjoy life. :)