To cause infertility chlamydia has to progress to the testicles, to the tubes that carry sperm to the sacks, and it has to left unchecked until has caused irreparable tissue damage, now if you have had testicle inflammation, and that lingered with months or years infertility is a possible, but it would be unlikely, there are other more concerning causes of infertility than chlam. If you are concerned you can take action, improve your testosterone levels, improve you diet and start taking supplements that promote male urogenital health...or you can always test and confirm whether your fertility was affected.
You're fine. Avoid any sexual contact for 7 days or until your medication is gone, whichever is longer. You'll be fine. :)