A lot of people have ureaplasma naturally, so you may not have gotten it sexually.
https://www.healthline.com/health/ureaplasma
You could be one of those people. It can be sexually transmitted, or not. There's no way for you to know now.
I agree with your doctor - if you aren't having any symptoms, there's no cause for concern. If you continue to have sex with people other than your wife, you should consider using condoms for all activities, including oral sex.
No judgements, just offering information.