Your early ultrasounds indicate your conception date was either February 6th, 5th or 8th. Even without the condom use, the guy on the 21st of January is not going to be the dad, all his sperm would have been dead (if any had gotten out of the condom) by about the 27th, and your earliest projected ovulation by the three (not just one but three) ultrasounds would be the 5th. And, he wore a condom.
In your shoes, I wouldn't do a prenatal DNA test. The ultrasounds say when you ovulated and his sperm would have to have gotten out of the condom and then lived two weeks or more, and sperm just doesn't live that long.
You apparently misunderstood what you read about removing your IUD to be saying that if you remove your IUD it triggers ovulation. Removing your IUD, does NOT trigger ovulation. What I'm sure was meant by what you read is that if a woman has unprotected sex, and then removes the IUD, and then happens to ovulate when the sperm is still viable (about 4-6 days after the sex), she might get pregnant. But the sex you are worried about was not near when you ovulated (because, remember, your ultrasounds show when you ovulated).
If the event on the 21st is a secret or is making you feel guilty, it's no wonder you're sitting and thinking of worst-case scenarios. But your worst-case scenario is not based on your actual, real medical evidence, more just on your fears.
Did you have the IUD removal when you were having your period?