How was the estimation of your due date calculated? Did you (or a doctor) just calculate from the first day of your last regular period, or did the due date result from a sonogram or lab work?
That sounds pretty reliable. It would be unusual to be 4 weeks further along and not have that caught at an ultrasound performed around 16 weeks.
Well, definitely the more likely candidate is the guy you had relations with at the end of December.