I understand some people have Afib and it is not apparent to them, and it does not cause a rise in heart rate. I have it and it comes with a rapid ventricular response -- making my heart rate 140-150 just sitting around and 180 when moving around. Mine was paroxamal, now is happening more often and I am on an anti-arrythmic medication (Norpace), Atenolol (beta blocker), Coumadin and Xanax to block the adenaline surge, to stay in normal sinus rhythm. Are you sure your heart rate is going down to 20? seems very low. Are you on a blood thinner or asprin? The Afib must be showing up on your ekg. Your doctor should be answering questions for you.
I don't know who diagnosed the AFib, but if it was your doctor, better a cardiologist, then I'd say it mus be so. But, I'd be more worried about the HR in the 20s. That is atypical to my knowledge for the AFib condition. AFib usually causes irregular and higher HR, not lower.