This can be quite normal at only 20 weeks, you can go days with not feeling a thing, then a few days feeling lots. I felt like this too at 19 weeks, i thought i'd felt him more at 17-18 weeks than at 19! But an ultrasound showed he was facing towards my spine therefore i couldn't feel any kicks. About 24 weeks you should be feeling more prominent kicks almost on a daily basis.
I'm happy as long as I feel him move, first time mum worries haha but if he goes a day without kicking I'm definitely calling my midwife. Thanks
Your baby is still pretty little he probably moved into a spot you just can't feel him, if your worried though you should call your doctor