I'm 23 weeks and my little girl loves being against my bladder! It really blows I'm using the rest room so much!
The baby might be in a position where he is putting a lot of pressure on your bladder. As long as you aren't having any pain when you go and you're able to actually go when you feel like you need to, I'd say you're fine. The frequency is probably just gonna get worse from here on out