Not sure about the incontinence, but renal cysts DO cause pain. Not acute stabbing pain, mind you, but more of a dull, throbbing pain (3-4 on the pain scale, but ALL THE TIME).
Talk to your urologist about the possibility of removing them. It's a 50/50 shot as to whether or not it'll relive your pain, but I did it and it reduced my pain-level by a factor of 10.
If they're not operable, have your urologist refer you to a PM-doc (Pain Management Doctor). They'll be able to prescribe you some longer acting pain killers to deal with the long term chronic pain you're in for.
Good luck to you.
LMNO
I also have a renal cyst and I have stones as well. This causes me a lot of pain that can radiate through my back, into my groin/thighs, etc. The most amazing thing of all for the pain is HEAT. I swear by thermacare heating pads - the back ones wrap nicely around the kidney area and the heat reduces inflammation and pain. OTC pain relievers can actually do a good job of taking the edge off as well. As for the incontinence I'm not sure... I haven't dealt with that yet although sometimes I do start to feel like it's going to be an issue soon. Best of luck to you!