As long as you are gentle with yourself and your hands are clean as anniebrooke says (just wash your hands before hand or use hand sanitizer), it's not a problem.
If you do fingering, the hands must be scrupulously clean, including under the fingernails. Hands carry a lot of bacteria.