Well, that sounds like no fun. Are you sexually active? You may want to get it checked out as herpes can present like an ulcer. However, you could also simply have an infected hair follicle or something simple like that. Keep your hands off of it but I do understand friction from normal activities is irritating it. Maybe dab a bit of vasoline on it to cut down on the friction. But it would be a good idea to get it checked out. Pus often means infection and you want to head that off. good luck