There is no way that herpes is spread this way.
Herpes isn't spread by licking, and once exposed to air, it would die. The skin on the breasts is too thick to be infected - not mucous membranes, correct - so there's no concern here.
Guilt doesn't equal risk. Just because the place is dirty doesn't mean people in it are infected with anything. Even if people inside the dirty place have something, you have no risk.