If he was sleeping with her while being involved with you, you have good reason to be scared about it. I also find that people are more honest when they're drunk. Their filters are off. Some people say things they don't really mean depending on the type of drunk they are, but oftentimes, there's some element of truth. I don't know that anything will be able to calm this fear and it may be healthy to consider moving on. How long has this been consuming you? How long ago did you find this all out?
If he cheated on you, my opinion is to leave him. People like that need not be trusted.