There's very little risk here.
The biggest risk is for him to get herpetic whitlow, which is herpes on the finger. If his skin was intact - no cuts - then he won't get it.
By the time he touched his penis and his mouth, there was too much exposure to air, movement, etc., for the virus to infect him.
However, when you have an outbreak, you are most contagious, and should tell partners when you do. He may have forgotten. What seems like a big deal to us often escapes others. I told a guy once, and a few years later, we reconnected, and when I brought it up again, he was all, "you have herpes?"
Is this someone you are in some kind of relationship with - even a friends with benefits thing? You might need to remind him if you are.