The hand is not conducive to a herpes infection with very thick skin. With no sores present it is essentially a zero chance of infection.
Your partner's infection will be just genital and as such receiving oral sex is zero risk of infection.
Also I noticed I have a tiny cut on my middle finger, im not sure if it was there when it happened, but he did ejaculate into my hands, I washed them a couple of minutes afterwards. Would that drastically change things?
No change, extremely low risk. No need for any concern.