Did you have symptoms at the time? If not, it isn't likely to happen. If you did, it's possible, but it would be more likely if you touched the outbreak and then touched your genitals. If the saliva didn't come into contact with your sore, then it's not possible.
Anytime you have genital symptoms of anything, you should abstain from sex. Even if it's just a fungal infection like jock itch, you don't want to share that with your partner.
Slight itching isn't likely to be herpes, and given that you had no symptoms when you masturbated, I'd assume that you don't have genital herpes. For the next few months, be really careful when you masturbate - don't use spit or lick your hand, or anything like that to aid in lubrication. Go get some KY Jelly or something instead.
Once you have antibodies, you can relax about this because auto-inoculation is a lot less likely.
If you have one swollen lymph in your groin, it doesn't mean much. The best advice I can give you right now is to go live your life. Do the fun things you like to do, don't focus on herpes. See if that helps your genital symptoms disappear. If they linger for more than a few days, follow up with your doctor. It could be something else going on.