You said he confused you. Well, he's trying to confuse you. He's also creating a workplace that is much less comfortable for you, and creating a hostile workplace is against the law. Unsolicited touching of someone's body in the workplace is entirely improper. (If he were doing this to someone who reports him, he could get fired, sued for sexual harassment or even get the company sued for turning a blind eye to him doing it.)
If you welcome it, you aren't obliged to report his actions, but touching someone in a way not necessary for the job and during work hours, is more of a domineering act than a friendly one. (If he were going at it for romantic reasons, it's been long enough that he would have asked you out by now.) In your shoes, I wouldn't trust the guy or his motives. I'm sorry if you like it, because it sounds like he's not going about this the way someone would if they were trying to develop a romance.
Don't waste any more time wondering. Ask him to have a coffee break with you in a cafe or cafeteria when there are other people around, and then straightforwardly ask him what the heck he is doing with the touches, and the staring, and all. Give him a pretty comprehensive list of all the things you've mentioned here, and ask him about all of it. This should clear the air. If romance is really what he is trying for (and he's not just on some kind of a power trip to dominate women), he probably will ask you out if that is really what is on his mind. (Then you have to decide what to say, so be ready with a response.) But it sounds like a very odd way to initiate a romance, which suggests he's probably mostly just trying to confuse and dominate you. Asking him flat-out what he's doing will take the dominance out of it for him (since he suddenly won't be the one in charge of the narrative). At least you'll be giving him warning that you're keeping a record of what he's doing.
Some guys get a kick out of seeing how far they can go touching women without their permission, and often they know how to do it in such a way that they can pretend it's innocent if challenged. You don't want to go down a gamesmanship rabbit hole with him about this. Bluntly asking what's going on will make the whole game less fun for him and probably take his heat off of you.
If things escalate, take the whole list to your HR department and make a complaint.
I have a semi strict rule that work place dalliances should be avoided. I know we spend a lot of time there but if they go bad, our social life bleeds over into our work space.
I think I'd look for a different place to meet someone nice. You are wise to be aware that not all are on the up and up. If you meet someone doing an activity (at a gym, a volunteer organization, a book club, a running club, etc.), you are in a positio of already having something in common.
I just don't do work place things, it can backfire.