Hi, being a retired elementary school principal...I do have a few ideas.
First, give back the toys. I assume you figured he wanted to play with them rather then do the work. But, toys are gone an now what. Tell him you want him to take a 10 min break from playing...get something done...and then go back to playing for awhile.
But the main thing is I/we don't know what you are trying to do with him. Is the work you are trying to do with him, packets that have been sent home? is he doing zoom lessons with the teacher? Does he have any interaction with the teacher at all? Oh, does he know his letters and numbers (to 20)?
And I realize its tough to do...but, typically trying to do work after dinner at this age is difficult. It would be better to try and get something done in the late afternoon....but maybe impossible with your wife's schedule.
Anyway, please try and answer the questions I have asked. By the way, my grandson is 5 and is in kindergarten - a combo K/1 class - all done by Zoom. Its kind of crazy. And having a two year old brother running around is not helpful.
I don't know what is going on and I believe you need professional help form someone who might, but not having Pre-K shouldn't be dwelled on to make you feel guilty. Nobody had Pre-K for most of human history, it didn't exist, so it's hard to see how it's a necessity now. Something is going on, though, and if you and your wife can't figure it out, don't feel bad about needing to seek out help. Peace.
He's so young, and I wonder if he's had a chance to be evaluated for any kind of learning or processing differences. If he has even mild dyslexia, for example, he may be getting frustrated with not understanding, leading to the defiance and acting out.
Missing Pre-K, while valid, means he missed a chance for evaluations from professionals for his age group.
It could also be some social anxiety - he was home alone with you and Mom for a long time. Have you asked him, without anger or punishment, what's going on and why he does what he does? I'd ask him specifically - like when he gets defiant, what does he do? If he gets mad, and leaves the room, ask him, "What are you feeling when you leave the room? What is it inside you that tells you to leave?" He won't know what defiant is, so just be really specific, non-judgemental and don't label anything as bad, angry, mad, etc. Let him tell you how he's feeling.
You can let him lead the conversation, and go from there.
Good luck. It's awful seeing the little ones so frustrated.