This is such a difficult situation. I think it's highly likely that it's true that he behaves better now that you and your children are out of the house, and that he behaves fine at school. Children in unstable home situations are very motivated to protect their "turf", and he may be purposely acting out to get you and the new baby to leave, and secure his father's attentions for himself and his brother. That's a very common dynamic in children.
Do you have the resources - income and childcare - to raise your 3 daughters by yourself without him?
You have moved out and need to stay that way - you do have a job. This kid is showing signs of being a problem to society down the road. If it’s not too late gather evidence of abuse - pics, bruises, recordings - to show as evidence that your children are not safe in his company.
You said the guy has two children. How is the mental health of the other boy? Have you ever talked to the 8-year-old's mother about what is going on? I wouldn't be an ostrich about this, a child's life is being ruined, and of course your own kids' lives would be ruined if you would stay. I'd see what can be done through a social worker or social services to help the man understand his son has huge troubles and get some help. It's not illegal for you to remove a baby from an unsafe environment. But you may sooner or later need witnesses to what has been happening, and it would be a good idea to write up a record of everything that happens, so you can show your reasons that you took your baby and kids and left. If the boy is 8, has he been evaluated at school? Have you ever spoken to a school counselor about what is going on? Somewhere, you have to find help.