If I were you, I would consult a behaviorist on this situation. The fact that someone is "allergic" to a dog or cat is THE NUMBER ONE excuse given when surrendering them for adoption. The plain fact is there are too many dogs and cats in the rescue system NOW, and to add more is to continue to overload an already-groaning cart. When people attempt to surrender an animal, oftentimes the rescue they are surrendering him or her to will attempt to help them deal with the situation through training instead of just immediately accepting the animal, and people have learned this. They know, however, if they say that someone in the house is "allergic" to the animal, there can be no arguments, because nobody is going to make you prove that someone is allergic to the pet.
As I said, most shelters and rescue groups are totally overloaded these days. So where they should be taking weeks to evaluate animals, many times they take the animal at face value and think "he seems OK. He's healthy, he passed the food aggression test, he passed his other personality tests, he can go up for adoption." The thing is, once they are out of the home situation they are accustomed to, it often takes weeks for the true personality to surface, because the dog is somewhat in a stress-induced shock at having been dumped off by what are often the only humans he knows how to live with. Then he gets into a situation like your home, which might be very much like the home he came from, kids and all, and he starts to feel more like his old self and he begins to display the behaviors that landed him in rescue in the first place.
Growling is not a behavior to mess around with. Growling is a warning. A warning you don't want to ignore or try to get past alone, especially if you have children. Ask your vet to recommend a good behaviorist. They will be able to help you work with the dog to get him out of his aggressive behaviors before the growl turns into something more. It's going to take persistence, a lot of work, and most of all, consistency. Dogs respond to consistency. You can't do things one way one day, and another way the next and have them "get it".
Something you can start with right away until you find someone to work with is to let the dog know where his place is in your family "pack". When you feed him, mix his food up in his bowl, and then put a little bit of food in your own hand. Hold his bowl like you are eating from it and substitute the food in your hand for the dog food, and you eat while he watches you eat from his bowl. HAVE YOUR KIDS DO THIS AS WELL. This is very important because it establishes their place ABOVE HIM in the pack in his mind. This is extremely important. He HAS GOT TO LEARN THAT YOUR KIDS ARE HIS SUPERIORS. If not, he will try to be their superior, and problems will continue.
Thanks Ghilly- we have done that. He will eat out of our hands and we can stick our hands in his dish when he eats. He only growls after 9pm at night when he's tired except for the very 1st night when he growled at my son. When playing we can take stuffed animals away from him etc. He retrieves and releases very well. I will call the vet but we live in a rural area so I don't know if we have access to a behavioral specialist. He did live in a foster home for a month before we adopted him and she said he showed no agression towards her. Hopefully we can get this figured out.
I agree with everything Ghilly recommended - particularly the bit about hiring a behaviorist. We have the same issue with our rescue dalmation, Doc. He's a perfect cupcake all day until around 9:00 p.m. when he's ready to crash out. I am the only one who can get near him with no problem. My husband has scars on his hands and face from "messing" with Doc after bed time.
To be fair, Hubby has had unrealistic expectations from the beginning. He expected Doc to be exactly the same personality as our older dog, Maggie. You can do anything with Maggie and won't get so much as a grunt out of her. Hubby likes to "drape" himself on top of Maggie and "suck up" on her. She loves it. Doc can't stand it. I get that. You have to let Doc know you're there, and once he opens his eyes and sees who is with him, he's fine. It's a matter of training Hubby instead of training Doc in my opinion.
A behaviorist is invaluable in teaching humans to read Dog Language. The more you can read a dog's body signals, the better you can communicate. Once you have communication down, anything is possible. :-)