I agree that being overweight puts a lot of strain on muscles/joints, as well as causing breathing difficulties.
You might want to check with your doctor and get some simple blood tests to insure that you don't have hypothyroidism, insulin resistance, etc, as these can all make losing weight more difficult, as well as causing aches/pains by themselves.
I totally agree that you need to opt for low impact exercise, such as swimming, water aerobics, yoga, etc. Once you start losing some weight, you should see some of the pains diminish.
Yes yoga can help you out as it also works for stress management and energizes most of your body parts.
Being overweight puts a lot of pressure on your joints and may be the cause of your breathing problems during exercise. Perhaps you should look into something low impact like swimming to help you get a bit more fit. Aerobics and yoga might also be an option because they're not so much cardio- so in my experience- they don't make it as hard to breath as say running would or walking up a steep hill.
I'd also suggest talking to your doctor. Perhaps you have asthma. It's a possibility that needs to be ruled out. It's also a good idea to talk to a dr before starting an exercise program.