Since the depression got worse after you went off the meds, it was probably going off the meds that did it. I think that you have to prepare your self in case you decide to go on and then back off. Realize what the meds are doing for you and than compensate for that when you go off. And probably "the other thing in your life" was the biggest factor.
Depression does happen to about 30 to 60% of people with ADHD, so learning how to cope with depression is worth taking the time to do so. Good Luck