If you think the B complex stirs up your anxiety, why take it, just take B12. I have only heard from one other member who felt they had problems with D3 supplements. We recommended taking D3 injections and then when it was optimal try D3 supplements daily. I don't recall ever hearing anything further from the member. I mention cortisol because it is an antagonist of thyroid. Ferritin is very important for its effect on conversion of T4 to T3. A ferritin deficiency can also cause symptoms. Ferritin needs to be at least 100. It would be good to get a full iron test panel, including ferritin.
You need Vitamin D at least 50 ng/mL. I doubt that you will get sufficient D from the sun. I suggest that you supplement with about 2000 IU of D3 daily, or if that does not work for you then ask the doctor to give you injections to optimize your level. Vitamin D is very important for you. Also, B12 should be in the upper end of its range, so you can supplement with about 500 mcg of B12 daily. It is also important to know your cortisol and ferritin levels since they affect thyroid metabolism.
Of course the most important thing is to understand why your doctor is reducing your dose, when it is likely that you need a slightly higher dose. Everyone is different regarding what thyroid levels work best for them, so med dosages should not be based solely on FT4 and FT3 test results, and never based on TSH levels. So, you should try to get agreement with the doctor that the objective is to get your meed dosage to a level that relieves your hypothyroid symptoms.
There is much to discuss, but first let's make sure it is clear that your T4 and T3 tests were for Free T4 and Free T3, so that is good. Always make sure they test for the Frees, not Total T4 and Total T3. Most of the Totals is bound to protein and thus not active. The small portions not bound to protein are Free T4 and Free T3.
Next is to make clear that hypothyroidism is not just "inadequate thyroid hormone". Instead it is best defined as "insufficient T3 effect in tissue throughout the body due to inadequate supply of, or response to, thyroid hormone". So TISSUE T3 EFFECT determines your thyroid status. In addition to having adequate Free T4 and Free T3 levels, there are other important variables that affect the response to thyroid hormone. Two that are very important are cortisol, Vitamin D and ferritin. D should be at least 50 ng/mL and ferritin should be at least 100. Cortisol should be tested to assure adequacy. Also, since you mentioned fatigue, your B12 level should also be tested and supplemented as needed to reach the upper part of its range.
A good thyroid doctor will do all the needed tests and diagnose and treat clinically (based on symptoms), supported by test results, especially Free T4 and Free T3. In the untreated state, TSH is useful as a diagnostic only when at extreme levels. Within their reference ranges TSH has a weak correlation with either of the thyroid hormones. When a patient is being medicated with thyroid hormone, TSH is basically irrelevant. That is because when taking adequate doses of thyroid med in only one or two doses, it causes TSH to become suppressed for most of the day. That does not mean the patient is hyperthyroid, unless there are hyper symptoms due to excessive Free T4 and Free T3. Most doctors don't understand this and with a suppressed TSH they think there is a need to reduce the dose. Since you had no test result for TSH even, I have no idea why the doctor decided to reduce your dosage. As long as you continue to have hypothyroid symptoms, the dosage should be gradually increased until there is symptom relief. Every person is different in their thyroid heed, but typically this seems to take Free T4 approaching the middle of its range, and Free T3 in the upper part of its range. In addition you need to get tested and then supplement as needed for Vitamin D, B12 and ferritin.
So you are going to need to discuss this with your doctor and see if you can give him enough info to persuade him to increase your dosage not decrease. Toward that goal, you might make good use of this link.
https://www.ncbi.nlm.nih.gov/pubmed/29396968
In this study the authors quantified for the first time the effect of Free T3 on the incidence of hypothyroid symptoms. They concluded, 'Hypothyroid symptom relief was associated with both a T4 dose giving TSH-suppression below the lower reference limit and FT3 elevated further into the upper half of its reference range. ' Of course to achieve these levels usually requires the addition of T3, or a switch to a desiccated type like Armour.
Also you can click on my name and then scroll down to my Journal and read at least the one page overview of a paper on Diagnosis and Treatment of Hypothyrodisim: A Patient's Perspective. There is also a link to the full paper. Everything I have stated here is supported with extensive scientific evidence. You can also use the Overview with your doctor.
Before further discussion, do you take your Armour dose all at onece, or split the dose and take 1/2 in the morning and 1/2 in the afternoon? Also, do you defer your morning dose of Armour until after the blood draw for thyroid tests? If not, how long after the morning dose was the blood draw for your latest tests?