What thyroid test was actually done and what was the result? There are more than 1 thyroid tests that should be done to determine thyroid function and it doesn't sound like your doctor is being very thorough. When you post the result(s), please post reference ranges, as well, since ranges can vary from lab to lab.
There's always cause for concern when thyroid hormones are not adequate since the thyroid controls several important body functions. We'll be able to help you better, once we know exactly what your doctor is testing and what results are.