I can only comment on my case - I had a similar symptom, but in my case, the swelling was almost the size of a golf ball. In that case, they did an ultrasound, and based on the visual symptoms, they removed half my thyroid almost immediately for biopsy. They told me that if it came back benign, the remaining half would be left in, and would function just fine. Unfortunately, it came back as malignant follicular variant, and I had the second half removed a couple weeks later. Completely manageable with thyroid hormone ever since, and luckily there'd been no invasion outside the thyroid itself. If you're not personally satisfied with the results/direction you've been given, my recommendation would be to get a second opinion.
You need clarification of the pathology report. However, it sounds like they are reading this as a benign follicular adenoma (not to be confused with "follicular neoplasm") -- this is one way of describing a benign thyroid nodule, we usually use benign colloid nodule to avoid this type of confusion however....