I was ravenous in general on lithium, I just grazed all day long. Before I stopped taking it, I started eating only whole grain for carbs, avoiding pure starch, so I got the fiber, protein, minerals, and complex sugars all together. My weight stabilized by filling up on whole grain rice, quinoa, millet, or whatever. I went full psychotic still on Lithium, so not sure if that method would work long term or for you. Good luck.
Lithium is known to increase hunger in some. Bipolar itself is believed to have something to do with serotonin levels at least in part, though nothing is 100% certain. You're experiencing a normal side effect. Definitely a good suggestion from Weaver to try to satiate the craving with something at least wholesome when you choose to give into it.