If I ever saw a tick in my skin again, I would go get antibiotics right away. I would never take the risk of getting this disease again! The risk is that you might not get the usual rash, and then don't know you have Lyme until it's established. Then it's harder to treat.
I agree with Jackie, I would get on antibiotics right now because you don;t want to take the chance of it turning into much more like we all have. if caught early you can avoid that. Good Luck.
I'd call the doc tomorrow, or go to urgent care, and tell them about the tick and request a course of antibiotics. If the first doc blows you off, go to another one, and keep going.
Some docs will say that if the tick is not attached for at least X hours/days, then it cannot transmit disease. I know of no studies showing the absolute cutoff, and most people don't know when the tick first bit. That the bite site swelled up even a little says that there was some reaction going on between your immune system and the tick, and whatever it was, it's not necessarily benign.
The risk/reward weighs heavily in favor of two weeks of antibiotics rather than waiting to see if you become terribly ill with Lyme, which is a long and uncertain cure after the very early stages (which is where you are now).
It's like driving without a seat belt: maybe nothing bad happens, but if something bad happens, it could be really REALLY bad, and the inconvenience of wearing a seat belt is nothing compared to an unrestrained crash.
So ... I'd 'buckle up' and get antibiotics -- quickly.