There are a number of different types of nystagmus.
Most are from birth and some folks are much worse than others. This is a brain controlled phenomemon and many things can influence it's manifestation.
If you want to discover the kind of nystagmus you have, see a neurologist or neuro-ophthalmologist.