Sometimes certain cats dander can cause a reaction and other cats dander is fine- it's also possible that there is a issue with something in the house of the friend who has roaches- perhaps it's not as clean? It could be dust, mold, or just it not being sanitary that could be causing you the issue. I'd lean more towards it's the cat, especially if it's an outdoor cat.
Are you able to not go to that friends house for a while and see if it clears up?