From your description, it seems highly unlikely that you would not awaken immediately if being physically attacked. To me it sounds like an overactive imagination. If you just let this go and not worry about, and find that nothing bad happens, you will prove this to yourself.