You present a very complex question, so for starters let me offer a simple indea (sorry if it was in your post and I missed it).
Ask your primary car or other doctor most familiar with your issue to recommend a specialist. From what you said I don't see how a "sleep study" could help - but I myself have only considered and yet may asking my Primary Care to give advice on a sleep study or other medical approach to my sleep issues, which are different than yours.
My reply my help stimulate other inputs, hope so and hope you get an answer and get good sleep every 24 hour period.
That's what I was thinking I was going to need to do. The only problem is my Primary care doctor is a allergy/asthma specialist. I only see other doctors when I need to go to the gynecologist or I'll see a general medicine doctor when I can't get in to my allergist but I have a very low opinion of them as they didn't diagnose my asthma and they gave me two different diagnosis as to what was wrong with me. It was only when I went to see a specialist about my allergies that I was diagnosed with asthma.
I do however plan on seeing someone about my anxiety which is in a partial feedback loop with my insomnia
In spite of my browsing in this Community I have no experience with a Sleep Specialist, I don't even know what they are called, if they in fact exist.
I too think mental issue affect my sleep problem. I had no problem sleeping thought my adult life, problems started only when I reached senior age - so another big difference between us, age. My wife has suggested I see a psychiatrist about my dream problems which are at the root of my sleep problem. I think I have some anxiety due to the reality of accepting aging, it looked a lot easier when I was middle age and my mother was in her sixties asking "how did I get so old?" Now I understand the question. Seems I'm talking about something that has nothing to do with your situation, and that may be the case, but I did want to describe my personal experience which I think connects dreaming and sleep problems with mental health.
If you know that there is a MD specialist for sleep I'd appreciate learning the name of that specialty.
They exist but they usually have studied a field like neurology or pulmonology. My problems at this point I think were biological to start with but definitely exasperated by psychological stuff, like my anxiety. In an attempt to improve my overall stress level I'm going to be trying some different relaxation techniques and I'm hoping they'll help me sleep as well.
My reasoning for the biological aspect is that I had insomnia before, it got worse after I was on meds not recommended for people under 18 (I was 14 at the time), and it hasn't gotten better since I've been off meds.
What I'm probably looking for is a psychologist that specializes in anxiety and sleep disorders. I know I'm planning on going back to see one, just not the one I had been seeing way to much baggage.
My dad is going to talk to the guy who my brother saw for his ADD to see if he is either willing to take me on or can recommend me to someone.
My wife has diagnosed anxiety and depression problems and has a psychiatrist and psychologist. She is not seeing the psycologist at this time but is still taking anxiety and depression medications prescribed by the psychiatrist. She also takes a generic sleeping pill which seems to be less effective the longer she takes it. She doesn't have much remembered dream problems but often turns on a light to read, or goes out of the bedroom if she is having a real struggle, such as last night. I mention only as information, I can't reach any hard conclusions other than it is an ongoing battle. She does come from a family with other cases of serious depression. She had no problems that I was aware of until about he age of 55 - but I suppose the depression was present, just not dominate when we had children and jobs to occupy our minds. We're now retired - as I said that too is a difficult life-style/health transition.
I am, it may be again, trying to stop snacking after 8 PM. I usually stay up until about 11:30 PM. Last night the first, and while I had the usual amount of remembered dreams, it seems they were less depressing/troublesome. The point here being, sleeping is known to be better on an "empty" stomach, not starving noted.
Well empty stomach doesn't always work. If I haven't had something to eat (little like fruit) I wake up with a gnawing stomach in the middle of the night. When I can get to bed early-ish I tend to sleep like 8 hours if I don't have to wake up before then. I know I don't fall asleep nearly as fast as most people.
I am back to seeing a psycologist and plan on bringing it up at the next appointment as it didn't come up in the first appointment.
Thanks for letting me know of your efforts - I have this thread on my "watch list" so I will know when a reply is made here without having to search to see if there is one.
I didn't stay on the no eat after 8 PM long, and last night I attended a talk by a young woman researcher from Princeton University on the Cosmic Microwave Background and our Universe. I got home about 10 PM and immediately had a bowl of cereal - then some chips and cheese... no wonder I need to lose 20 pounds. Yep, I had a full double 0r more feature dream this morning. I know much of my mental disturbance is my lack of preparations for end of life. I am in no immediate danger other than my age and heart condition (guess that's enough to get serious) but I have so much junk that is precious to me that needs to be thrown out, and then there's the whole issue of down sizing, selling our home and moving to something like a condo, still unassisted living but all on one floor with no big outside maintenance to do. This is why I don't go get help, I know what my problem is - well maybe I do need help, help getting started at getting the job done. This would make my wife much more comfortable too.