I would say get back in the saddle
I was the same as you, I started horse riding when i was 5..i had a bad accident when i was 12, broke 3 ribs and fractured my skull..nearly 10 years old and hadnt been riding since.
3 weeks ago i picked up the courage to go to my local riding school. i just watched the horses and others having a lesson, then the next week i had a go myself...ive been back in the saddle 2 weeks now and i had forgotten how much i LOVE it!!
My confidence is slowly coming back, but my instructor is really happy with me and the progress ive made in just two weeks!

GO FOR IT