RachyBobs is right, build a bond between you and the horse first.

Try to feel ok with the horse you gonna ride before you get on it, then I suggest identifying some ground work to do with your horse, don't just go for random riding, say to yourself for example, today I will do this ground exercise with my horse, 2 circles of trot (left & right) then one canter (left & right), when you finish this, calm your horse down and walk with him for few minutes, relax and end your workout, i believe you will feel much better and confident after several times of successful planned workouts, then you and the horse will be fine together.