Just as an update - I took heart from all your replies - you do begin to think you are the only person with a particular problem, don't you??
So, I went and had a talk with my neighbour and explained as tactfully as I could, that I really needed her not to feed Remus or let him in as it was very confusing for him, especially as he is still getting used to his new home, and that I was worried he would just move in with her and that I would lose him. I also explained that worried me because I knew their finances weren't good and it would be an extra financial burden for her to take on another cat - that seemed to alarm her, I don't think she thought I would pass on the financial responsibility along with the pleasure of owning him (he is a lovely cat with a great personality). She said she understood where I was coming from and agreed not to let him - not sure if she will keep it up - but hopefully he will get used to the state of affairs as their door is always open in the summer so it will be even harder to keep him out then!! Remus seems a bit miffed though, he was in and out like a Jack in the Box last night - I think he is struggling to understand where his second home went!!
So, thanks for all your support - I really appreciate it.