I'm glad you talked to her.
Maybe you should keep him in a little bit again, just to re-settle him into your home.
I find this subject quite intriguing really as if my neighbours came to me and asked me not to feed their cats, I'm not sure how I would take it, as I love their cats too. Obviously I'd try not to do it for a while, but I would find it hard.