Sometimes when I had some fresh chicken or fresh fish that I wanted to give to my neighbours cat, I took it round to them to give to the cat, as she was pregnant and I thought it would be nice for her. they gave it to the other cats, not the pregnant one,lolol. Anyway, another time they did ask me not to feed the nursing queen as she had the runs that day, so I didn't even though she begged all day at my door.
One time just after the kittens were born they went on holiday for a week and left a friend to feed the cats who didn't turn up, I was worried and then I got a text asking me to go and do the cats and check on the kittens, which of course I loved, and did so.
I think in my case it's ok to feed these cats.
But i do totally see things in a different light now too, and would never tempt another cat into my home.
Although in the back of my mind I do worry that they think I am stealing their cats!!