Thanks for your responses.

Yes, he's been neutered so it's not that.

We don't have a cat flap and it's a rented property so no chance of putting one in either. Ditto the garden - it's not secure and would be difficult to catproof due to the layout.

I let him out in the garden with me earlier and he shot off under the fence to terrorise the cats in next doors garden. That has pretty much made my mind up for me - he'll have to have some sort of enclosure or stick it out in the house.