I would say if he's not showing any signs of wanting to go out then keep him inside, secure the garden or build a run and let him out there. Cats don't need to wander off for days on end they're more than happy as indoor pet. I would never let mine out I couldn't live with myself if because of something stupid I did my cat ended up in the hospital because it had been hit by a car or something