I think there is always one kitten in every litter that you just bond with that bit more. Sometimes you have to go with your sensible side and let them go, other wise you would want to keep them all

As the others have said, it will be taking up a place that you will need in the future, better to part now than a few years down the line when it will be much harder. As long as kitty has an excellent home to go to where he will be loved for the rest of his life thats all that matters