Mine generally refuse to leave the kittens at all the first day, then from day two they are desperate to get away from them for a while - but only for a very short time then they are back. The time away of course gradually increases. As long as the kittens look well fed and the mother keeps going back, try not to worry.

Liz