Some cats only stay with their babies whilst they feed and others are glued to their kitts- so don't worry too much on that score.
Have you managed to weigh the babies?- it's the best way of working out if they're feeding or not.
Generally if a baby is unwell or failing to thrive Mum will shove it off to one side - so the fact that your little one is in the heap sounds quite positive.