I do that as well - hand stays really still

I think sometimes it isn't the cat's fault - it is instinct kicking in. If you watch a cat with a toy mouse they often pull it into their stomach and kick it with their back paws. Sometimes they start to do this with your hand.