“I don’t know how many Christmases I’ve gone to and they’re like ‘You think you’re ever gonna want a kid?’” singer says