The character you're describing is "Tumnus the Faun", who has a distinctive goat-like appearance, including a small mustache. He is a gentle and kind creature who befriends Lucy Pevensie, the youngest of the Pevensie children, in the first book "The Lion, the Witch, and the Wardrobe."
Let me know if you have other book-related questions!