They wouldn't know just by looking at the house.
Firemen are trained to put out fires, not to judge the contents of a home. The idea that they would be able to tell if a house has books just by looking is a playful exaggeration.
Let me know if you'd like another riddle!