Here's why:
* He recognizes the writing: Calder has seen the lady writing before, recognizing it as his own work. He even mentions he has a similar piece in his own home.
* He knows it's valuable: Calder knows the lady writing is a rare and valuable piece of art. He understands the significance of what he stole.
* He doesn't want to hurt anyone: Calder is a good person at heart. He knows that stealing the lady writing is wrong, and the thought of the owner's loss makes him feel terrible.
This guilt is a key driver of Calder's character arc throughout the novel. He's torn between his desire to keep the writing and his desire to do the right thing. His guilt ultimately leads him to try and find the rightful owner and return the stolen art.