The diary, written in Dutch, refers to him as "Vader" which means "Father" in English. The Franks, like many families, used formal terms of address even in their private lives.
You might be thinking of another source, or perhaps remembering a film adaptation where Otto's name is used more frequently.