The first theory about the origin of the term "gum shoe" is that detectives often had to walk around in bad neighborhoods, trek around back alleys and generally pay more attention to where they were going than what they were stepping on. This type of activity invariably led to the detective ending up with wads of spent gum on the sole of his shoe.
Another theory tossed around on the Internet is that private detectives were hard for a person to avoid. This led to the belief that detectives were as sticky as the gum that gets stuck to the bottom of your shoe. A good private detective was just like that piece of gum -- you didn't notice him until he had caught you.
The rubber-soled shoe theory is derived from the actual shoes worn by turn-of-the-century detectives. The shoes were made with gum-rubber soles. These soles allowed detectives to walk around more quietly than when wearing other shoes available at the time. The first time the term showed up in literature, in "Confess" by A. H. Lewis in 1906, it had this meaning.
Although the gum-on-the-sole-of-the-shoe theory is a nice idea, there's little evidence that it's true. The sticky-as-a-piece-of-gum theory is also clever but has little evidence to back it up. This leaves the gum-rubber theory. The evidence includes the colloquial term "gum-shoe man," which was originally also slang for a quiet-footed thief, adopted by pulp literature and shortened to "gum shoe." This, coupled with the actual gum-rubber shoes that detectives wore, is the accepted etymology of the Online Etymology Dictionary, which makes it the clear winner.