What was the Science Fiction novel that featured a world-wide computer accessed by each citizen carrying stick-like device called joystick?

The science fiction novel you are thinking of is likely "The Moon is a Harsh Mistress" by Robert Heinlein, published in 1966.

In this story, the inhabitants of a lunar colony use a device called a "mike" (short for "microphone") to access the colony's central computer, "Mike," and interact with it. This "mike" is described as a small, handheld device, similar to a joystick, and is used for communication, data entry, and other functions.

While not exactly a "joystick" in the modern sense, the "mike" in Heinlein's book shares some key characteristics with the concept you described, including its handheld form and use for accessing a central computer system.