Obtain a French alphabet pronunciation chart. These are available online and in French language books. Note the difference in pronunciation for letters such as "g", which sounds like "ghay", or "y", which sounds like "ee-grek." Most of the French alphabet is pronounced differently than the English alphabet.
Listen to a recording of the French Alphabet being pronounced. You can do this one of several ways. Ask a French teacher to recite the alphabet for you and record it (but let her know first you will be recording). Check your local library for any tapes or videos of the French alphabet being spoken. You can also view a video online.
Speak along with the correct pronunciation of the alphabet. Repeat listening to the alphabet more than once and then try your hand at mimicking the pronunciation of the letters. Go slowly at first and appreciate that it will take time and practice to get it right.