Observe closely. Your speech critique will only matter if you heard all of the speech. Pauses, utterances like "um" or "uh," and small nuances like posture, where a person looks and where he puts his hands will all come into play. Listen with your eyes as well as your ears.
Take notes. For a thorough critique it will be difficult to remember every single point you wanted to make, unless you write your ideas down as soon as you have them. Create your own shorthand so you're not missing the speech while taking time to write long sentences. For example, for things you liked, put down a plus sign and then your compliment. For areas that need improvement, put down a minus sign and then your suggestion.
Look for the arc. A good speech should have a clear beginning, middle and end. Notice whether the speech you're critiquing has a clear narrative arc.
Pinpoint the overall message. When the speech is over, you should know the exact point the speaker was trying to make. If you're unsure exactly what the speaker was getting at, include that in your notes. Note whether the speaker took time to develop his ideas rather than simply stating them without using examples to support his main point. Observe whether the speaker seemed to make up facts or used solid research.
Pay attention to the style. Speeches should flow naturally, in a dramatic version of a conversational tone. The speaker should not appear to be reading and/or speaking overly loud. Rather than a monotone delivery, the speaker's voice should have subtle nuances and a certain musicality. Notice whether the speaker used several big words the audience had trouble understanding, or used terms that all could interpret.
Notice whether the speaker kept it together. Was the speaker nervous? Did he stumble across certain words and mumble others? Did the speaker make eye contact with his audience? Did he deliver his lines clearly or fill the silence with "um's" and "uh's"? Write the answers to these questions in your notes.
Make suggestions for improvement. Don't state what you didn't like without offering a helpful alternative. Point out the things the speaker did well.