Make a list of all of the things you've done to prepare yourself professionally. It's very easy to forget that talk you gave to the first-year teachers in the second year of your MA, or that time you helped design the biology department's newsletter. All of these activities make you more appealing to the people who can actually grant you that fellowship.
Organize your CV into different sections. You'll want to list your education, of course. Include your teaching, speaking and relevant work experiences. Add in the awards you've earned along the way. You should definitely mention any journal publications you have. Sections may vary depending on the field you're in. If you study insects and you discovered a new species of beetle, you should definitely mention that!
Type all the information into your word processing program so you have everything in one place. Play around with the location of each section. What do you want the committee to see first? Your teaching experience or your list of awards? If one of these is more impressive than the other, you may want to put the most impressive credit first.
Ensure that you're not using the same words throughout your CV. Okay, you'll likely use "university" a lot, but that's expected. Scrutinize each verb you use to make sure you're using the most powerful, appropriate ones possible without repeating yourself. For example, avoid saying that you "organized" a fundraiser, "organized" class materials and "organized" a department activity. Instead, say that you "compiled" class materials and "arranged" that department activity.
Proofread your CV. This is very important! It's incredibly easy to make a small but fatal error. The committee may not think well of you if you apply for a position in the "psychics" department and not the "physics" department. Put the CV aside for a few days and then look at it with fresh eyes. Show what you have to a friend for their feedback.