Define your expectations. If you are testing math ability; what aspects of math are most important to you? The same goes for graphic design, English, analytical ability and others. Define the milestones you expect people taking the test to pass. Then add on some wild hopes; things you do not expect people to pass but you hope they will.
Break the ability down into its core test components. For example, if you are testing English ability you will break it down into speaking, listening, reading and writing. Complement this with your testing criteria. For English this would be vocabulary, fluidity, style, spelling and grammar.
Design your questions. Choose carefully because you want to touch a wide range of bases. You can offer multiple choice questions and direct questions with a simple answer such as; who was the first King of England? But you also need to test critical thinking with essay style questions. When testing language abilities you also need to include missing words, word rearrangement and so on. Make the difficulty of questions vary in range and weigh them so tougher questions yield more marks.
Review and revise the test. Make sure it hits your expectations, covers the basics and goes beyond them, tests the core elements of the ability, and is clear and understandable.
Test the test. Ask someone you know whose ability is beyond reproach and ask them to sit the test. Ask for honest feedback and make any adjustments needed.