This is the obvious one, but it doesn't have to be as difficult as it may seem. Rather than memorizing every single aspect of your class, memorize the broad outline and a few specific details. Once you've done the activities a few times, you will likely remember them even better. It may also help to memorize the parts of your lesson in relation to each other. Remember the order of activities, and you will find it simple to teach without notes.
You probably won't need to make notes if you have prepared sufficiently for class. Go over all your activities ahead of time, and look for any problems that might come up. If you have kids of your own at home, try practicing some of the activities with them to see what clicks. Build a stable of activities that have worked well in the past. If you find yourself at a loss, substitute one of these activities for one that is failing.
Many new teachers make the mistake of thinking they have to maintain a firm control over the class at all times. It's important that students respect and listen to you, but that doesn't mean you shouldn't do the same. Let the students tell you what activities they enjoy, and help guide and direct their learning instead of controlling it. If students seem to be reacting well to one activity, continue to teach it for as long as it seems reasonable. Don't be afraid to go off on tangents if the students are interested in what you're teaching.
Kids almost always enjoy games, and using them in the classroom is a great way to teach a class without notes. Some teachers will use games in lieu of classroom activities, but a good teacher knows how to turn the activities into games. Try putting a spin on old games, such as giving students equations and having them solve them to find a number on a Bingo card or use games such as Pictionary to build vocabulary. Using games is a great way to keep kids invested in what they're learning.