Unlike many other languages, the spelling system in the English language can be very tricky to learn. Phonetics are often not as useful in learning English, as similarly spelled words (such as "through" and "trough") can be pronounced very differently from one another. The fact that many English words are pronounced differently in certain parts of the world (America, Britain) can make learning how to pronounce certain English words even more difficult.
Depending upon the inflection given to certain words or syllables in English sentences, their meaning can be altered completely. For example, in the sentence "I ate the pizza," stressing the word "ate" tells you what the person did with the pizza, while stressing the word "pizza" tells you what the person ate. This is one of the biggest hurdles for those attempting to learn the subtle nuances of the language.
Pronouncing certain sounds in English is another large obstacle for non-native speakers. For example, the "ou" sound in the word "south" differs from the "ou" sound in the word "Southern." Learning the correct way to pronounce similar words such as these is often a matter of trial and error, and requires quite a bit of practice and experience with the language.
Grammatically, English can be a very difficult language to learn. There are more grammatical nuances in English than languages such as French or Spanish, and learning to be proficient in grammar is something that even native English speakers struggle with. For example, many people have issues understanding past and present tense (played and plays) in English, which can be rather confusing at times. Without a doubt, the best way to internalize English grammar is to read as much English text as possible.