The English language has one of the largest vocabularies in any known language, and therefore poses a difficulty for any ESL student. The biggest difficulties arising within the vocabulary are first and foremost phrasal verbs. The list of phrasal verb variations and meanings is seemingly endless, and students find it confusing when one phrasal verb is employed for different meanings. Additionally, collocations (ex. play an instrument) -- which are widely used by native speakers -- are quite hard for English learners to grasp. Finally, slang -- also widely used and always changing -- confuses students particularly since it is commonly used in informal situations, which is not the same as the more formal English taught in classrooms.
The English language poses challenges to the student first and foremost in its verb tenses: some of the tenses are nonexistent or hardly ever used in other languages. For example, romance language speakers have a great deal of trouble to understand and properly use the present continuous tense. Furthermore, the wide variety of idiomatic usage in the English language, in addition to its vast array of articles -- which include definite and indefinite articles, as well as the zero article -- are two aspects of the language's grammar that learners have a hard time learning; particularly the latter one since English uses articles in a different way from other languages.
The correct pronunciation of sounds in quite hard to master for English students. Many of the sounds in English are not common in other languages, but must be learned because English relies on many subtleties in pronunciation to communicate the right message. Additionally, English pronunciation has many variations depending on the country in which it is spoken. While for native English speakers this can be a matter to joke about, for ESL students it represents a true challenge in oral comprehension if they are in an environment in which the English they learned is not the one being spoken. For example, for a ESL student that has had a British teacher for years, visiting the US can make them lose confidence in their knowledge of their language, since it is hard for them shift from a British accent to an American one.
This includes the difficulty presented when reading a word and not being able to pronounce it correctly, and knowing a sound but not being able to write it correctly. English is not written as it sounds, nor pronounced as it's read (Spanish is, for example). Although there are many rules to guide the students through speaking and writing in English, none are 100% reliable: all of them have exceptions, which must simply be memorized.