Find a "remainder list" to start the conversion. This is done by dividing the number by 16 and calling the quotient Q1 and the remainder R1. The new dividend becomes Q1. Divide Q1 by 16 to get quotient Q2 and remainder R2. Continue doing this until the dividend is less than 16. Make this dividend the last remainder. This is easier to see by going through a typical example. Let N be the number to convert and N/16 has quotient Q1 and remainder R1. Q1/16 has quotient Q2 and remainder R2. Q2/16 has quotient Q3 and remainder R3. If Q3 is less than 16, make Q3 the last remainder -- set R4 = Q3 so the remainder list for N is R1, R2, R3 and R4.
Convert each number in the remainder list into its hexadecimal equivalent, then reverse the list. For example, if the remainder list is 5, 11, 0 and 3, converting each number to hexadecimal would give 5, B, 0 and 3. Reversing the list would give 3, 0, B and 5. This list is called the "hex digit list."
Combine the digits of the hex digit list to get the hexadecimal number that represents the conversion. This procedure is easier to understand with an example. To convert 11,000 decimal to a hexadecimal number, start by dividing 11,000 by 16. The quotient is 687 and the remainder is 8. The new dividend is 687 and the first item in the remainder list is 8. 687/16 = 42 with a remainder of 15. The new dividend is 42 and the remainder list is 8 and 15. 42/16 = 2 with a remainder of 10. The new dividend is 2 and the remainder list is 8, 15 and 10. The dividend is less than 16 so the final remainder list is 8, 15, 10 and 2. The hex digit list is 2, A, F and 8. Therefore, 11,000 decimal is the same as 2AF8 hexadecimal.