Here's why:
* The speaker expresses deep feelings of love and longing: "When, in disgrace with fortune and men's eyes," "And on my low-degree that low regard," "When to the sessions of sweet silent thought"
* The speaker finds solace and joy in their friend's presence: "And in my love's sweet semblance, sweet reward"
* The speaker celebrates their friend's virtues: "And think how one man may get himself release from all these sorrows by obtaining grace"
Therefore, the most likely addressee is a friend who offers the speaker comfort, inspiration, and a sense of belonging.