These are the humble, nameless villagers whose lives and deaths are the subject of the poem.
While the speaker doesn't name specific individuals, he uses them to represent the common humanity and the shared experience of life and death. The poem explores the lives of these forgotten people, their potential, and the impact their lives had on their community.
So, the forefathers are the unknown villagers, whose lives, while seemingly insignificant in the grand scheme of things, are still worthy of contemplation and respect. They are the true heroes of the poem, not because of their achievements, but because of their shared human experience.