William Burnes - the poet's father - was born in Dunnottar near Stonehaven in 1721 and became, like his own father, a gardener. He had to leave his native county in search of work and, after a period spent in Edinburgh, arrived in Ayrshire in 1750.

He began to build a cottage in Alloway in 1757, the year in which he married Agnes Brown. It was there that their eldest child, Robert, was born on 25 January 1759.