The Priory Church of St. Peter and The Priory Gardens. The sheer beauty of Dunstable Borough Council’s addition of a curved path leading to a Pergola lends much to the sheer splender of the warm British Spring weather. The centuries old Priory Church of St. Peter and its gardens are situated in Bedfordshire, England.