In 1239 the Augustinian Priory of Black Canons was founded near the ancient ferry at St Olaves. The original dedication was to "St Olave, The Blessed Virgin Mary, and St Edmund, King and Martyr". Saint Olaf was King of Norway. He was born ca. 995 AD and Christianised Norway. In Suffolk, there was no other dedication to Saint Olaf, but two in Norfolk, and over fifty in the rest of England, with six in London.
The Priory was allowed to hold an annual fair on St Olave’s Day, 29 July. It was also given the lordship over Herringfleet and Burgh St Peter.
The area has been excavated and several burials in the Canons’ cemetery discovered.