In about 1363 the abbey was moved away from what was evidently a rather unhealthy location on swampy ground, and rebuilt on its present site. The old abbey was dismantled for building materials (though a fragment of it can still be seen) and as a result the 14th century abbey incorporates some Norman features.
Robert de Ufford, Earl of Suffolk, oversaw the rebuilding of the abbey on a much larger scale than the original and included several new chapels.
The outside walls were finished with fine chequerwork, while the windows had delicate Perpendicular-style tracery. The abbey was home to Augustinian canons who followed the Premonstratensian rule. Their domestic buildings were damaged by fire in the 1380s and rebuilt.
After the suppression the king bestowed the abbey on his brother-in-law Charles Brandon, Duke of Suffolk. A farmhouse was built into the corner of the nave and north transept and the abbey ruins were used as farm buildings, the church itself being used as a barn.
A new front was added to the house in the Georgian period and it is currently owned by music school Pro Corda. The Lady Chapel was also restored and furnished in 1918.