The Romans arrived in Caister-on-Sea in the 1st century AD. The name Caister-on-Sea derives from the Latin castra meaning castle, and Caister-on-Sea was the site of a Roman fort associated with the Saxon Shore. To the west of the naval base, a civilian settlement, a vicus, was established. Most of the original site now lies under modern housing. A small section, managed by English Heritage, is open to the public.
The later, medieval Caister Castle is actually in the adjacent parish of West Caister.