Famous former residents include Charlie Chaplin and Michael Caine, who were born and grew up locally.

In the medieval period it was part of rural Surrey, in the manor of Walworth. The parish was called St Mary, Newington, which church occupied the southwest side of today's southern roundabout, near the Tabernacle, and was first recorded by name in 1222.

St Mary's Church was rebuilt in 1720 and completely replaced in 1790, to a design of Francis Hurlbatt. Within another hundred years this too was to be demolished, with its replacement on Kennington Park Road ready in 1876. It was destroyed in 1940 by enemy action. The remains of the tower and an arch were incorporated into its replacement of 1958. The open space is still known as St Mary's Churchyard, and the narrow pedestrian walk at its south end is Churchyard Row.

The name Elephant and Castle is derived from a coaching inn. The earliest surviving record of this name relating to the area is in the Court Leet Book of the Manor of Walworth, which met at "Elephant and Castle, Newington" on 21 March 1765. Previously the site was occupied by a blacksmith and cutler - the coat of arms of the Worshipful Company of Cutlers features an elephant with a castle (possibly meant to be a howdah) on its back, which in turn was used because of the use of elephant ivory in handles.