The chapel (first mentioned in the twelfth century) was located at the highest point of the Island – one of the few areas naturally above the level of the tidal Thames. Chapel House Farm was built on the same spot but during the 1860s, Chapel House Farm was demolished to make room for Millwall Docks.
A short Chapel House Street, a cul-de-sac, arrived about 1895. In 1904, Chapel House Street was extended to link Westferry Road to East Ferry Road. No houses were yet built along it. A short section of the 1895 street would be renamed Chapel House Place (now Julian Place).
After the First World War, Chapel House Street became the centre of the Chapel House Estate.