It was built between 1765 and 1784 on land belonging to Henry William Portman. It included residences of Alexander Hamilton, 10th Duke of Hamilton, Sir Brook Bridges, 3rd Baronet, Henry Pelham-Clinton, 4th Duke of Newcastle-under-Lyne, George Keppel, 6th Earl of Albemarle, Sir Charles Asgill, 1st Baronet and William Henry Percy. Alexander Duff, 1st Duke of Fife maintained his London residence at 15 Portman Square.
Montagu House, at the northwest corner, was built by James Stuart for Elizabeth Montagu. She used to give a roast beef and plum pudding dinner for chimneysweeps and their apprentices on Mayday. One of them, David Porter, grew up to be a builder and named Montagu square in her honour.
