The land on which Carlton House Terrace was built had once been part of the grounds of St James’s Palace, known as "the Royal Garden" and "the Wilderness". The Wilderness was at one time in the possession of Prince Rupert of the Rhine and was later called Upper Spring Garden.

From 1700 the land was held by Henry Boyle, who spent nearly £3000 on improving the existing house in the Royal Garden. Boyle was created Baron Carleton in 1714. On his death the lease passed to his nephew, Lord Burlington, and thence in 1732 to Frederick, Prince of Wales, eldest son of George II. After Frederick’s premature death in 1751, his widow Augusta continued living in the house. After her death in 1772, the house devolved to her son - George III - who in turn granted it to his eldest son, George, Prince of Wales. The Prince spent enormous sums on the property, running up huge debts. The house became a rival Court to his father. When the Prince became King George IV in 1820, he moved to Buckingham Palace.

George IV ordered Carlton House to be demolished in 1826. Architect John Nash proposed six terraces of houses along the north of St James’s Park. Only two were built and these became Carlton House Terrace. Nash planned to link the two with a large domed fountain between them, but the idea was vetoed by the King. The Duke of York’s Steps took the place of the fountain and in 1834 the Duke of York’s Column was erected at the top of the steps.

The terraces are unusual for expensive Regency housing in that they have no mews to the rear to maximise the views of the park. The service accommodation was placed underneath in two storeys of basements.

In the 20th century the Terrace came under threat of demolition and redevelopment, due to a lack of demand for large central London houses. Plans were drawn up for the scheme but there was such an outcry which persuaded the Crown Commissioners not to proceed.

The Terrace was severely damaged by German bombing during the Second World War and rebuilt.