The London Stone, from which it has been suggested distances were measured in Roman times, was originally situated in the middle of Cannon Street. It was later set into the wall of St Swithin's Church, and now rests in a case to the side of the street. The Roman governor's palace Praetorium may also have been located in this area, between the principal street of Roman Londinium and the River Thames. The remains of a very large high status building were found with a garden, water pools and several large halls, some of them decorated with mosaic floors. The plan of the building is only partly preserved, but was erected in the second part of the 1st century and was in use until around 300, rebuilt and renovated several times.

The area around Cannon Street became the place of residence of candle makers. The name first appears as Candelwrichstrete Street in 1190. The name was shortened over 60 times as a result of the local dialect and settled down as Cannon Street in the 17th century.

In the late 19th century Cannon Street was occupied by large wholesale warehouses, especially of cotton goods and other fabrics.

Opened by the South Eastern Railway on 1 September 1866, Cannon Street station was designed by Sir John Hawkshaw and J.W. Barry and was characterised by its two Wren-style towers, 23 ft square and 135 ft high, which faced on to the River Thames. The towers supported a 700 ft long iron train shed crowned by a high single arch, almost semicircular, of glass and iron. To this was joined in 1867 an Italianate style hotel and forecourt designed by E.M. Barry which provided much of the station's passenger facilities as well as an impressive architectural frontispiece to the street. The station is carried on a brick viaduct over Upper Thames Street. Below this viaduct there are remains of a number of Roman buildings, which form a scheduled ancient monument.

On 6 October 1884, the final section of the Underground's Inner Circle was opened along with Cannon Street underground station.

The station's prime location coupled with the property boom of the 1950s and the need for British Rail to seek alternative revenue streams made war-damaged Cannon Street a prime target for property developers. In preparation for redevelopment the remains of the once magnificent train shed roof had been demolished in 1958, and Barry's hotel (which had been used as offices since 1931) soon followed in 1960.

All that now remains of the original station architecture are twin red-brick towers.