The Tower of London was a prison, fortified royal residence, and UNIT headquarters in London. The Doctor visited it on more than one occasion and across several incarnations managed to be thrown in it as a prisoner.
Location
The Tower of London was located on the northern bank of the River Thames, close to Tower Bridge. The road which ran along the outside of its eastern wall was Tower Bridge Approach, while that which ran along the outside of the wall to the north was Tower Hill. The Traitors' Gate was located in the southern wall of the Tower of London. (TV: The Lie of the Land)
History
The Tower was built in the 11th century, construction starting in the year 1066, (PROSE: Magic of the Angels) and contained, among other buildings, the White Tower housing the Armour Museum and the Bloody Tower. One of the entrances to the Tower was called Traitor's Gate. (AUDIO: The Battle of the Tower)
The Doctor was imprisoned in the Tower on the Duke of Exeter's daughter (i.e., the rack). (AUDIO: Trail of the White Worm)
In 1400, the Archbishop of Canterbury Thomas Arundel threatened to imprison Queen Isabella, the wife of the deposed King Richard II of England, in the Tower. (AUDIO: The Doctor's Tale)
Henry VIII once sent the First Doctor to the Tower of London, which was where the Doctor wanted to be, as his TARDIS had also been taken there. (TV: "Strangers in Space")
Jane Grey was befriended from 19 to 20 July 1553 by her new lady-in-waiting Rani Chandra, a teenage girl from 2010, whilst residing in the Tower during her brief reign as Queen. (TV: Lost in Time) The First Doctor assisted Lady Jane and was present at her execution at the Tower. (PROSE: The Nine-Day Queen)
The Sixth Doctor and Evelyn Smythe were imprisoned there by the order of Queen Mary I in 1555, who was under the erroneous impression that they were plotting to assassinate her and place her Protestant half-sister Elizabeth on the throne. (AUDIO: The Marian Conspiracy)
On a different occasion, the Doctor shared a cell with Sir Walter Raleigh. (TV: The Mind of Evil)
In 1562, the War Doctor, Tenth Doctor, and Eleventh Doctor were imprisoned in the Tower by Elizabeth I. The Eleventh Doctor left a message in the Tower, which in the future allowed Clara Oswald to use Captain Jack Harkness' vortex manipulator to escape the Zygons and free the Doctor. (TV: The Day of the Doctor)
In 1588, the Fourth Doctor was imprisoned in the Tower. (AUDIO: The Devil's Armada)
In November 1605, King James I of England jokingly threatened to imprison Vicki Pallister in the Tower. (PROSE: The Plotters)
In March 1616, Sir Walter Raleigh was released from the Tower and arranged for William Shakespeare to be poisoned. (PROSE: The Empire of Glass)
In December 1648, the Second Doctor, Polly Wright and Jamie McCrimmon were imprisoned in the Tower on Oliver Cromwell's orders. (PROSE: The Roundheads)
The Eleventh Doctor was once imprisoned in the Tower without trial on the personal order of King Charles II. Two days later, a sphere twenty feet across was seen heading away from the tower carrying the Doctor. (TV: The Impossible Astronaut)
After the Glorious Revolution in November 1688, Lord Chancellor George Jeffreys was imprisoned in the Tower. (AUDIO: The Glorious Revolution)
The Fifth Doctor would return to a cell there, this time with Nyssa, after falling foul of Sir Isaac Newton in the late 17th century. (AUDIO: Summer)
In an alternate timeline in which John Hart became prince consort to Queen Victoria, he claimed to Jack Harkness that he owned a pair of keys to the Tower of London. (AUDIO: The Death of Captain Jack)
In summer 1897, Vastra, Jenny Flint, and Strax were asked to guard the Giant's Heart until it could be set into Queen Victoria's crown during her Diamond Jubilee. (PROSE: The Giant's Heart)
During World War II, Nazi officer Rudolf Hess was imprisoned in the Tower. (PROSE: The Domino Effect)
Barbara Wright visited the Tower when she was thirteen. (PROSE: Byzantium!)
While exiled to Earth, the Third Doctor once helped the Brigadier track down Hingrad, an alien master of disguise who had a secret base at the Tower Hill tube station. From this base, directly under the Tower, the thief hoped to gain access to the British Crown Jewels. Although a combination of the Doctor, UNIT soldiers and Beefeaters were unable to capture the thief, they did prevent him from stealing the jewels. (COMIC: Secret of the Tower)
Ian Chesterton opined to the First Doctor that Frankenstein's House of Horrors at the 1996 Festival of Ghana (which they mistakenly believed to have been within a human mind) had more spooks than the Tower of London. (TV: The Chase)
The newly regenerated Eleventh Doctor flew west over the Tower, whilst clinging by his fingers from his TARDIS' doorway, before climbing back aboard. (TV: The Eleventh Hour)
By 2006, UNIT headquarters was located beneath the Tower of London. From there, Prime Minister Harriet Jones directed the responses of UNIT and Torchwood to the Sycorax invasion of Earth. (TV: The Christmas Invasion) The headquarters was still used in 2009, although UNIT also maintained mobile bases in lorries. (TV: The Sontaran Stratagem)
In an alternative timeline, both the Sixth Doctor and the last surviving Dalek were imprisoned in the Tower from 1903 to 2003. (AUDIO: Jubilee)
In another alternative timeline, the Eleventh Doctor was imprisoned in the Tower by Holy Roman Emperor Winston Churchill on the perpetual 22 April 2011. (TV: The Wedding of River Song)
The Tower was graffitied by Monos, alongside other London landmarks. (COMIC: Sticks & Stones)
In, 2012, before the events that brought him together with his former selves, the Eleventh Doctor said he had been imprisoned in the Tower five or six times. (PROSE: Magic of the Angels) However, he had been held prisoner in the tower three more times than what he recalled when he made this statement, one of which had not taken place yet from his perspective. At the time, he was unaware two of his past incarnations had forgotten being imprisoned along with this version of himself due to their time streams being out of sync, which the Eleventh Doctor learned about when he experienced these events firsthand and was finally able to retain the memory of them. (TV: The Day of the Doctor)
Later in the first half of the 21st century, Kate Stewart summoned the Eleventh Doctor to the Tower to assist in UNIT's study of the newly-activated Shakri cubes. (TV: The Power of Three) By 2013, the entire Tower was TARDIS-proofed, preventing TARDISes from landing there. By this point, the Black Archive was now located inside UNIT HQ. (TV: The Day of the Doctor)
In September 2015, Kate Stewart, Osgood, and Bonnie watched Daniel Claremont's speech on global warming from the Tower of London. (COMIC: Clara Oswald and the School of Death)
When the Department controlled Britain around the year 2050, the Tower of London became Dauntless Prison, a prison for all the aliens that arrived on Earth. When K9 Mark 2 and his friends released the alien prisoners there in 2050, the Department's treatment of the prisoners was leaked to the media and the prison was shut down. (TV: Liberation)
Some time around the 29th century onwards, during the reign of Elizabeth X, the tower or a new facility named after the original tower, was incorporated into Starship UK. Its dungeon contained the devices used to torture the star whale beneath the craft into moving. Amy Pond visited the Tower during this period with the Eleventh Doctor. (TV: The Beast Below)
Other references
The Tower of London was seen on a map of London used during the Monk invasion of the late 2010s. (TV: The Lie of the Land)