The first palace to be built by the Ottomans after the conquest of Istanbul in 1453 was in Beyazit. This later became known as the Old Palace or Saray-i Atik-i Amire. Overlooking the entrance to the Golden Horn and dominating the skyline of much of the city as it did, the site seemed auspicious. But, some say it lost its favor being too near to the commercial heart of the old capital of Byzantium, while others claim that a new palace became necessary as this hastily-built first imperial residence proved inadequate as a center for the head of state.
The chosen site for the new palace was on a promontory overlooking the confluence of the Istanbul waters, the Marmara, the Golden Horn and the Bosphorus, on the so-called first hill of the city. The promontory was already surrounded by the old sea walls of pre- Ottoman times. By adding a wall across the land side, which they called Sur-i Sultani, the Ottomans turned the ancient acropolis of Constantinople into an ideal site for a royal palace, set apart from the city and configured for easy defense. Then the foundations for the 'New Palace' or Saray-i Cedid-i Amire, that complex of buildings which grew organically over the centuries, were laid. Repeated additions to the Palace resulted in a vast sprawl of buildings clustered over the site, including the hermetic Harem complex and a number of sections for chamberlains, stewards and the many hundreds of palace staff. Behind the sea walls were a number of pavilions, which along with lodges outside the walls and some actually perched on the wall itself, were known by Westerners as the summer harem. These shore residences, the best known of which were the Yalikosku, Sepetciler Kasri, Incili Kosk and the Topkapi Pavilion, were built at various periods along the sea walls between Sirkeci and Ahirkapi. The Topkapi or Cannon Gate Pavilion was a wooden building that stood at the cannon gate in the sea walls. We know this particular pavilion, popularly called the Topkapi Saray, or Topkapi Palace, was restored by Mahmut II, and it was only when it was destroyed by fife during the reign of Abdulaziz in 1862 that the name seems to have been adopted for all of the Grand Seraglio. An inscription belonging to the original "Topkapi Saray" is now preserved in the Topkapi Palace.
Looking at the Grand Seraglio, we see that it appears to consist of a series of courtyards, four in all, around which are clustered groups of buildings, mainly one- story structures sprawling away from a central square.
Additions over the centuries to the nucleus of the Palace made to accommodate a growing court entourage apparently disregard any conscious plan. Each building bears the decorative features of its respective era, too, and so in all, the Seraglio presents an impressive spectrum of styles, both architectural and decorative, over four centuries, from the 15th to the 19th century.
The most logical way to approach any description of the Palace would be to start at the outermost gate and move inwards through the various courts to the inner sanctum, the sultan's private apartments, the harem and the privy gardens.
Above text and pictures are from the book titled "Topkapi Palace Hardcover".
You can purchase "Topkapi Palace Hardcover" book and other Turkey related books from Explore Turkey Bookstore.