It was one of the five castles built by the Venetians in the 15th century. The other ones were those of Akrotiri, Skaros, Aghios Nikolaos in Oia and Emporio.
The Kasteli followed the style of a settlement fortification with a subsequent yard: around the core, in the centre of which there was a tower initially and a church at a later stage, a fortification yard was later developed, consisting of the external walls of houses which were almost contiguous, very thick and high, and had a few air openings –a practice followed since antiquity. According to D. Denaxas, there was a tower in the castle, which was completely demolished in 1735 to give its place to the church of Aghios Georgios.
The outer contour of the castle was irregular and polygonal. In its centre, stood the small square with the churches of Panagia and Aghios (Saint) Georgios. A cobbled road separates the castle from the buildings of the contemporary part of the settlement. The oldest historical evidence about the castle is found in a work by M. Crucius entitled "Turcograecia". There, it is referred to as Pyrgos, "Kainourgios Pyrgos", "Kasteli Kainourgiou Pyrgou", or "Kainourgiompourgo" (meaning new castle). This indicates that it was built after at least one of the four other castles. The castles of Skaros and Akrotiri were most likely built before it.
If one looks closely at the way the houses in the castle of Pyrgos are arranged, one can understand that it was very similar to the one of the castle of Skaros, in Imerovigli, during the last decades of the 18th century. The houses were narrow and elongated, with a common dividing wall on which a cylindrical vault was based. They had two or three stories, with the bottom ones being partly built and partly caved. The only entrance of the castle to the western part of the perimeter remained closed at night if there was danger. Above the entrance, there was a murder-hole called "loophole" or "murderess".
An important milestone in the history of the castle –as well as the rest of the island- was the earthquake of 1956. The buildings suffered severe damage and later on, its form was defaced by interventions conducted by state authorities.
- According to Antonis Kontaratos and the book "Thera, a march through time"/Heliotopos Publications, in the castle of Pyrgos seventy independent residences of horizontal ownership were found each with its own cistern and toilet in which equal number of families lived.
- According to tradition, the castle was connected to the outer space from the side of Exo Gonia through a secret tunnel with both accesses to it buried under the ruins of the 1956 earthquake. In 1996 the community of Pyrgos tried to excavate the tunnel. They discovered the entrance, but the project didn't continue due to technical problems.
Sources: A study on the castle of Pyrgos, by architect Clairi Palyvou / Santorini, Society and Shelter, 15th-20th century), by Dr Dora Monioudi Gavala/A publication of Loukas and Evangelos Belonias Foundation.