The Vatican hill is famous today for the great basilica of St Peters, constructed in the third decade of the fourth century by Constantine, and demolished and rebuilt in the 16th century. A collection of essays on this building appeared in 2013, edited by R. McKitterick, which contains various interesting snippets.
Few today are familiar with the layout of the church, so the diagram at the side is useful. A flight of steps led up to a gatehouse, behind which was a courtyard. This later contained the immense bronze pine-cone now in the Vatican museum. Behind this was the church proper, with a nave and two aisles. The transept gave access to two circular structures, the mausoleum of Honorius (which was turned into the chapel of St Petronilla during the early Dark Ages) and the chapel of St Andrew.
Around the church were all sorts of structures, not depicted on this diagram. The church was the constant resort of beggars, seeking alms, and doubtless many of the dwellings were hovels. Theodoric ordered the distribution of grain to them in the late 5th century; Pope Symmachus had shelters constructed for them near the church, and the Dialogues of Gregory the Great record a crippled girl who more or lived in the church until she was healed by a miracle.
A plan of the church by Alfarano, who had been associated with the church since the 1540’s, was published as an etching by Natale Bonifacio in 1590, when construction on the western end of the new basilica was well advanced. It shows the new construction as a ghost under the old.
Tiberio Alfarano drew the plan in 1571, and the hand-drawn original, known as the Ichnographia, is extant in the archive of the Reverenda Fabbrica di San Pietro. Comparison shows that the printed version tinkered with the original in various ways, and that not every architectural feature on the drawing appears in the etching.
On the south side of the basilica were two circular structures, the chapel of St. Petronilla, actually the Mausoleum of the emperor Honorius; and the chapel of St. Andrew. Beyond these was the obelisk which now dominates St Peter’s square.
The function of the structure as a mausoleum was remembered as late as the 8th century, but thereafter forgotten until 1458 when a splendid late Roman burial was discovered under the floor, possibly of Galla Placidia and her child. Another was found in 1519, and finally in 1544 the intact sarcophagus of the empress Maria, wife of Honorius, complete with 180 precious objects in two silver chests, all of which were dispersed or melted down. The depiction of the basilica in the Nuremberg chronicle of 1493 depicts a round, squat building, which was doubtless the mausoleum.
The structure to the east of it, labelled “Vatican rotunda” in the plan, must predate the basilica as it appears in a gem of the 3rd century. It was converted by Pope Symmachus in the 5th century into a chapel of St Andrew.
I have also seen a paper suggesting that the “mausoleum of Honorius” was itself a 3rd century tomb, as was the Rotunda di Sant’Andrea. The mausoleum was demolished during the building of New St Peter’s, but the Rotunda remained until the 18th century, becoming the church of Santa Maria della Febbre. A 1629 painting of it, still behind the obelisk (which was surely moved by then?) and with New St Peter’s half-built behind it is available online:
The obelisk is an interesting feature, since it is quite unlikely that it was placed there by Constantine. We learn from Pliny’s Natural History that Caligula erected an obelisk from Heliopolis on the spina of his Circus, in the Horti Agrippinae on the Vatican. There is apparently consensus, among interested scholars, that the only certain fact about the location and orientation of the circus is that this obelisk was in the centre of it..
Two different circus plans appear online. I don’t know the source of the second one.
What can be said with certainty is that material from the circus was found during excavations in St Peter’s square, some 5 metres down.
Somewhere nearby, in all this, is the temple of Cybele and Attis, the Vatican Phrygianum. That such a temple existed in 160 A.D. is recorded by an inscription from Lyons which reads:
Taurobolio Matris d(eum) m(agnae) I(daeae) / quod factum est ex imperio ma tris deum /pro salute imperatoris Caes(aris) T(iti) Aeli Hadriani Antonini Aug(usti) Pii p(atris) p(atriae) / liberorum eius /et status coloniae Lugdun(ensium) / L(ucius) Aemilius Carpus IIIIIIvr Aug(ustalis) item / dendrophorus / uires excepit et a Vaticano trans/tulit ara(m) et bucranium /suo inpendio consacrauit / sacerdote / Q(uinto) Samnio Secondo ab XVuiris /occabo et corona exornato / cui sanctissimus ordo Lugdunens(ium) perpetuitatem sacerdoti(i) decreuit / App(io) Annio Atilio Bradua T(ito) Clod(io) Vibo / Varo co(n)s(ulibus). 
Various inscriptions from the end of the 4th century consist of dedications to Cybele by the last holdouts of the pagan aristocracy, suggesting that perhaps the temple was still in use in this period, and recording that the ritual of the taurobolium – being bathed in bulls’ blood – was taking place here.
Pensabene states that the 1959-60 excavations by Castagnoli – I don’t have a reference for these – revealed that there were major works in this area during the Severan period. The ground level was artificially raised by several metres and a large circular building was constructed whose foundations were contiguous with the obelisk. The foundations of this building contained Severan stamps from the first quarter of the 3rd century A.D. The suggestion is that this was to allow the building of a new Phrygianum, and that this was done under Elagabalus, who was enthusiastic for the cult.
The text is accompanied with a very poor quality image which appears to suggest that the Rotonda di Sant’Andrea stands on the site of the Phrygianum, and that the building was originally circular, with a south-facing portico:
My Italian is not good enough to work out whether Pensabene is suggesting that the Rotondo was, in fact, the carcase of the Phrygianum, stripped of its portico and reused for something else. But if so, this would certainly be very cramped, next to the basilica, and the presence of the vile eunuch priests and their revolting sacrifices right by the south door sounds rather unlikely to me. Even if it was a state cult, which Constantine might have been unwilling to interfere with, this seems improbable.
So where was the Phrygianum, if not here?
-  R. McKitterick, Old Saint Peters, (British School at Rome Studies), 2013. “Look Inside” on Amazon here. ↩
-  For these details I am indebted to Paulo Liverani’s paper “St Peter’s and the City of Rome” in the McKitterick volume, of which I was able to read parts via the Amazon “Look Inside”. The material may be found on p.26; Gregory, Dialogues I, 3.25.1, 108; Life of Symmachus, 53, c. 7, in the Liber Pontificalis I 262; Theodoric in Procopius, Anecdota 26.29. ↩
-  These details appear in the front matter of the McKitterick book, whose footnotes were sadly inaccessible to me. ↩
-  Meaghan McEvoy, “Chapter 6: The mausoleum of Honorius” in: R. McKitterick &c., p.119 f. Accessible via Google Books preview here. ↩
-  Via Wikimedia Commons ↩
-  See Italian Wikipedia article here. ↩
-  Plin. NH XVI.201; XXXVI.74; CIL VI.882 = 31191. All these references I owe to a remarkable discussion in the Ancient Coins forum here. ↩
-  Patrizio Pensabene, “Culto di Cibele e Attis tra Palatino e Vaticano”, Bollettino di Archeologia 2010, Online at http://www.archeologia.beniculturali.it/pages/pubblicazioni.html; except that, at the time of writing, this is offline and I was only able to access the article via the Google cache. UPDATE: Later I found it at Academia.edu here. ↩
-  The first is from R. Lanciani, Pagan and Christian Rome, 1892, and appears on Wikimedia Commons here. Both have been copied from here. ↩
-  Or so it claims on this website; it would be interesting to have proper details of these excavations. ↩
-  CIL XIII, 1751. ↩