Gossaert has used classical architectural details that he'd seen on a trip to Italy, such as rounded arches and pillars topped with decorative capitals.
This both gives his ruins a believably ancient, pre-Christian character, and also allows him to show off his well-travelled status.
He's used the complexity of this fantasy ruin to create hiding places for some of the figures.
The man in red who hovers rather hesitantly in an archway behind Mary is Joseph, shown as a husband who has been somewhat sidelined by recent events.
To his right, we catch a glimpse of another man looking through the doorway above the ox - this may be a portrait of the artist.
