The construction prestigious building complex, residence of the noble family Ferretti, home of the Museum since 1958, dates back to the mid-sixteenth century. The external façade is attributed to Antonio da San Gallo the Younger, civil and military engineer active in Ancona in the 40s in 1500.
The decoration of the main floor, commissioned by Count Girolamo Angelo Ferretti, was built the Mannerist painter Pellegrino Tibaldi and little aid after 1560; wooden ceilings, the walls of the main hall were painted and different environments were built elaborate connected to it: the emblems of the room, the myths of the room, the room of the maze. The 3rd floor has frescoed ceilings with grotesque, fantastic landscapes, monsters, mythological figures and masks in the late sixteenth Roman style (Federico Zuccari, Luzio Luzi?).
In 1759 the original building was restored, probably designed by Luigi Vanvitelli, with the expansion of the façade, the construction of the balcony, the central door, the staircase and the roof terrace, porch and overlying lodges. This expansion was decorated with statues and busts of Ancona probably shop Varlè Joachim (1731-1806). Always the '700 times the refined grotesque of some living rooms of the 3rd floor.
The building has undergone several restorations: in the years 1928 to 1931 on behalf of the Ferretti family who entrusted the work to 'Tornaghi Milanese architect; in the early 50s, as a result of air strikes; Finally in the 70s and 80s, anconitano after the 1972 earthquake.