During the 12th century, the Clutinck family steen, or fortress, stood where the Centre for Fine Arts is today. This noble Brussels family controlled the route de Louvain. Their influence, however, gradually waned, in great part due to the arrival of the Duke of Brabant, who took over part of the estate.
In the late 12th century, the often-persecuted Jewish population of Brussels settled at the foot of Duke of Brabant’s castle to benefit from his protection. In 1370, the community was wrongly accused of desecrating eucharistic hosts and was banished from Brabant. Their possessions were confiscated by the duke.
In the 15th century, when the Duchy of Brabant was taken over by the dukes of Burgundy, the city of Brussels began construction of the Aula Magna, a sumptuous reception hall. As successor to the Duke of Burgundy and the most powerful Western Emperor, Charles V established his court there and went on to build a formidable palace during the first half of the 16th century.
On 3 February 1731, a fire raged through the palace. The court moved to the neighbouring Nassau House (on the grounds of the present-day Royal Library of Belgium). The books and manuscripts that escaped the flames were kept at the Domus Isabellae  – later renamed the “Library” – and subsequently moved to Nassau House, which, in the 18th century, became the Palace of Charles of Lorraine. Portions of the ruins of the royal palace were used to create the Place Royale and the neoclassical buildings that surround it.
In the 19th century, the site where the Centre for Fine Arts stands today was home to the Héger boarding school. The Héger name became famous thanks to two of its boarders who went on to become distinguished writers: sisters Emily Brontë, author of Wuthering Heights, and Charlotte Brontë, author of Jane Eyre and, in particular, Villette, which tells the story of her romantic relationship with her professor Constantin Héger. The school remained open until 1908.
 The building, which abutted the palace walls, belonged to infanta Archduchess Isabella of Austria. The old rue d’Isabelle, named after the archduchess, was partly destroyed in 1907-1908 during the construction of the Société Générale’s headquarters, and entirely during the urban renovation of rue Ravenstein.