The Sardar Palace, also known as the Khan's Palace—one of the masterpieces of Azerbaijani architecture—was constructed in 1578 by Tokhmaq Khan, the beylerbey (governor-general) of Iravan. It was situated opposite the fortress, on the right bank of the Zangi River, where a large garden was also established.

The Mirror Hall and the Summer Mansion within the Khan's Garden were added by Muhammad Khan Qajar in 1791, completing the palace complex. In 1810, under the rule of Huseynqulu Khan, extensive restoration and new construction work were undertaken. The wall paintings in the Sardar Palace were finished in 1815 by Mir Abdurrza Khan, a member of the Tabriz painting school. After the Russian occupation of Iravan in 1827, the palace was badly damaged. 

A plan from 1837 shows the palace complex covered about one hectare. It was restored again in the mid-1800s, with major restoration work carried out repeatedly between 1867 and 1874 by Mirza Gadim Iravani. 

From the early 20th century, the palace rapidly fell into decline and was completely demolished in 1914. Its portraits were removed and sent to Georgia, where they are now kept at the Georgian National Museum. Later, a brandy factory was built on the site of the palace.