The fortified town of Cakovec was first mentioned in documents in 1333. The town have been founded on the site on which Count Dimitrije Cak, the ruler of Medjimurje, built a wooden fortress at the end of the 13th century.
In the course of the 14th and the 15th centuries, the Cakovec Fortress was reconstructed by the Lackovic ducal family, the Counts of Celje, and the Ernust family.
Following the earthquake of 1738, the Czech Counts of Althan, who were then the lords of Medjimurje, restored the palace and the fortress according the design of the court architect Anton Erhard Martinelli. The palace was rebuilt in the baroque style and the Renaissance tower pulled down, while a new one was erected on the palace fortifications.
The new, imposing two-storey baroque palace with a courtyard has preserved its original appearance to the present date.
When in 1546 the Zrinski Counts acqiured possession of Medjimurje, Governor Nikola Zrinski rebuilt the Cakovec Castrum into a strong Renaissance fortress.
The fortress surrounded by water ditches was linked to the land by a long wooden bridge.