Born in 1829 as the sixth son of Matsumae Akihiro, ninth lord of the Matsumae domain, at Fukuyama Castle. He inherited the domain in 1849 as its twelfth lord and was that same year ordered by the shogunate to build a castle to strengthen the defense of Japan's northern coast. Commissioning military scholar Ichikawa Ichigaku the following year, he completed Matsumae Castle (also called Fukuyama Castle) in 1854 — the last traditional Japanese-style castle ever built. He went on to an unusually rapid ascent within the shogunate, serving as magistrate of temples and shrines from 1863 and being promoted to senior councilor (roju) in 1864. In 1865, his unilateral decision to open the port of Hyogo drew imperial censure and led to his downfall. He died of fever at Matsumae in 1866, aged 38, never living to see the Boshin War of 1868 — in which the very castle he had built would fall to former shogunate forces under Hijikata Toshizo just two years after his death.

Though merely the lord of an outlying domain, he rose to become one of the shogunate's senior councilors — one of the most remarkable career ascents of the Bakumatsu era. Keenly attuned to the need for northern coastal defense, he was also a capable administrator who built Japan's last traditional-style castle at the shogunate's order.

Matsumae Castle, which Takahiro built, survives today as Hokkaido's only traditional Japanese-style castle and is renowned for its cherry blossoms. Hekirichi Bastion Fort, also built at his order, was Japan's first Western-style star-shaped fortification and is now a National Historic Site.