After burning down in the fires of the Battle of Ichinotani, the temple was rebuilt in 1188 by order of Minamoto no Yoritomo, carried out by Kumagai Naozane and Kajiwara Kagetoki.

Katsuo-ji became renowned as the "Temple of Victory Fortune" after Priest Gyojun's prayers reportedly cured Emperor Seiwa's illness. Warlords including the Minamoto and Ashikaga clans revered it, and Toyotomi Hideyoshi—who later unified Japan—is also said to have devoutly worshipped there as a place to pray for victory in battle.

In the 2nd month of 1184 (Genryaku 1), Katsuo-ji lost its halls to the conflagration of the Battle of Ichi-no-Tani, ignited by Minamoto no Yoshitsune's surprise assault on the Heike. For Yoritomo, restoring war-damaged ancient temples in western Japan was a key project to assert his legitimacy as warrior leader, and in 1188 (Bunji 4) he ordered the reconstruction of Katsuo-ji, appointing Kumagai Naozane and Kajiwara Kagetoki as joint magistrates of the works. By this date Kagetoki already served as Samurai-dokoro shoshi (deputy commissioner of the Board of Retainers), overseeing the eastern gokenin — being entrusted with rebuilding a great monastic complex in distant Settsu Province testifies to Yoritomo's confidence in him as both administrator and treasurer. The Azuma Kagami records this work as part of 'the restoration of temples and shrines in the western provinces,' with Kagetoki and Naozane managing fundraising, materials procurement, and the supervision of master carpenters. The reconstruction of Katsuo-ji thus reveals a side of Kagetoki often obscured by his reputation as a warrior — that of the administrative officer who underpinned the cultural and religious policy of the Minamoto regime. Portions of the Kamakura-period restoration survive in the precincts, and from this era onward Katsuo-ji's halls drew samurai devotion as the renowned 'Temple of Victory.'