In the early Kamakura period, Retired Emperor Go-Toba constructed the Minase Detached Palace at this site and gathered poets such as Fujiwara no Teika and Jakuren to enjoy the elegant pursuits of waka, including the compilation of the Shinkokin Wakashu. After his defeat in the 1221 Jokyu War and exile to Oki, his deep affection for Minase remained — his Oki-period poem "I now am the new island guardian — wild waves and wind of the Oki sea, blow with mindfulness!" conveys his isolated state of mind. After his death on Oki in 1239, his close attendants built a Mieido hall on the site of the Minase Palace, the origin of the present shrine.

Fujiwara no Teika (1162-1241), as the waka mentor of Retired Emperor Go-Toba, was frequently invited to poetry gatherings at the Minase Palace and served as one of the compilers of the Shinkokin Wakashu. Although his relationship with Go-Toba was once close, his views on waka gradually diverged from the emperor's, and they grew distant around the time of the Jokyu War. Even so, Minase remained a symbolic site of medieval poetic refinement, continuing to influence Teika's poetic thought.

In the fifth month of 1221 (Jokyu 3), Retired Emperor Go-Toba issued a declaration calling for the subjugation of Hojo Yoshitoki and raised troops (the Jokyu War). Yoshitoki (1163–1224), second regent of the Kamakura shogunate and the brother of Masako, held effective control of the shogunate. After Masako rallied the gokenin warriors by invoking Yoritomo's benevolence, Yoshitoki dispatched his son Yasutoki and Tokifusa at the head of what is said to have been an army of 190,000 men marching west. Within about a month they had annihilated the court forces and sent Go-Toba into exile on Oki Island. Minase was the beloved retreat palace of the exiled emperor: this shrine preserves the irony of history — that the very war led by Yoshitoki turned the place Go-Toba cherished most into the moment his exile began.