Born in Kagoshima to a Satsuma samurai family, he was a cousin of Saigo Takamori. In the Boshin War he fought as part of the imperial forces, particularly distinguishing himself in the siege of Aizu-Wakamatsu Castle. After the Meiji Restoration he studied in France and Switzerland, acquiring modern military technology and tactics. He observed the Franco-Prussian War (1870-71) firsthand, learning the realities of modern warfare at close range. After returning home he worked to develop and modernize the army, strengthening the foundations of the Imperial Japanese Army as Army Minister. In the First Sino-Japanese War (1894-95) he served as commander of the Second Army. In the Russo-Japanese War (1904-05) he served as Commander-in-Chief of the Manchurian Army, commanding Nogi Maresuke, Kuroki Tamemoto, and others, and led the army to victory at the Battle of Mukden. Though he rose to the rank of field marshal, his gentle character and broad perspective earned him the honorific title taijin (great man). His wife Suematsu (Yamakawa Suematsu) was an accomplished woman who had studied in the United States and was also known as Japan's first female tennis player. He died at seventy-four on December 10, 1916.

A gentle commander with excellent strategic perspective. Rather than a flamboyant personality, his greatness lay in his magnanimity in trusting and delegating to subordinates. Though Saigo's cousin, he made the painful choice to stand with the government forces in the Satsuma Rebellion.

He was hailed as a hero by the Japanese people as a key contributor to victory in the Russo-Japanese War. His enormous contribution to modernizing the Imperial Japanese Army, and the role he played in introducing the French military system, are still highly regarded today.