Alfred Howe Terry did not receive any formal military training before the outbreak of the Civil War.  He spent some time studying at Yale Law School, and was admitted to the bar of Connecticut in 1849, but withdrew from the school. 

At the outbreak of the Civil War, he helped raise and became colonel of the 2nd Connecticut, a ninety-day militia regiment.  He commanded them through the First Battle of Bull Run, after which he helped to recruit the 7th Connecticut. Terry and the 7th Connecticut were then transferred to South Carolina.

While in South Carolina, he commanded troops during the capture of Port Royal, as well as during the Siege of Fort Pulaski.  He was promoted to brigadier general on April 26, 1862, and placed in command of a division in the Tenth Corps.  Terry and his men were involved in a siege on Charleston, South Carolina as well as the siege at Morris Island.  In September of 1863, Terry and his men succeeded in capturing Fort Wagner. Following his success in South Carolina Terry and his division of the Tenth Corps were transferred into the Army of the James under Benjamin Butler.  He commanded his troops through the Bermuda Hundred Campaign around Richmond, and took part in the Siege of Petersburg. 

Terry is most known for his role in capturing Fort Fisher, North Carolina.  Although General Benjamin Butler had attempted and failed to capture the fort, Terry worked together with the Union Navy and on January 13, 1865, successfully captured the fort, sealing off one of the Confederacy's last ports.  He was promoted to major general on January 16, 1865, for his actions, after which he took part in the conclusion of the Carolina Campaign. 

After the war, he continued to serve in the army, becoming one of the only non-graduates of the United States Military Academy to reach the rank of general in the regular army.