Everything changed for Milford Station when war came to Spotsylvania Court House in May 1864. As the forward supply depot for the Army of Northern Virginia, the small collection of houses and lone depot saw a constant stream of trains, wagons, and hungry soldiers. On the morning of May 21, 400 members of General James L. Kemper's brigade were here preparing to march to Spotsylvania when rumors of a Union advance began to filter through the ranks.
The Union cavalry were the lead element of Gen. Winfield S. Hancock's advance column, arriving after their night march from Massaponax Church. Not knowing that almost 20,000 Federals were bearing down on them, the few Virginians gamely decided to defend the depot and nearby bridge over the Mattaponi River. Major George F. Norton of the 1st Virginia took command of the varied band of Confederates in the town. He dispatched members of the 11th Virginia to a small hill behind the station where they took cover in the foundation of an old ice house. Union cavalry approached the position, and the sounds of gunfire echoed over the station. Detachments from the 5th New York Cavalry, 1st Maine Cavalry, and 1st Pennsylvania Cavalry encircled the hill and soon overran Milford Station, but the Confederates on the hill continued to fight.
Soon ammunition began to run low. Finding that the rest of Kempe's men had escaped across the river and destroyed the bridge, the small band of Confederates knew they were trapped. "It then dawned upon us that we had been sacrificed to save the troops across the river," one Southerner recalled. "Good generalship, I suppose, but 'tough on the frogs.'"
Hancock's men captured Milford Station, along with 68 Confederates, but the news of enemy reinforcements caused Hancock to reconsider any further movement south. He repaired the Mattaponi River bridge and moved to the high ground beyond to entrench, should Gen. Robert E. Lee decide to come in search of him.