Youngest son of Charles Stanhope, 3rd Earl Stanhope, Lt Col James Hamilton Stanhope (1788–1825) joined the British Army aged just fifteen by the advice of William Pitt the Younger (the 3rd Earl's second cousin). He served in Spain, Portugal, Flanders and France, acting as aide-de-camp to General Sir John Moore and Lord Lynedoch, before being appointed Assistant Quartermaster General in the Peninsula in 1813. He sustained a grape-shot wound in the spine at the storming of San Sebastian in 1813, which was to cause him immense suffering for the rest of his life. Nevertheless, he remained in the army, and in 1815 he served as an assistant adjacent to the Duke of Wellington during the Waterloo Campaign. He took part in the Battle of Waterloo and the subsequent march upon Paris.