Horace Gray was born in Boston, Massachusetts, on March 24, 1828. His family was wealthy and both parents were socially prominent, although his mother died when Gray was 6 years old. By the time he entered Harvard at the age of 13, he had already reached his full height of 66. At Harvard, Grays studies included natural history with Louis Agassiz, with a special interest in birds; and modern languages. He graduated in 1845 at age 17, in the lower half of his class. After graduation, he took a tour of Europe, but his journey was interrupted with news that the family business had failed, leaving the family penniless. Gray entered Harvard Law School in 1848, this time applying himself earnestly to his studies, and graduated a year later. He continued his legal studies and clerked for a law firm, and was admitted to the bar in 1851.
Gray became successful and well-respected as a lawyer, and , at the age of 26, was the youngest person ever to be appointed to the highly prestigious position of reporter of decisions for the Massachusetts Supreme Court. In 1864, he became an associate justice of the state Supreme Court, then chief justice in 1873, employing as his law clerk Louis Brandeis, a young Harvard student and future justice on the US Supreme Court. In addition to his judicial work, he advised the governor on questions of the law and the constitution, and took an active role in the state historical society. On December 19, 1881, President Chester A. Arthur appointed Gray Associate Justice of the US Supreme Court. Because of his obvious qualifications for the position and his wide base of support, his confirmation, which took place the following day by a 51-5 vote, was celebrated by the press a rare political action free of partisanship or political machine control.
Gray was a strong nationalist and an opponent of slavery. His career as a jurist was marked most strongly, however, by his command of historical legal scholarship. He felt that history and legal precedent were the ultimate determinants of the law. This fascination with history, as well as his other intellectual interests, brought him into contact with many of the finest American minds of his day. He was soon invited to join the "Saturday Club" of Boston, a group of writers and other intellectuals that included individuals such as writers Ralph Waldo Emerson, Oliver Wendell Holmes and Nathaniel Hawthorne and politician Charles Sumner.
A bachelor for his first sixty years, Gray married Jane Matthews, thirty years his junior and the daughter of his colleague Justice Stanley Matthews, in 1889. Grays health had begun to deteriorate by 1894, so he started to scale back his share of the work on the Court in 1896. Failing to recover sufficiently after suffering a stroke in 1902, he resigned from the Court on July 19 of that same year. Gray died on September 15, 1902, before his successor could be chosen. Oliver Wendell Holmes, Jr. was chosen to replace Gray. Ironically, Holmes had also been Grays successor on the Massachusetts Supreme Court over twenty years earlier.