On 11 Jul 1784 James married Molly Whittemore (57707) , daughter of Lieut. John Whittemore (46568) (ca 1721-17 Jul 1809) & Elizabeth Earle (45669) (18 Oct 1731-6 Aug 1812), in Paxton, MA. Born on 28 Nov 1753 in Leicester, MA. Molly died in Hardwick, Worcester County, Massachusetts, on 15 Feb 1825; she was 71.