Ca 1752 James married Martha Doty (25640) , daughter of Benjamin Doty (10068) (30 May 1689-bef 25 Jun 1770) & Hester Beamont (11259) (ca 1690-bef 19 May 1773). Born ca 1733 in Saybrook, CT. Martha was baptized in Westbrook, CT, on 28 Apr 1733.

Children of James and Martha (Doty) Denison, born at Lyme include:
    i. James Denison, b. ca. 1753;
    ii. William Denison, b. ca. 1756;
    iii. Ashbel Denison, b. ca 1764, m. Lurana (d. 26 Mar 1825 at East Haddam, CT); and
    iv. Doty Denison, b. ca. 1770.