On 16 Jun 1743 when Elisha was 31, he first married Lucy Huntington in Norwich, CT. Lucy died in Franklin, CT on 12 Oct 1751.

On 16 Apr 1754 when Elisha was 41, he second married Elizabeth Dorr (27402) , daughter of Edmund Dorr (16924) (19 Oct 1692-ca 1776) & Mary Griswold (11102) (22 Apr 1694-21 Feb 1776). Born on 16 Apr 1735 in Lyme, CT. Elizabeth died in Norwich, CT, on 23 Nov 1781; she was 46.

Children of Elizabeth and Elisha (Dorr) Tracy:
    i. Phineas Tracy, b. 29 Jun 1755;
    ii. Philemon Tracy, b. 30 May 1757;
    iii. Elizabeth Tracy, b. 29 Jun 1760, 30 Oct 1773;
    iv. Charlotte Tracy, b. 27 May 1762;
    v. Mary Tracy, b. 3 May 1764;
    vi. Elisha Tracy, b. 11 Aug 1769, d. 6 Jul 1770; and
    vii. Deborah Dorr Tracy, b. 9 Nov 1770.