On 16 Nov 1757 Sylvester married Martha Wightman (55416) , daughter of Abraham Wightman (40010) (15 Jul 1711-3 Feb 1800) & Susannah Stark (ca 1715-1813), in Colchester, CT. Born on 16 Apr 1738 in Norwich, CT. Martha died in Colchester, CT, on 15 Jun 1819; she was 81.

On 2 Dec 1757 when Rufus was 19, he married Margaret Wightman (55414) , daughter of Daniel Wightman (40007) (23 Nov 1703-6 Jun 1753) & Katherine Westcott (17 Apr 1705-30 Jan 1753), in Norwich, CT. Born on 24 Aug 1740 in Norwich, CT. Margaret died in German Flats, Herkimer County, New York, on 9 Jul 1813; she was 72.