She was the daughter of Foster Swift and Deborah Delano. In 1821, she married George Washington Whistler by whom she had three children: Deborah Delano (1825-1909); George William (1822-1869); Joseph Swift (1824-1840).
Her brothers Joseph Gardner and William Henry Swift had both attended the USMA, West Point, NY where they met George Washington Whistler and William Gibbs McNeill (AMW's brother). AMW was friends with Mary R. Whistler, and years after Mary's death she married George Washington Whistler.
Elizabeth Mumford, Whistler's Mother, The Life of Anna McNeill Whistler, Boston, 1939, pp. 20-21.