Joseph Swift Whistler was the son of George Washington Whistler and his first wife, Mary. He was thus Whistler's half-brother. He died at an early age.
Mumford, Elizabeth, Whistler's Mother: the Life of Anna McNeill Whistler, Boston, (Mass.), 1939; McDiarmid, Kate R., Whistler's Mother: Her Life, Letters and Journal, [n.p.], [n.d.].