Byng married, first, Florence Miles (d.1862) and secondly, in 1866, Emily Kerr.
He graduated from Christ Church, in Oxford 1857 and was ordained in 1858. He started ass Rector of Little Casterton, Rutland, from 1859-62, when he moved to London. He held numerous prestigious positions including that of Chaplain to the Queen 1867-72, Chaplain to the Speaker of the House of Commons, 1874-89 and and Grand Chaplain of Freemasonry in England 1889. He was also the Vicar of St Peter, Cranley Gardens, South Kensington, and it was probably as the local Vicar and a welcome visitor in Kensington social circles that he was known to Whistler in the late 1880s.
Burke, E., The Peerage and Baronetage of Great Britain and Ireland.