The site of the house, built 1770-1772, probably faced Hyde Park on the site of Rutland Gate. It was not a large house, suitable for a bachelor. Chambers prepared many designs for houses of varying sizes for Calcraft, who quibbled about paying Chambers and William Collins for their work. Once a design had been chosen, the house was built very rapidly but Calcraft died before the completion, leaving the tradesmen largely unpaid. The house was probably demolished about 1838 when Rutland gate was laid out.

Chambers was born in Sweden and died in London. He travelled widely, visiting China, and studied architecture at the Ecole des Arts, Paris, from 1749 and in Italy from 1750 to 1755. Many of his drawings from this period are contained in his important 'Franco-Italian' album, held in the V&A. Chambers moved to London in 1755 and published his influential Treatise on Civil Architecture in 1759. Chambers demonstrated the breadth of his style in buildings such as Gower (later Carrington) House and Melbourne House, London, in such country houses as Duddingston, Scotland, and in the garden architecture he designed for Wilton House, Wiltshire, and at Kew Gardens. He became head of government building in 1782, and in this capacity built Somerset House, London.