Before he was known in Chicago, Joseph Silsbee's work had gained notoriety across New York State. His notoriety spread east in the late 1870's with a series of commissions around the state's capitol. The largest and most notable of these was the Madison Avenue Reformed Church, also known as the Second Dutch Reformed Church. The cornerstone was laid on the Romanesque structure in July of 1879 and was complete, except for it's tower, in 1881.
The church was demolished after a fire in the 1930's and only exterior photographs and images are readily available. Newspaper accounts recall a highly colored vestibule with complimentary brick, tile and art glass with coordinated colors throughout the church. Carved cherry wainscoting surrounded the main auditorium area, a large open space free of columns. The seats were arranged in a horseshoe shape. In the rear of the church, meeting and school rooms were located.
The exterior of the church was made of grey limestone with granite trim and columns. Weather vane, foundation stones and pulpit were some of the materials that were moved from the old Dutch church to this structure when it was constructed. When designed, the bell tower was to be the tallest point in the city of Albany at the time. Silsbee earned the commission partly on the strength of his design for Syracuse's First Reformed Church, completed a year earlier, and partly through a reference from good friend and influential president of Cornell University, Andrew Dickson White.