Source: Ephemeral NY The Vanderbilt family built a number of mansions on Fifth Avenue in the 1880s. The home pictured here, at 57th Street and Fifth Avenue, belonged to Cornelius Vanderbilt II.