<P> As late as 1820, the community had dwindled to 91 families, a church, a school, and a library . Wealthy farmers, known as "patroons", maintained these country estates largely on the heights overlooking the Hudson River . Service connecting the outlays of Harlem with the rest of the City of New York (on the southern tip of the island of Manhattan) was done via steamboat on the East River, an hour - and - a-half passage, sometimes interrupted when the river froze in winter, or else by stagecoach along the Boston Post Road, which descended from McGown's Pass (now in Central Park) and skirted the salt marshes around 110th Street, to pass through Harlem . </P> <P> The New York and Harlem Railroad (now Metro North) was incorporated in 1831 to better link the city with Harlem and Westchester County, starting at a depot at East 23rd Street, and extending 127 miles (204 km) north to a railroad junction in Columbia County at Chatham, New York by 1851 . Charles Henry Hall, a wealthy lawyer and land speculator, recognized the changes that this railroad would make possible in Harlem and began a successful program of infrastructure development, building out streets, gas lines, sewer lines, and other facilities needed for urban life . Piers were also built, enabling Harlem to become an industrial suburb serving New York City . The rapid development of infrastructure enabled some to become wealthy, and the area became important to politicians, many of whom lived in Harlem . New York mayors Cornelius Van Wyck Lawrence and Daniel Tiemann both lived in Harlem in this period . For many in New York City, Harlem was at this time regarded as a sort of country retreat . The village had a population of poorer residents as well, including blacks, who came north to work in factories or to take advantage of relatively low rents . </P> <P> Between 1850 and 1870, many large estates, including Hamilton Grange, the estate of Alexander Hamilton, were auctioned off as the fertile soil was depleted and crop yields fell . Some of the land became occupied by Irish squatters, whose presence further depressed property values . </P> <P> During the American Civil War, Harlem saw draft riots, along with the rest of the city, but the neighborhood was a significant beneficiary of the economic boom that followed the end of the war, starting in 1868 . The neighborhood continued to serve as a refuge for New Yorkers, but increasingly those coming north were poor and Jewish or Italian . Factories, homes, churches, and retail buildings were built at great speed . The Panic of 1873 caused Harlem property values to drop 80%, and gave the City of New York the opportunity to annex the troubled community as far north as 155th Street . </P>

What did the new york city neighborhood of harlem become during the early years of the 20th century