Castle Hill is a neighborhood located in the southeast section of the borough of the Bronx in New York City. Its boundaries are Waterbury Avenue and Westchester Avenue to the north, Westchester Creek to the east, the East River to the south, and White Plains Road to the west. Unionport is a subsection of Castle Hill, typically considered north of …Castle Hill is a neighborhood located in the southeast section of the borough of the Bronx in New York City. Its boundaries are Waterbury Avenue and Westchester Avenue to the north, Westchester Creek to the east, the East River to the south, and White Plains Road to the west. Unionport is a subsection of Castle Hill, typically considered north of Lafayette Avenue.