The park was formally created in 1895 by the newly-created Essex County Parks Commission, making it the nation's first county park. The area had formally served as an Army training ground during the American Civil War. At the time, the northern portion of the area had been a marsh known as Old Blue Jay Swamp. In 1898, a public appropriation financed the conversion of the swamp into a landscaped lake. The initial park was only 60 acres (0.2 km²) in size but grew in the 1920s through private donations from prominent Newark families, such as the Ballantines, eventually reaching the city limit with Belleville and becoming one of the largest urban parks in the United States. The Morris Canal originally ran on the park's west side, until its old bed was turned into the Newark City Subway, providing access to the park from Downtown Newark.

The first designs of the park, based largely on romantic garden themes, were proposed in 1895 and 1898, after the Parks Commission hired several architectural firms to plan the park. In 1900, the commission hired the firm of Frederick Law Olmsted to redesign the park. The result was the park's current naturalistic look and feel, with acres of meadows and forests, in a manner similar to Olmsted's earlier designs of Central Park and Prospect Park.

The famous cherry trees were the result of a 1927 gift from Caroline Bamberger Fuld, sister of department store magnate Louis Bamberger and widow of the store's vice president. The Cherry Blossom Festival attracts approximately 10,000 visitors each April.

During World War II, the park's grounds served a tent city for recruits, as well as a landing strip for airplanes of the United States Postal Service.

The neighborhood on the east side of the park, Forest Hill, is Newark's most affluent. Also on the east side of the park is the Cathedral of the Sacred Heart Basilica, one of the largest cathedrals in the United States.

It has been placed on both the New Jersey (1980) and National (1981) Registers of Historic Places.

Branch Brook Park is currently in the midst of $25 million, ten-year, restoration program. In 2004, the Park Avenue bridge was repaired, as were the ballfields in the center of the park. By 2010, there will be more than 5,000 cherry trees in the park due to a $650,000 grant from the Essex County Recreation and Open Space Trust Fund in 2006 and private donations.

In July 2010, as reported in The New York Times and the Newark Star-Ledger, a man was fatally shot in the park by an undercover police officer. The shooting occurred near where Bloomfield Avenue crosses Branch Brook Park during a police sweep of an area known for years as a place where men were looking to have sex with other men. Residents had complained about sexual acts in public, and the police made more than 200 arrests in the area within the last 18 months. The incident remains under investigation.