Mengelberg was born in 1935 in Kiev to musical parents -- his mother was a harpist, his father a well-known pianist and conductor -- but his family immigrated to Amsterdam when the political climate around them became hostile toward his outspoken, activist parents. Mengelberg began playing both chess, his favorite game, and the piano, well before the age of ten. His first years on the piano found him improvising, then learning boogie-woogie. He heard his earliest major jazz influences in the late '40s: Charlie Parker and Thelonious Monk, who was not highly regarded by many of the musicians in Mengelberg's community at the time, but whom Mengelberg considered to be the most harmonically interesting jazz musician he had yet heard.

Misha Mengelberg, son of notoriously stringent conductor Karel Mengelberg (nephew of Willem), gave up studying architecture after a brief time to attend the Royal Conservatory (in the Hague) and focus his efforts on music, namely composition and classical music. At the conservatory, he studied under, among others, his father's friend Kees van Baaren. Mengelberg was incapable of playing the piano fast (something he has claimed hasn't changed, he simply adapted such limitations into his own style) and his compositions during that time were conceptual, and certainly experimental, which failed to impress his instructors. The young pianist also came into contact with John Cage and his music during this time, while attending a program at Darmstadt. This further opened his mind to a different, modern, more experimental kind of approach to composition, which Mengelberg did not completely abandon even when changing his musical focus to jazz; his compositions kept an ironic stance. For instance, in the early '70s, he created a piece with the translated title of "With the Very Polite Greetings of the Camel." The performance of this work involved Mengelberg sawing a chair into the shape of a camel accompanied, of course, by an orchestra.

Two future collaborators, Han Bennink and (saxophonist and Kollektief leader) Willem Breuker, first heard Misha Mengelberg play around the time that he won the 1959 jazz competition at Loosdrecht. On a personal note, it would be just a few years later that Mengelberg would meet his life partner Amy Chattelin while in the Hague. Around the same time at the Hague, Mengelberg first heard saxophonist Piet Noordijk play. The early '60s brought a quartet with Bennink and Noordijk, which had a few different bassists (including Gary Peacock), and a trio that backed Johnny Griffin in 1963 and, the following May, backed Eric Dolphy for a few gigs, including the concert that was released as the album Last Date.

Mengelberg's first album as a leader, The Misja Mengelberg Quartet As Heard at the Newport Jazz Festival 1966, was recorded during his first trip to the U.S. Around that time and back home, Mengelberg won the Wessel Ilcken Prize. He also judged a local jazz competition in which he heard the young saxophonist Willem Breuker for the first time. Later that decade, Mengelberg would join in on the start of the label and group concept ICP begun by the now-collaborating Bennink and Breuker. ICP stood for "Instant Composers Pool," referring to a phrase and style coined by Mengelberg: "instant composing." The name ICP quickly became an umbrella for a wide variety of lineups, and remained such during its first decade of existence. It wasn't too long before these groups included either Breuker or Mengelberg but not both, as the two musicians had clashing opinions on approaches to live performance, what ICP should be and many other musical issues. Breuker wanted tunes and rehearsals, Mengelberg wanted instant composing. Breuker wanted more people admitted who would have equal voting rights, Mengelberg wanted the core three members to have final say. Both also had different takes on music theater, in which ICP got involved during the late '60s. Mengelberg was also not as driven to perform as often as either Breuker or Bennink. So, both Mengelberg and Breuker led their own ICP gigs, with Bennink (who didn't choose sides) performing in both, although more often with Mengelberg.

Mengelberg and Bennink had a trio with the frequently visiting British saxophonist Evan Parker beginning in 1969, followed by a quartet with reedsman John Tchicai and guitarist Derek Bailey, which toured briefly and recorded two albums between 1970 and 1971. Months before this, the core duo of Parker and Bailey were joined by saxophonist Peter Brotzmann, trombonist Paul Rutherford, and saxophonist Peter Bennink (Han's brother) for what became the septet album Groupcomposing. During this time, Mengelberg and Bennink also recorded an untitled duo for the ICP label. Breuker and Mengelberg were also key instigators in the overhaul in Dutch government funding of jazz, resulting in regular grants somewhat controlled by the musicians in an organization called BIM and a space for the musicians to use that was dubbed the BIMHuis. Mengelberg ended up president of BIM and, by 1972, Mengelberg was also the artistic director of STEIM. The Studio for Electro-Instrumental Music was begun with a few other people including pianist Louis Andriessen (son of modern classical composer Hendrik Andriessen), a friend of Mengelberg's since his days at the conservatory. Some musicians pooled the equipment they had managed to buy so far with grant money and, through STEIM, tried to educate interested persons. During the early '70s, Mengelberg held a course to teach musicians how to utilize electronics. He remained director of the organization until the end of the decade. Not too long after his departure from STEIM, he also stepped down as BIM president. His grass-roots efforts in music education continued, however, and he has held a weekly open-door composition workshop (for anyone who wants to show up) at a conservatory.

The early '70s also brought the inevitable splintering of ICP, and musicians went with either Mengelberg or Breuker, ultimately resulting in the Willem Breuker Kollektief and the ICP Orchestra. The latter group had a continuously changing lineup for several years, during which only Mengelberg, Han Bennink, and American tuba player Larry Fishkind were mainstays. Some of the rotating members included Brotzmann, John Tchicai, cellist Tristan Honsinger, and more before the group stabilized years later with the personnel of Mengelberg, Bennink, trombonist Wolter Wierbos, reedsman Michael Moore, bassist Ernst Glerum, cellist Ernst Reijseger, saxophonist Ab Baars, trumpeter Thomas Heberer, and, off and on, cellist Tristan Honsinger. Due to the larger roster, the "instant composing" tenet shifted slightly to "conducted improvisation," with Mengelberg conducting from the piano. The group's recordings came relatively few and far between, but include ICP -- Tentet in Berlin (SAJ, 1978), Japan Japon (DIW, 1982), and Jubilee Varia (Hatology, 1999). In addition to this, Mengelberg has paid homage to Thelonious Monk and Herbie Nichols with this group, and has led his own releases on several prestigious avant-garde jazz labels such as Hat, FMP, and Soul Note, including Who's Bridge (Avant, 1994) and Two Days in Chicago (Hat, 1999). During the 1998 Chicago sessions that became the Hat release, Mengelberg and a few other Dutch musicians (including Ab Baars) performed with some of the best current avant-garde jazz musicians of the city (and the U.S.), including AACM tenor sax elder Fred Anderson and the most recent McArthur Genius Grant recipient saxophonist Ken Vandermark.

Over the decades, Mengelberg collaborated with many of the best and best-known players in European avant-garde and American free jazz. He was part of the Dutch group that played with Cecil Taylor in 1967 (around the same time he was participating in absurdist music theater), has played with leading British, German, and other European players, and has contributed compositions to Alexander von Schlippenbach's groups such as the Berlin Contemporary Jazz Orchestra. In addition to his years of collaborating with explosive percussionist Han Bennink, Mengelberg performed regularly with Wim T. Schippers between 1974 and 1982. The pianist has also led one of the consistently best creative jazz orchestras for over two decades. Even in the late '90s, and into the next century, the ICP Orchestra was touring regularly, including a return to North America for many highly anticipated and well-received performances.