Mary Boone Gallery was founded in New York in 1977. The first Gallery was a small ground floor space at the renowned SoHo address 420 West Broadway. From the outset, the Gallery was committed to showing the work of innovative young artists.

By the early 1980s, two artists who started with the Gallery—David Salle and Julian Schnabel—had attained international recognition. Both were given dual exhibitions with the Gallery's upstairs neighbor, Leo Castelli Gallery. In September 1981 Mary Boone Gallery expanded by completing a dramatic renovation of a truck garage directly across the street.

Other prominent New York-based artists under Gallery representation during this period include Richard Artschwager, Jean-Michel Basquiat, Ross Bleckner, Eric Fischl, Roni Horn, Barbara Kruger, Sherrie Levine, Brice Marden, Matt Mullican, and Richard Tuttle. Significant international artists were also represented by the Gallery during the 1980s, including Georg Baselitz, James Lee Byars, Francesco Clemente, Jorg Immendorff, Anselm Kiefer, Jannis Kounellis, Malcolm Morley, Sigmar Polke.

The Gallery intermittently mounted historical shows such as works by Francis Picabia, an installation by Marcel Broodthaers, 1960s paintings by Agnes Martin, a group of the “monument” for V. Tatlin sculptures by Dan Flavin, the rarely seen paintings of Clyfford Still, and the "Mirror Paintings" of the 1970s by Roy Lichtenstein. These exhibitions proved to be influential to the upcoming generation of artists.

By the 1990s, SoHo had evolved into a lively commercial district no longer chiefly inhabited by, and nurturing to, artists. In May 1996 the Gallery moved uptown to Fifth Avenue and 57 Street, long established crossroads of the art world.

With the move came the opportunity to show the work of young artists associated with “downtown” in a new context. Through this program the Gallery has shown and developed relationships with new artists such as Carol Bove, Thomas Demand, Ellen Gallagher, Terence Koh, Aleksandra Mir, and Mika Rottenberg. A number of these artists proceeded to mount solo shows at the Gallery.

In November 2000, Mary Boone Gallery opened an additional space in Manhattan's developing Chelsea art district – again in a former garage. This Gallery allowed for an expanded exhibition schedule and provided facilities for large-scale works and dramatic installations, such as Peter Halley's explosive hanging of paintings and wallpaper, Marc Quinn's complete series of carved marble statues of persons with missing limbs, monumental sculpture by James Lee Byars, and, most importantly, a four-channel DVD installation by Barbara Kruger.

Today, in addition to artists long-associated with the Gallery – Ross Bleckner, Francesco Clemente, Barbara Kruger, Barry Le Va, Keith Sonnier – the gallery represents internationally established artists such as Ai Weiwei, Robert Barry, Ericka Beckman, Sadie Benning, Judith Bernstein, Tomoo Gokita, Jacob Hashimoto, KAWS, Ed Paschke, and Peter Saul. Mary Boone Gallery is actively involved in the secondary market of artists important to its history, including Jean-Michel Basquiat, Jeff Koons, Brice Marden, and Julian Schnabel.