The town's founding dates to the mid-1850s with the building of a flour operation, Forbes Mill, by James Alexander Forbes along Los Gatos Creek. The mill's two-story stone storage annex has been preserved as a museum just off of Main Street.
The settlement that was established in the 1860s was originally named for the mill, but the name was changed to Los Gatos after the Spanish land grant. The town was incorporated in 1887 and remained an important town for the logging industry in the Santa Cruz Mountains through the end of the 19th century. In the early 20th century, the town became a thriving agricultural town with apricots, grapes and prunes being grown in the area. By the 1920s, the Los Gatos area had a local reputation as an arts colony, attracting painters, musicians, writers, actors and their bohemian associates as residents over the years. The violinist Yehudi Menuhin lived there as a boy; the actresses Joan Fontaine and Olivia de Havilland (sisters) were graduates of Los Gatos High School, John Steinbeck wrote The Grapes of Wrath there (the location is now located in Monte Sereno), and Beat hero Neal Cassady lived there in the 1950s. Along with much of the Santa Clara Valley, Los Gatos became a suburban community for San Jose beginning in the 1950s, and the town was mostly built-out by the 1980s.
Downtown Los Gatos has retained and restored many of its Victorian-era homes and commercial buildings. Other notable buildings are the Forbes Mill annex, dating to 1880 and now housing a history museum; Los Gatos High School which dates from the 1920s; and the Old Town Shopping Center, formerly the University Avenue School (the school was established in 1882; the current buildings date to 1923).
A number of brick buildings in Downtown Los Gatos were destroyed or seriously damaged in the 1989 Loma Prieta earthquake, though the district was quickly rebuilt and has made a full recovery.
Recently the Main Street Bridge has emulated the Ponte des Artes bridge in Paris by displaying love locks for anniversaries and engagements.