Gopher and the case for hierarchy
The cleanest case of the contest treated in this book is Gopher. It was released in April 1991, two months before the first public version of the web outside CERN; it grew faster than the web for two years; it had, in late 1992 and early 1993, more servers and more users than the web; it was widely deployed across the academic internet, with substantial industrial uptake beginning; it was technically simpler than the web in several respects that mattered for operators; and it was, by the end of 1995, a finished product whose use was almost entirely confined to a small archival community. Gopher’s story is the story of how an internet protocol can win the early adoption contest, hold the technical lead by most measures, and lose anyway, decisively, in a way that turned on one institutional decision made in March 1993 that the people who made it have publicly regretted. This chapter is about what Gopher was, what its argument for hierarchy was as a design choice, why the contest with the web tilted the way it did, and what hierarchical navigation has been recovered as on the web that absorbed Gopher’s ideas in degraded form.
What Gopher was
Gopher was created by a small team at the University of Minnesota’s Microcomputer and Workstation Networks Center, principally Mark McCahill, Farhad Anklesaria, Paul Lindner, Daniel Torrey, and Bob Alberti. The first public release was in April 1991. The team had been working, before Gopher, on a campus-wide information system to help students and faculty find information distributed across various Minnesota servers; Gopher was the result of that work. The name was a pun: a “gopher” was the colloquial term for an office assistant who went to fetch things, and the gopher was also the University of Minnesota’s athletic mascot. The protocol was as simple as the name suggested.
A Gopher server, running on TCP port 70, presented a hierarchy of menus. Each menu was a list of items. Each item had a type — file, directory, search service, Telnet session, image, sound, binary executable, and a few others — and a name and a target. A Gopher client connected to a server, requested a menu, displayed the menu items to the user, and let the user select an item to follow. If the item was a directory, the client requested that directory’s menu and displayed it. If the item was a file, the client retrieved and displayed the file. If the item was a search service, the client prompted the user for a query and submitted it. If the item was a Telnet session, the client opened a Telnet connection to the specified host.
The whole protocol fits on a few pages. RFC 1436, the Gopher specification, was published in March 1993 and is one of the shorter RFCs in the protocol series. There were a few command bytes, a tab-separated response format, and a small set of item-type codes. Implementing a Gopher server was a weekend project. Implementing a Gopher client was a weekend project. The simplicity made the protocol portable: Gopher clients existed for Unix, VMS, DOS, Windows, Mac, and most other systems with TCP/IP stacks. Gopher servers existed for an even wider range of platforms. By late 1992 there were a few hundred public Gopher servers; by mid-1993 there were several thousand.
The case for hierarchy
The design argument Gopher made — implicit in its protocol, explicit in its documentation — was that information should be organized hierarchically and navigated by selection from menus. The argument has more substance than the dismissive accounts of Gopher tend to acknowledge.
Hierarchies are predictable. A user navigating a hierarchy knows what kind of thing to expect at each level: the top-level menu describes broad categories, the next level describes subcategories, and so on down to the leaves. The user can form expectations about where to find a specific kind of information without having to know in advance where it is. This is the property that has kept hierarchies the dominant organizing principle for libraries, file systems, and bureaucracies for thousands of years. The web’s reaction against hierarchy was, in a sense, a reaction against the central organizing principle of all prior information systems, and it is worth taking seriously the possibility that the reaction was wrong.
Hierarchies make orientation easy. A user navigating a hierarchy always knows where they are: they have descended from the root along a known path, and they can return to any ancestor by ascending one step at a time. The web’s reaction against hierarchy was a reaction against this property too. On the web, a user can arrive at a page from anywhere; the page does not always make clear where it sits in a larger structure; there is no canonical “up” from most web pages, and the back button is the closest thing the web offers to a hierarchical retreat. Web usability research throughout the late 1990s and 2000s repeatedly identified disorientation as one of the most common user problems, and design conventions like the “breadcrumb trail” exist specifically to retrofit hierarchical orientation onto a system that does not have it natively.
Hierarchies are explicit about authority. The administrator of a Gopher hierarchy decided what categories existed and what fit under each. A user looking at the menu was looking at the administrator’s classification of the available material. This is, in some ways, paternalistic; it imposes a particular structure on the user’s exploration. It is also informative: the user knows, by looking at the menu, what the administrator thought was important enough to surface and how the administrator thought the material related. The web’s reaction against hierarchy was also a reaction against this kind of explicit editorial framing. A web user navigating a corpus of pages typically encounters the pages in the order their search engine returned them, with no editorial structure visible. The cost of the web’s anti-editorial bias is that the user must do the structuring work themselves; the benefit is that the user is not constrained by some other party’s structure.
The case for hierarchy, then, is a case for predictability, orientation, and explicit editorial framing as virtues of an information system. The case is not foolish. The systems that compete with hierarchy — full-text search, associative links, faceted classification — make different trade-offs and lose what hierarchy provides. Modern interfaces typically combine all of them: a file system has both a directory hierarchy and a search box. Gopher was a system that took the hierarchy side of the trade-off seriously and made the others available only as auxiliary services.
Veronica, Jughead, and the index ecosystem
Gopher was not pure hierarchy. The protocol included a search-service item type, and Gopher quickly accumulated search tools that indexed gopherspace as a whole. The most important was Veronica, developed at the University of Nevada, Reno, by Steve Foster and Fred Barrie in 1992. The name was a pun on Archie, the FTP search tool, with both names eventually accumulating the additional reference to the Archie Comics characters as the joke deepened. Veronica indexed the menus of Gopher servers worldwide and let users search for items by title across the whole gopherspace. A second tool, Jughead (developed by Rhett Jones at the University of Utah), did the same for single Gopher servers or limited subtrees.
The Veronica index was the closest thing the pre-web internet had to a search engine in the modern sense. It was less rich than what AltaVista would offer in 1995 — Veronica indexed only menu titles, not the contents of documents — but it was a working federated index of a substantial corpus, deployed at multiple sites, with reasonable performance and broad usefulness. The Veronica architecture had a property that the search engines that displaced it did not preserve: there were multiple Veronica instances run by different operators, each indexing some portion of gopherspace, each available for use through a Gopher menu, each producing results that the user could see were limited to that index’s coverage. The federated structure of Gopher search was visible to the user. Search was a peer service in Gopher, exactly as WAIS proposed for the broader internet (the next chapter), and the contrast with the eventual centralization of web search at a small number of companies is part of what was lost when the web absorbed Gopher’s audience.
The 1993 licensing decision
In March 1993, the University of Minnesota announced that it would begin charging licensing fees for the Gopher server software when used in commercial settings. The fees were modest — five hundred dollars per server, with smaller amounts for smaller deployments — and were specifically restricted to commercial use; academic and non-commercial users could continue using Gopher without charge. The university’s reasoning was that Gopher had developed into a real software product, that supporting it cost real money, and that commercial users who were running it as part of their business should contribute to the cost.
This was a normal kind of decision in the academic software-licensing world. Many universities licensed their software to commercial users; the X Window System, for instance, was licensed by MIT with similar terms; the Berkeley Software Distribution had complex licensing arrangements that became, in the early 1990s, the subject of significant litigation. Minnesota’s terms were not unusually restrictive. They were not, in absolute terms, expensive. The decision was, in retrospect, reasonable on its own merits.
The effect on Gopher’s adoption was nevertheless catastrophic. The institutional users who had been considering Gopher for commercial deployment now had to negotiate license terms, get sign-offs from their legal departments, and assume the maintenance of an ongoing commercial relationship with the University of Minnesota. Many of them decided this was more administrative friction than the system was worth. The decision arrived at the worst possible time, in early 1993, just as commercial interest in internet services was beginning to accelerate. Commercial decision-makers were comparing Gopher with a hypothetical alternative — there were still very few web servers — and the comparison shifted in the web’s direction the moment Gopher acquired an ongoing licensing cost and the web did not.
CERN, which had been considering its own licensing position, responded to the Gopher decision the next month. On April 30, 1993, CERN released a statement making the web protocol specifications — HTTP, HTML, the URL syntax, the reference implementations — public-domain, with no restrictions on use, commercial or otherwise. The decision was made by Robert Cailliau and others at CERN, with Tim Berners-Lee’s involvement; the statement was specifically intended to remove any uncertainty that institutions might have about whether using the web would create future licensing obligations. The contrast with Minnesota’s announcement could not have been clearer, and the announcement was widely read at the time as a deliberate competitive response to Gopher.
The Minnesota team has been candid, in subsequent interviews, that the licensing announcement was the moment Gopher lost. McCahill has said the announcement was a mistake. Lindner has said that what looked like a routine commercialization decision became, in a few months, the inflection point at which institutional momentum shifted to the web. The team modified the licensing terms later in 1993, dropping the fees, but the damage was done. Institutions that had paused their Gopher rollouts to evaluate the licensing situation never came back. Many of them moved to the web instead.
This is, in many ways, an unsatisfying story. A single institutional decision, made under reasonable considerations, became a major factor in determining the shape of the internet for the next three decades. The dependency of the outcome on the decision is real but feels disproportionate. It is worth saying directly: Gopher was not doomed in March 1993. Several other factors were also pushing toward the web, particularly Mosaic’s arrival in early 1993 and the web’s faster development pace through that year. But the licensing announcement was the moment the trajectory became visible to the people who were paying attention, and it was the moment most of the technical and institutional momentum behind Gopher began to dissipate.
Gopher+ and the late phase
The Minnesota team did not give up. In late 1993 they released Gopher+, an extended version of the protocol that added several features the web had and Gopher had lacked: attributes for menu items (a kind of richer metadata), multiple representations of the same item (so a file could be retrieved in several formats), forms for user input, and a set of conventions for richer interaction. Gopher+ was, in many ways, a closer competitor to early HTTP than original Gopher had been. It was deployed at a number of sites and supported by some clients, but it never reached the saturation level of original Gopher, and it did not stem the migration to the web.
The Gopher protocol continued to be developed in small increments through the mid-1990s. The University of Minnesota maintained the canonical server software. Various third-party clients and servers continued to be developed, including some open-source ones. By 1996 or 1997 the protocol was essentially in maintenance mode; new development had stopped, the user base was shrinking, and the institutional sites that had been the protocol’s backbone were largely migrating their content to web servers. Some content remained on Gopher servers — particularly archival material that was never migrated — but the protocol’s role in the live internet was over.
The Gopher community’s persistence
What is striking about Gopher’s afterlife is that the protocol did not die. It went into a kind of stasis, and a small but persistent community continues to use it. The Floodgap Gopher server, run by Cameron Kaiser since 1999, has been a focal point of the modern Gopher community; it hosts and indexes a substantial corpus of Gopher content, including new content produced specifically for the protocol. The Pygopherd server, written by John Goerzen, has provided a modern Gopher server implementation since 2002. New Gopher clients are available for current operating systems; new content is being produced; small communities of users use Gopher daily as their primary internet protocol.
This persistence is partly nostalgia and partly principled choice. The community of people who use Gopher in 2026 is small — perhaps in the low thousands — but it is articulate about what Gopher offers that the web does not. The protocol is simple. The content is free of advertising, tracking, and javascript. The clients are fast. The corpus is small enough that browsing it feels like a navigable activity rather than a search-mediated one. The hierarchy is intact. Gopher in 2026 is, for the people who use it, a working illustration of what the internet had been before the consolidation and commercialization of the web. The Gemini protocol, treated in chapter twenty-three, is a partial successor to Gopher that has emerged in the 2020s and is, in many ways, a deliberate effort to recover what Gopher had been.
What the web absorbed and degraded
The web absorbed parts of Gopher’s design. Yahoo, founded in 1994 by Jerry Yang and David Filo, was a hierarchical directory of web pages — essentially Gopher’s organizing principle applied to the web’s content. Yahoo was, for much of the 1990s, the dominant entry point to the web; users typed yahoo.com into their browsers and walked the hierarchy of categories to find what they wanted. The Yahoo directory was Gopher’s idea, retrofitted onto a system that had been built specifically against the hierarchical model.
Yahoo’s directory was overtaken, in the late 1990s, by full-text search engines — first AltaVista (1995), then Google (1998). The shift from directory to search was the moment the web definitively chose against hierarchy as its primary organizing principle. Yahoo’s directory was deprecated, then dismantled. The DMOZ Open Directory Project, a volunteer-edited hierarchical directory of the web that had been a major Yahoo competitor through the early 2000s, was closed in 2017 after twenty years of operation. The hierarchical-directory model lives on in vestigial form (the navigation menus of most websites are local hierarchies; Wikipedia’s category system is a directory) but is not how anyone finds anything on the broader web.
The cost of the shift is hard to measure but real. A user navigating the Yahoo directory in 1996 was looking at an editor’s classification of the web’s content into categories. The classification was imperfect, biased, slow to update, and inconsistent across categories. It was, however, an explicit editorial framing of the available content, produced by humans who had read the material and decided where it fit. The user could survey a category and see, at a glance, what the editor considered the relevant material; the user could trust, or distrust, the editor’s judgments; the user could form expectations about what was available and what was not. The replacement of this kind of explicit editorial framing with the implicit framing of search rankings was a real change in what kind of relationship a user had with the corpus they were navigating. Search hides its editorial choices behind an algorithm; the directory put its editorial choices on the page. Both are editorial. Only one was honest about being editorial.
The other thing the web absorbed and degraded was the federated index. Veronica had been a federated search across many independently operated Gopher servers; the early web had Archie-style federated indexes for a few years (the Aliweb, Lycos, WebCrawler era). The consolidation of search at Google in the late 1990s ended the federation. Search on the modern web is provided by a small number of companies, with Google holding overwhelming dominance, and the federated alternative that WAIS and Veronica had embodied is recovered only in narrow domains (academic search across federated repositories, library OPAC federation) and not as the user’s primary experience of finding things on the internet.
What hierarchy keeps doing for us
For all the web’s reaction against hierarchy, hierarchy keeps coming back where it is needed. File systems are still hierarchical. URLs are still partly hierarchical, with the path component reflecting an organizational structure even when the structure is internally invented. Navigation menus on websites are local hierarchies. The Atom and RSS feed standards are hierarchical. The Markdown table-of-contents convention is a hierarchy. The site map standard for search engines is a hierarchy. The web has hierarchy embedded in many of its conventions even though its protocol-level design was against hierarchy.
The recovery in Part V of the static renaissance — Gemini, Markdown, static site generators — is in part a recovery of hierarchy as an organizing principle. The structure of a typical static site is a hierarchy of categories and posts; the URL paths reflect the hierarchy; the index pages are essentially Gopher menus in HTML clothing. The Gemini protocol, treated in chapter twenty-three, makes the hierarchy explicit again, with index pages and a structural model that is close to Gopher’s. The static renaissance has not killed full-text search, and most static sites have search added in. But the foundation has shifted back to hierarchy as the primary organizing structure, and the search is a supplement rather than the foundation.
What Gopher’s case continues to make
Gopher’s case for hierarchy was a serious case, made by serious people, that was decided against by a combination of one consequential institutional decision and a set of broader pressures that the web’s design happened to fit. The decision was not foolish, the broader pressures were not capricious, and the outcome was, in a sense, the right outcome for the kind of internet the next thirty years would build. But the case Gopher was making — that information should have a place, that users should be able to walk a known structure, that an editor’s classification is a contribution worth surfacing — has not gone away as an intellectual position. It survives in the small Gopher community, in the static renaissance, in the personal-knowledge-tools community, in libraries, in archives, and in the various organizational schemes that quietly continue to structure how humans actually find things. The web has spent thirty years building elaborate systems to retrofit hierarchy onto a substrate that explicitly rejected it. Gopher had hierarchy built in.
The next chapter takes the protocol that, alongside Gopher, made the most serious pre-web argument about how internet services should be structured. WAIS — the Wide Area Information Server — was federated where the web is centralized, peer-distributed where the web has concentrated, and explicit about search as a service where the web has buried it inside a small number of companies. It is a system that has been almost entirely forgotten, and the forgetting matters because what WAIS knew about how search should work has been re-learned, painfully, by people who did not know they were re-learning anything.