Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Usenet, FidoNet, and federated-by-default

For roughly twenty years, between the late 1970s and the late 1990s, the dominant venue for online conversation was a federated peer-to-peer system that no single party controlled. It worked. Tens of thousands of independent servers ran the software, exchanged messages through store-and-forward propagation, and produced an aggregate conversation space that, at its peak in the mid-1990s, included tens of thousands of distinct newsgroups, millions of participants, and an estimated daily message volume in the terabyte range. There was no central server. There was no central administrator. There was no terms-of-service document anyone could sign. The participants in the conversation were on machines administered by their own institutions or paid for by their own subscriptions, and the conversation moved between those machines by a propagation mechanism that resembled the way gossip moves through a village. This was Usenet. Its dial-up cousin FidoNet operated on the same principles for a similar audience of personal-computer users. Both systems worked, neither was perfect, and both were displaced by web-based alternatives in a process that took about a decade and that produced a structurally different model of online discussion. The web model is the model we now have. This chapter is about the one we did not keep.

How Usenet started

Usenet was created in 1979 and went live in 1980. The originators were Tom Truscott and Jim Ellis, graduate students at Duke University, and Steve Bellovin, a graduate student at the University of North Carolina at Chapel Hill. They had a small problem: they wanted a way to exchange announcements and discussions between Duke and UNC, and the existing tools — email, file transfer over UUCP — were not designed for many-to-many bulletin-board-style conversation. Truscott and Ellis wrote a set of shell scripts that propagated messages between hosts using UUCP, the Unix-to-Unix Copy Protocol that had been part of Unix since 1977. The messages were organized into “newsgroups,” each newsgroup being a topical channel that subscribers at any participating host could read and post to. Bellovin’s role was the network design: the propagation model in which each host kept a list of its peers and forwarded messages to them, with duplicate-detection mechanisms to prevent loops.

The first installations were at Duke and UNC. Within a year, several more universities had joined. The growth was organic and unmanaged: a sysadmin at a new institution would obtain the news software, configure it to peer with someone running it elsewhere, and join the network. There was no application process. There was no registration. There was no central directory. The peering arrangements were bilateral, negotiated between the sysadmins who agreed to feed each other news. The protocol’s design meant that any peering arrangement was sufficient to put a host on Usenet; you did not need to be peered with a “main” server because there was no main server. You just had to be connected, somehow, to the rest of the graph.

The original transport was UUCP over modem links. Messages traveled by store-and-forward: a host would batch up its outgoing messages, dial its peers periodically (typically nightly), exchange messages, and disconnect. Latency was measured in hours or days. A message posted at MIT might take three days to reach Berkeley, depending on the connectivity in between. This sounds slow now and was slow then, but it was acceptable to the participants because the alternative — no conversation at all — was worse.

The network grew

Through the 1980s, Usenet grew steadily as Unix systems proliferated through universities and corporate research labs. The propagation infrastructure also grew. Eric Allman wrote B News in 1981, replacing the original A News shell scripts with a more efficient C implementation; Henry Spencer and Geoff Collyer wrote C News in 1987, replacing B News with a more efficient and more correctly behaved implementation; Rich Salz wrote INN, the InterNetNews server, in 1991, which combined news transport with the NNTP protocol that had become standard. NNTP — the Network News Transfer Protocol, specified in RFC 977 by Brian Kantor and Phil Lapsley in 1986 — let news propagation happen over TCP/IP connections instead of UUCP, which dramatically reduced latency and allowed real-time reading and posting against remote news servers. By the early 1990s, most Usenet news traffic was moving over NNTP, although UUCP propagation remained important for sites without persistent internet connectivity.

The hierarchies of newsgroups were organized into a structure called, by 1987, the “Big Seven” — comp, sci, soc, talk, news, rec, and misc. Each top-level hierarchy was the home of newsgroups on a category of topics: comp.* for computing, sci.* for science, soc.* for social issues, talk.* for politically charged discussion, rec.* for recreation and hobbies, news.* for Usenet itself, misc.* for everything else. A “humanities” hierarchy was added in 1995, making the Big Eight. Creating a new newsgroup in the Big Eight required a formal process: a proposal, a discussion period, a vote, and (if the vote passed by the required margins) a control message that propagated through the network and created the group on participating servers.

The alt.* hierarchy was created in 1987 by John Gilmore, Brian Reid, and Gordon Moffett as a workaround for the formal creation process. Alt groups could be created by anyone with the ability to send a control message; there was no vote, no waiting period, and no central approval. The alt hierarchy quickly became larger than the Big Seven combined and contained newsgroups on every conceivable topic. The alt structure was, in some sense, the first deliberately ungoverned region of the internet, predating the libertarian rhetoric around the early web by several years. It was also where most of Usenet’s actual cultural life happened, especially through the 1990s.

The governance, such as it was

The governance of Usenet was informal and distributed. There was no Usenet Inc. There was no Usenet board. There were, however, a number of social institutions that emerged from the practice of running and reading news.

Backbone cabal. In the early years, a small group of sysadmins at well-connected sites — the “backbone cabal” — handled the high-volume news propagation that connected major regions of the network. Their cooperation was necessary for news to flow efficiently; their authority over newsgroup creation, in the Big Seven, was correspondingly substantial. The cabal was never formal; its membership was understood by participants but never officially listed; and its existence was sometimes denied by its members for rhetorical effect. By the late 1980s the cabal had effectively dissolved as backbone connectivity diversified, but the legend of the cabal — “There Is No Cabal” became a recurring joke — remained part of Usenet’s culture.

Newsgroup moderators. Some newsgroups were moderated, meaning that posts to the group were sent to a moderator, who approved or rejected them before they propagated. Moderation was specified in the group’s creation; once a group was moderated, posts to it were rejected unless they came from the moderator. The moderator’s identity was specified in the group’s control message and could be changed only with another control message. Moderation was a practical solution to the spam and noise problem in groups with topical focus; it required the moderator to do real work, and the burnout rate among moderators was high.

The FAQ. Many newsgroups maintained Frequently Asked Questions documents that summarized the group’s conventions, scope, and answers to common questions. The FAQs were posted to the group periodically — usually monthly — and copies were maintained at archive sites. Reading the FAQ before posting was considered a basic obligation of new participants. New participants who posted without reading the FAQ were corrected, often unkindly. The FAQ was, in a sense, the legible part of the newsgroup’s culture; the rest of the culture was implicit and had to be learned by reading the group for a while before participating.

Killfiles. The reader’s primary defense against unwelcome content was the killfile: a local list of authors, subjects, or message patterns that the reader’s newsreader would automatically discard. Killfiles were per-reader; they did not affect the content other readers saw. The decision about what to read was the reader’s, not the network’s. This is a property the modern social media platform has not preserved: in current platforms, the platform decides what to show each user, and the user’s ability to filter is limited to the controls the platform provides. Usenet’s killfile was a programmatic interface; users could write arbitrary patterns and the newsreader would apply them.

These governance institutions were lightweight, unglamorous, and worked. Usenet was a fully functional discussion system for two decades, run by no one, controlled by no one, and yet sustaining a coherent and productive conversation across thousands of newsgroups. The system’s flaws were real — and treated below — but the basic question of whether a federated, ungoverned discussion system could work at scale was answered, affirmatively, by Usenet’s existence.

FidoNet

Usenet’s parallel for the personal-computer world was FidoNet. Founded by Tom Jennings in 1984 in San Francisco, FidoNet was a store-and-forward network of bulletin board systems running the Fido BBS software (or compatible alternatives) and exchanging messages over dial-up modem links. The architecture was hierarchical: zones, nets, hubs, and nodes, with messages propagating up and down the hierarchy according to FidoNet’s routing conventions. The propagation mechanism was the same as Usenet’s: each BBS dialed its peers periodically, exchanged a batch of messages, and disconnected. The latency was higher than Usenet’s — typically a few days for a message to propagate across the network — but the network grew rapidly as personal computers and modems became more affordable.

FidoNet’s structure was different from Usenet’s in important ways. The participating systems were not university or corporate research servers; they were home computers, mostly, run by hobbyists out of their basements or spare rooms. A FidoNet sysop was typically a single individual operating a single BBS. The cost of participating in FidoNet was the cost of a long-distance phone call to dial your peers; this cost was real, in the era of expensive long-distance, and was sometimes substantial, but it was not prohibitive for committed hobbyists. The participants in FidoNet skewed younger, less professional, and less academic than the participants in Usenet, with substantial overlap in the technical-hobbyist community that read both.

Echomail was FidoNet’s equivalent of Usenet newsgroups: topical message areas that propagated across the network, with sysops choosing which echoes to carry on their boards. There were hundreds of echoes by the late 1980s and thousands by the early 1990s. The echoes covered programming, hobbies, politics, religion, fiction, technical support — the full range of human interest, much as Usenet did, but among a different demographic. The two systems were sometimes connected by gateways that exchanged messages between specific FidoNet echoes and specific Usenet newsgroups, blurring the line but never fully merging the communities.

FidoNet peaked in 1995 with roughly thirty-five thousand listed nodes. The decline began as personal computer users gained internet access through commercial ISPs, which offered email, Usenet, and (after 1994) web access at lower cost and higher convenience than running a BBS. The BBS culture that FidoNet was the network of dissolved through the late 1990s as the modem-based community migrated to the internet. FidoNet still exists, in tiny form, with a small community of nodes continuing to exchange echomail over modern internet transport; it is a curiosity now rather than a working system.

The decline of Usenet

Usenet’s decline is the more important story for this chapter, because Usenet was the larger system and the more direct precursor of what online conversation might have been on the web. The decline had several causes.

Eternal September. In September 1993, AOL — at the time the largest commercial online service in the United States — began offering Usenet access to its members. The result was that the volume of new participants flooding into Usenet’s existing newsgroups overwhelmed the social mechanisms that had previously managed the integration of newcomers. Every September, in previous years, the new academic year had brought a wave of college freshmen onto Usenet, and the existing community had spent a few weeks teaching them the conventions. The 1993 wave was different: it was much larger, it was permanent (AOL members did not graduate), and the new participants did not, in most cases, want to learn the conventions of a community they considered themselves a customer of. The phrase “Eternal September” was coined to describe the situation, and it became part of Usenet’s vocabulary for the rest of its life.

Spam. On April 12, 1994, two lawyers named Laurence Canter and Martha Siegel posted an advertisement for their immigration services to several thousand Usenet newsgroups simultaneously. This was the first widely recognized commercial spam on the internet, and the response from the Usenet community was vigorous: complaints to Canter and Siegel’s ISP, retaliatory mail, and the development of automated spam-cancellation tools. The cancellation tools were partially effective. Spam, however, kept coming, in escalating volumes through the 1990s, and the social and technical apparatus Usenet had for dealing with it was repeatedly overwhelmed. By the late 1990s, many newsgroups were carrying more spam than legitimate content, and reading them had become a labor of filtering rather than a pleasure of reading.

Binary newsgroups. The alt.binaries.* hierarchy was the home of binary file distribution: software, images, audio, and (especially after the late 1990s) video. The bandwidth consumed by binary newsgroups grew enormously through the late 1990s and 2000s. By the mid-2000s, more than 99% of Usenet’s bandwidth was alt.binaries traffic, and the textual conversation that had been Usenet’s original purpose was a tiny fraction of the total. Many ISPs concluded that the bandwidth cost of carrying full Usenet feeds was no longer worth the small benefit to a small number of customers, and began dropping their Usenet service in the early-to-mid 2000s. The major ISPs — AOL, Verizon, Comcast, AT&T — had all dropped consumer Usenet access by the late 2000s. The textual newsgroups were almost empty by then anyway, having been hollowed out by spam and the migration of conversation to web forums.

Web forums. The phpBB, vBulletin, and similar web-forum platforms that emerged in the late 1990s offered, on the web, what Usenet had offered: topical discussion, threaded conversation, archived history. The web forums were not federated — each forum was a single site, run by a single administrator, with its own user accounts and its own database — but they were easier to set up, easier to moderate, easier to defend against spam, and easier for new users to participate in than Usenet was. The migration of conversation from Usenet to web forums was gradual and incomplete, but by the mid-2000s most of the active topical conversation that had been on Usenet was on web forums instead. The price was the loss of federation. A given conversation now belonged to a specific forum on a specific site; if the site closed, the conversation died.

Platforms. The web forums in turn were absorbed, through the 2010s, into a smaller number of large platforms — Reddit, Discord, the various Facebook and Twitter alternatives. The platforms were not federated, were operated by single companies, and concentrated discussion into a handful of large venues. By the early 2020s, the consolidation was nearly complete: most online discussion that was not happening in private messaging apps was happening on a few platforms, each of which had its own rules, its own moderation regime, its own algorithmic feed, and its own commercial interests.

What was lost

The loss in the migration from Usenet to platforms was specific. Several properties Usenet had, the platforms did not.

Federation. Usenet was federated by default. A user on one server could read and post to newsgroups whose primary readership was on other servers. A user dissatisfied with one server’s policies could move to another. A server administrator dissatisfied with the network could refuse to carry certain newsgroups, without preventing other servers from carrying them. The platforms are not federated. A user dissatisfied with a platform’s policies can leave the platform; they cannot, generally, take their participation in the community with them.

User-controlled filtering. The killfile let each Usenet reader decide what to read. The platforms control what users see through algorithmic feeds that are not under user control. Users can mute or block individual accounts on most platforms; they cannot, generally, instruct the platform to never show them content matching arbitrary patterns. The locus of filtering authority has moved from the user to the platform.

Topic stability. A Usenet newsgroup, once created, was a stable address for a topical conversation. The newsgroup persisted across decades; the conversation in it accumulated across years; an archive of the group was available to new participants who wanted to learn its history. The platforms’ equivalent — a subreddit, a Discord server, a Facebook group — is technically stable but is, in practice, much more easily disrupted by platform decisions: the platform can ban the community, change the moderators, alter the visibility, or change the underlying rules. The community on a platform exists at the platform’s pleasure.

Identity persistence outside the venue. A Usenet poster’s identity was their email address, which was their identity outside Usenet as well. The poster’s reputation was, in a sense, portable: it was attached to a name and address that meant something in other contexts too. The platforms’ identities are platform-specific. A Reddit username has no portability outside Reddit; a Discord username has no portability outside Discord. The user’s reputation, such as it is, is captured by the platform and does not follow the user when they leave.

Archival. Usenet’s full archive — every message ever posted — was, in principle, preserved at multiple sites. Google acquired DejaNews’s archive in 2001 and made it the basis of Google Groups, which retains a substantial portion of historical Usenet. The platforms’ archival practices are inconsistent: some platforms preserve indefinitely, some delete, some allow user-initiated deletion, and the long-term availability of platform content is governed by the platforms’ commercial interests rather than by any archival commitment.

What is being recovered

ActivityPub and the fediverse, treated in chapter twenty-two, are the most direct recovery of Usenet’s federated model. The fediverse is, in effect, Usenet with web-style identity and presentation: a federation of independently operated servers, each hosting its own users, exchanging messages with other servers through a standard protocol. Mastodon, Pleroma, Lemmy, Misskey, and the other fediverse implementations are not Usenet in the technical sense, but they share its structural commitments. Lemmy in particular is structurally close to Usenet: it offers federated discussion communities, similar to newsgroups, with each community hosted on a specific server but readable and postable from any server. The fediverse is growing, slowly, and is now in the position Usenet was in around 1988 or 1989: working, established at small scale, with a passionate community and a slow but steady growth curve.

NNTP itself continues to exist. Several news servers continue to run, primarily for binary distribution but also for some textual newsgroups that have small but persistent communities. Eternal-September.org, founded around 2010, provides free NNTP access for non-binary newsgroups and has served as a refuge for textual Usenet participants who want to continue using the protocol. The protocol works; the question is whether anyone uses it. The answer, in 2026, is yes, in small numbers.

The IRC ecosystem persists, with Internet Relay Chat servers federated across networks (each major network — Freenode/Libera, Undernet, EFnet, and others — is internally federated). IRC is real-time chat rather than threaded discussion, so it does not directly substitute for Usenet, but the federated structure has been maintained continuously since the early 1990s. Matrix, a newer protocol with similar federation properties for chat, has been gaining ground since 2014 and is the most prominent current example of a federated chat protocol with serious institutional backing.

What the federated history teaches

The lesson of Usenet and FidoNet, taken together, is that federated online discussion can work at scale, for decades, without central administration, as long as the participants are willing to do the work of maintaining the federation. The work is real: server operators have to peer, sysadmins have to handle spam, communities have to maintain their FAQs and welcome their newcomers, users have to maintain their killfiles. The work is also distributed: no single party bears the entire cost, and no single party has the entire authority. The platforms have replaced this distributed effort with concentrated platform-side work — moderation teams, algorithmic enforcement, terms-of-service enforcement — and have, in exchange, gotten near-total control of the resulting communities. The trade is not obviously good. Most users do not feel the loss directly because the platforms are convenient and the federated alternatives are unfamiliar. But the loss is real, and the recovery is being attempted in the fediverse on a scale that suggests at least some users feel it.

The next chapter takes the one federated service from the pre-web era that survived. Email is the protocol that did not get displaced; the system Postel specified in 1982 still runs, on the same architecture, with the same federation, and is the only piece of pre-web federated infrastructure that remains in mass use. It has not, however, survived in good condition. The chapter is about what happened to email that almost happened to all the other federated services in this book, and what it would mean to fix it.