The Genesis Story: an Unofficial, irreverent, incomplete account of how the UNIX operating system was developed

By August Mohr

from the January 1985 issue of Unix Review magazine

This is, so to speak, a history of how UNIX evolved as a product; not the “official” history of who was responsible for what features, and what year which milestones were crossed, but the “political” history of how decision were made and what motivated the people involved. Most of the readers of this mazagine are familiar with the system itself, so I don’t want to go into great detail about how the system got to be what it is internally, but rather how it how it got to be at all.

Three years ago, I was one of many espousing the idea that Bell Laboratories and Western Electric had engaged in a masterful piece of long-term strategy by first releasing UNIX to universities and then later releasing it to the commercial world. How clever, I thought, to get the universities involved in the development of the system and get a whole crop of UNIX experts trained in the process. But after talking to some of the people involved, I have reversed my opinion. Bell, it seems, was dragged kicking and screaming into providing UNIX to the world.

IN THE BEGINNING, THERE WAS MULTICS

Multics, in some ways UNIX’s predecessor, was a huge project, a combined effort of three of the largest computing centers of the day. All three principles — Bell, GE, and MIT, had already done operating systems, even time-shared systems, before, and following Fred Brooks’ Second System Syndrome, it was felt Multics was going to solve all the problems of its predecessors. Ultimately, the project proved to be too late and Bell dropped out, leaving Ken Thompson, Dennis Ritchie, and Rudd Canaday without a timeshared system to play with.

So they set about building an operating system of their own.

So until this point, timesharing systems had only been developed for big machines costing hundreds of thousands of dollars. It was unlikely that Computer Research was going to get another investment of that sort out of Bell so soon after Multics. In any event, Thompson, Ritchie, and Canaday weren’t particularly interested in working on another large scale project, so they set their sights on obtaining a minicomputer on which to build a timesharing system they could use as a program development environment. They wanted the kind of flexibility and power they had worked toward in Multics, without incurring the expensive rings of protection and interlocks.

The project was short on computing power, though. With only a PDP-7 to work on, the researchers yearned for another machine — preferably a VAX 11/20. To get the necessary funds for a new machine, though, it was clear they would first need to find an application.

Fortunately, the Bell Labs’ Legal Department — which was close at hand to Thompson and Ritchie’s Murray Hill office — was looking for a word processing system at about this time. A paper-tape system for an old Teletype machine was under consideration.

With a bit of salesmanship, Thompson and Ritchie got the legal team interested in a UNIX-supported system. It proved to be a momentous deal. The UNIX project got the machine it needed and the legal department got the word processing system it wanted — along with some rather impressive local support.

A number of important barriers thus were crashed. First, the business side of Bell Labs, generally held at arm’s length by the people in Research, got a healthy dose of Research assistance. More importantly for UNIX users, Thompson and Ritchie got their first live customer. UNIX word processing suddenly had to be usable by secretarial staff as well as by programming staff. The changes made to accomplish this transition were to have serendipitous effects as UNIX evolved.

Richard Haight, now AT&T Bell Labs Supervisor of Video Systems Software (a UNIX application), had his first exposure to UNIX at about this time. As he remembers it, “I came across Ken Thompson by accident and wanted to do some software development on a PDP-11. He said he had a machine I could use but that it didn’t run on a DEC operating system. I had lots of exposure to timesharing systems and it was pretty obvious that this was superior in many ways.

“It was uphill in the beginning. The fact that it was homegrown in the Labs didn’t cut anything with the people that I was working with at that time. They went and visited Ken and Dennis in their sixth-floor attic at Murray Hill and all they saw was hardware laying all over the floor and a bunch of guys in T-shirts and sneakers. It wasn’t the sort of place that would warm the heart of a manager who had been brought up in a traditional data processing environment. It really was the ‘two good guys in a garage’ kind of syndrome except that they happened to be in an attic.

“That first time I met Ken Thompson, he had a small bookshelf, maybe 2 1/2 feet long, above an old Teletype Model 37 terminal. On the shelf were hardware manuals from Digital Equipment for the PDP-11, occupying maybe five inches, and the rest of the shelf was nothing but chess books. I think Thompson will tell you himself that he developed UNIX as a good place to develop chess programs. It turns out that it’s everybody else’s idea of where and how to develop programs too.”

Haight has had many years of experience in the Bell Labs environment. Before moving to his current post, he served as part of the UNIX Support group and the PWB (Programmer’s Workbench) development group. He believes that the environment of the Labs was a major factor in the creation of UNIX.

“There’s a small fraction of people who just go crazy over computers,” he explained. We hire a bunch of them and they remain workaholics for several years after coming in. Eventually they get a house and a mortgage and they get married and have kids and settle down to be normal human beings, but it’s wonderful to hire these people because you get two or three people’s work out of them for several years. Those are the kind of people that give you things like UNIX.”

AN INFLUENTIAL FEATURE: PIPES

Dick Haight is one of the people who became hooked on UNIX at the very beginning, and even yet is an outspoken proponent of its value and power — although he will tell you he is still waiting for something better to come along. “There’s a lot of complaint about UNIX being terse and for experts only,” he said. “I’ve seen that blamed on the pipe mechanism. If terseness is the price for having pipes. I’ll take it any day.”

As one of the new system’s first users, Haight got to follow many of the changes. “I happened to have been visiting the research crew the day they implemented pipes,” he recalled. It was clear to everyone, practically minutes after the system came up with pipes working, that it was a wonderful thing. Nobody would ever go back and give that up if they could help it.”

Haight believes that the pipe facility has had a major effect on the evolution of UNIX. Because of the shell and the piping facility, programs running under UNIX can be conversational without being explicitly programmed that way. This is because piping adds to the shell’s ability to accomplish this purpose for the system as a whole.

Doug Mcllroy is usually credited with the idea of adding pipes, although the idea may have been around since Multics. Haight believes there may have been yet another reason for implementing them. The original UNIX had a file size limitation of 64K and, according to Haight, “one of the people there in research was constantly blowing that size restriction in an intermediate pass of the Assembler.” Between Mcllroy’s lobbying for the idea and this other problem with file size, Thompson and Ritchie were finally convinced to implement pipes.

MOVING INTO THE COMPANY

Independent of what was happening in the research area, Bell was starting to perceive the need for minicomputer support for its telephone operations. It needed Operations Systems, not Operating Systems. With the numbers of systems under consideration, the possibility of being tied to a single vendor, or having each site tied to a different vendor, induced a kind of paranoia. There just had to be another way.

The groups responsible for developing operations systems had many people from hardware and applications software backgrounds who were considering writing their own operating system — their first — when Berkley Tague, now head of Bell Labs’ Computing Technology Department, suggested they use UNIX to get started.

“I observed that people were starting to put these minis out in the operating company, and saw that it was an area of both opportunity and potential problems,” Tague remembered. “I found that some of the people in development had never built an operating system for any computer before; many of them had very little software background. They were coming out of hardware development and telephone technology backgrounds, and yet were starting to build their own operating systems. Having been through that phase of the business myself, it seemed silly to go through it another hundred times, so I started pushing the UNIX operating system into these projects.”

Tague’s backing of UNIX, as a development system for operations, was not just a personal preference. “I had every confidence in the people who built it because I’d worked with them on Multics,” he explained. “With their experience and training, I figured they could build a much better operating system than somebody who’s building one for the first time, no matter how smart that person is.”

SUPPORT?

UNIX had been running long enough in research by that time that Tague knew that the system the operations group would get would serve as a very good starting point. Unfortunately, there was no vendor support for it.

The argument Tague made for UNIX was: if the operations people were going to build their own system, they were going to have to maintain it themselves. Surely, UNIX could be no worse. They could use it to get started and do the development. If a more efficient or better operating system was needed for a target machine when they got into the field, they could always build it, but UNIX would at least get them off the ground. “Naturally I knew that once they got on this thing they wouldn’t be able to get off. It’s just like drugs,” Tague explained.

Tague also knew it was important to get some field support. “We were starting to put these things in the operating companies all around the countryside and the prospects were that there were going to be several hundred minis over the next few years that were going to have to be maintained with all their software and hardware,” he said.

Bell had already gained some field support experience maintaining electronic switching machines and their software. Supporting a network of minicomputers would be a significantly different problem, though. Maintaining an operating system is not at all like maintaining an electronic switching system. The minicomputers had different reliability demands, requiring a different support structure in the organization — one that did not yet exist in any form. In many ways, the operations group was breaking new ground.

Up to this point, Tague had served as head of the Computer Planning department responsible for systems engineering. After gaining support for UNIX in the operations group over the course of 1971 and 1972, he made a push for two significant changes. The first was to make UNIX an internal standard and the second was to offer central support through his organization. In September, 1973, he was permitted to form a group called UNIX Development Support, the first development organization supporting a “Standard UNIX”. While this group worked closely with Bell Labs Research, its concerns sometimes diverged.

One area the two groups could agree on, though, was portability. By 1973, it was already on the horizon. Tague foresaw the possibility of UNIX becoming an interface between hardware and software that would allow applications to keep running while the hardware underneath was changing.

From the support point of view, such a capability would solve a very important problem. Without UNIX and its potential portability, the people building the operations support systems were faced with selecting an outside vendor that could supply the hardware on which to get their development done. Once that was complete, they would be locked into that vendor. Portability obviated this limitation and offered a number of other advantages. When making a hardware upgrade, even to equipment from the same vendor, there are variations from version to version. That could cost a lot of money in software revisions unless there were some level of portability already written into the scenario. Fortunately, the integral portability of the system developed by Research proved adequate to make UNIX portable over a wide range of hardware.

The first UNIX applications were installed in 1973 on a system involved in updating directory information and intercepting calls to numbers that had been changed. The automatic intercept system was delivered for use on early PDP-11s. This was essentially the first time UNIX was used to support an actual, ongoing operating business.

To Tague, at this time, “our real problem was pruning the tree”. There were so many different sites using UNIX that each would come up with different answers to the same problems of printer spooling, mail, help, and so on. “The customers would invent this stuff and make it work and our problem was to get the slightly different variations together, get the best of all of those worlds, put it in the standard, and get it out again.” This was, in many ways, a political process. Tague credits the “technical underpinnings” for making the process easier than expected. “That made it easy to get the right stuff in without upsetting the whole world. I didn’t have to go to all of my customers and tell them that this was now my new version and that nothing they had out in the field would run again,” he said.

To provide a standard UNIX system, the support group had to establish what version it would back. This was a process of negotiation and compromise with the UNIX-using community — not a unilateral decision. The support team and customers often ended up arguing things out until everybody understood the issues and a suitable compromise was made. “As one of the local gurus put it to me one time,” Tague said, “one of the problems in UNIX is that everybody wants to carve his initials in the tree.” When the choice came down essentially to tossing a coin, Tague, as arbirator, tried to make sure that each group got at least one pet contribution into the system.

Fortunately, UNIX is flexible enough that even the particularly traumatic decisions, such as the ones concerning standard shell versions, could be patched in slowly — at the user’s discretion.

A FAMILIAR PATTERN

From the very beginning within Bell, UNIX followed what has become a familiar pattern of users leading their management. While this is riot the most common marketing strategy in the commercial world, it is typical of Bell Labs’ “bottom-up” organization. According to Rudd Canaday, now head of Bell Labs’ Artificial Intelligence and Computing Environments Research Department, change within the Labs often comes from the people doing the work. “UNIX spread throughout Bell Laboratories because people loved to use it,” he said.

Canaday first experienced UNIX, although it hadn’t yet been named, while working with Thompson and Ritchie. He left that group and later became head of a group building large, mainframe-oriented systems. Because of his previous exposure to UNIX, he wanted to bring it to his new group.

Canaday found lots of support among the programmers who had already tasted UNIX. One of those, Richard Haight, recalled, “I was trying to get my management to get UNIX and we dreamed up the idea of using it as a common timesharing interface to different kinds of host computers.”

Initially, the project involved interfacing with big IBM and Univac machines, and later expanded to interfacing with RCA, Xerox, and others. The basic idea was to edit programs and work with files under UNIX, but instead of compiling and executing under UNIX, you could send the remote job off to a big machine. This way, the programmer didn’t have to deal with complicated IBM JCL sequences since he could just give the UNIX utility the parameters it needed to know. The masses of printout could then come back as a file under UNIX and, as Dick Haight put it, “save cutting down a tree.” It also saved having to retrain programmers for a variety of host systems.

This original Programmer’s Workbench system was built on a PDP 11/45. The system eventually offered lots of utilities, including ones for analyzing host machine dumps on the UNIX system.

While work proceeded on the PWB system, an interesting discovery was made. The designers had assumed that the majority of the work cycle would involve the host computer. Users were thought of as editing a file, sending it to a host computer, getting the print file back, looking at it, and doing that over and over again. As it turned out, samples taken from different kinds of work groups on different systems showed people tended to use the text formatter, nroff, five times as often as they submitted Remote Job Entry programs.

This unexpected result might not have happened had UNIX not had fairly sophisticated word processing facilities available to programmers. The original development for Bell’s legal department could hardly be called “incredible foresight”, but happily for UNIX, word processing was to become the single most commonly used computer application. Once the facilities were there, programmers made massive, unexpected use of them. This happened, according to Haight, because programmers have to be able to document programs on the same machine used for development. “Things like pipes and the power of the shell are not to be slighted, but what’s really important is the fact that you can do your documentation and your programming on the same machine,” he said. You can be editing your documentation and break away from that to edit the source. When you’re finished with that, you can submit a compile in the background and go back to editing your documentation while the compile happens.

Flushed with the success of the PWB and the Remote Job Entry facility, Canaday and his group set about showing people what was possible. Once the users were convinced, Canaday said to management, “Well, if you want to keep on using this, you’re going to have to start buying machines to do it.” He knew that “once you let people get their hands on UNIX, they just won’t let go.”

A key piece in the rapid spread of UNIX within Bell Labs was the low price of minicomputers relative to mainframes. A department head’s urging was generally sufficient for purchase of a VAX. Mainframe purchases were considerably more sticky. A VAX had sufficient power to reasonably serve the needs of a department, so VAXen became increasingly commonplace.

More and more departments were becoming convinced that UNIX was part of the path toward improved productivity. According to Canaday, no one had yet been able to provide statistical evidence showing that timesharing was more productive than batch processing. In all the studies, the difference between individual programmers was greater than the difference between timesharing and batch processing. But the subjective evidence was that timesharing was superior. That evidence, in time, prevailed. Regardless of any “proof’, the users wanted UNIX and their management gave it to them. What has since become a universal pattern for the spread of UNIX was already underway. As Canaday put it, “The reason people are so devoted to UNIX is that Ken and Dennis wanted to build something that they would enjoy using, and they succeeded.”

Dick Haight agrees. “I don’t believe UNIX is Utopia. It’s just the best set of tools around. I think the reason they’re as good as they are is they weren’t invented by some committee sitting down and saying ‘we ought to have an editor and a move command and a copy command and a diff command.’ It was done by people who needed these things and who worked together in a little room where they could complain to each other immediately when something was done that was inconsistent with the spirit of things. Right from the start, Thompson and Ritchie put people on UNIX who had real work to do. In a way, it was done in a vacuum — just their little coterie in Bell Labs research, but these people turned out to be a very representative set.”

Two tracks of UNIX expansion within Bell — the UNIX Development Support department and the Programmers Workbench group — merged in 1975 under the Users’ Support Group heading.

LETTING GO

Somewhat earlier, there also had been a movement to release Research UNIX to the outside world. Universities, in particular, had heard of it and were very interested. Berkley Tague, sensitive to the real requirements of support, opposed the idea. “I was very concerned that some bank in the Midwest would put its payrolls on UNIX and get in terrible trouble. The fact is that didn’t happen. It went to a few technical shops; it went to universities; it was very influential in all the places it should be influential. The people who picked it up were, by and large, people who could deal with the kind of support we offered — or didn’t offer, if you’d like to put it that way. That was when we basically said, ‘Here’s a tape; take it.’

“If you’d asked me at the time if releasing UNIX to the universities was a good idea, I would have said no. I have been very grateful since then, particularly when hiring new people, that no one paid any attention to me.”

---

August Mohr is former editor of the /usr/group newsletter, CommUNIXations. He is currently acting as a consultant and developing in-house utilities for electronic publishing. He is also at work on a book.