The Social/Psychological Side of Software Architecture (my talk at reboot7)

Talk delivered at the reboot7 conference, June 11, 2005, Copenhagen, Denmark.

So my name is Peter Lindberg. Some of you might know me from my weblog, which is called ‘Tesugen.’ I started it about three years ago, without any particular ambition, but I soon found it to be the perfect medium for me to express my thoughts. So I’ve been writing a lot about things I’m interested in. Which is theories, systems, noticing recurring patterns, psychology and cognition, and so on. In some strange way, much of what I write about comes back to software development; I come to think of ways in which whatever I’m thinking about is like software development. I will share some of those things here today.

I work as a software developer at a small consultant company in Sweden called Oops. I’ve been a professional developer for twelve years, but I started programming as a kid. Today, I develop software on Apple’s platforms for the web and for the desktop.

Anyway, my talk here at Reboot is about software architecture. The title I decided upon was ‘The Social/Psychological Side of Software Architecture,’ but as I worked worked with the manuscript, I began thinking of it as ‘The Three Pillars of Software Architecture,’ for reasons that soon will become apparent.

These are things I’ve been thinking about for the last four or five years now. What caught my interest at the beginning was that I encountered the term ‘software architecture’ more and more often, but I didn’t quite get what it was about. The explanations I heard were along the lines of ‘It’s the structure of a system,’ or ‘It’s the skeleton upon which you “hang” the code,’ or ‘It’s the system seen from 10,000 feet.’

And everyone said it was important.

‘Without an architecture,’ they said ‘chaos will ensue. You will never end up with order, unless you establish that order from the start.’ And the motivations were purely technological. ‘If you take architecture seriously,’ they said, ‘your system will end up being flexible, robust, of high performance, maintainable,’ and so on. Something bugged me about this, and I couldn’t put the finger on what it was.

So I began reading about software architecture; books, mailing lists, websites—and eventually I began reading about ‘real’ architecture, ‘brick-and-mortar’ architecture, to try to find out why someone had thought it a good idea to use architecture as an analogy for software development. This led me to read about all kinds of things, seemingly unrelated to software architecture, but which seemed to say interesting things about it.

But this isn’t a talk about what software architecture is, or whether it’s good or bad to do it this way or that. This is a speculation into what might be missing in the talk about software architecture, using inspiring examples from various other fields such as cities, buildings, and the human memory.

Let’s start with how software architecture is commonly defined. Here’s a definition offered by Len Bass and others, in their book Software Architecture in Practice. It goes:

The software architecture of a program or computing system is the structure or structures of the system, which comprise software elements, the externally visible properties of those elements, and the relationships among them.

So software architecture is about the elements, or components, of a piece of software, and how these interact. That is, how they use other components and how other components use them. So it’s about structure, about things and the interplay between them.

The definition offered by Len Bass and his crew, has been selected by the Software Engineering Institute as a representative example of modern definitions of software architecture. They also list a number of ‘classic’ definitions—which to them means they date back to the 90s. But I wanted to go further. I wanted to know who first introduced this analogy. And I think I found it.

There’s an obscure book from 1962, recording the experiences from the ‘Stretch’ project at IBM, which would be the company’s first supercomputer, designed and built for the labs at Los Alamos. In this book, there’s a text titled ‘Architectural Philosophy,’ written by Fred Brooks, who later published the classic book The Mythical Man Month. In the book from 1962, Brooks writes that

Computer architecture, like other architecture is the art of determining the needs of the user of a structure and then designing to meet those needs as effectively as possible within economic and technological constraints.

So this is a definition which originates in the role of the architect as an intermediary between the future user of ‘a structure’ and the engineers. Brooks stresses that in architecture, the emphasis is ‘on the needs of the user,’ whereas engineering emphasizes ‘the needs of the fabricator,’ building the thing as cost-effectively as possible. This is a definition he maintains twelve years later, when The Mythical Man Month is published. There he also defines an architecture as ‘the complete and detailed specification of the user interface.’

So originally, software architecture was about representing the user, about presenting the needs of the user to the engineers. But when one reads the modern definitions of software architecture, it’s all about the design of the internal structure. This seems only indirectly related to the needs of the user. If architecture is about representing the user’s interests in a team of engineers, shouldn’t architecture documents describe the user interface, in the form of UI mock-ups, stories about how different types of users would use the software, and so on?

My theory is that since the late 1950s, early 1960s (the time of the IBM ‘Stretch’ project) the user has changed. Then the user was usually an engineer or scientist, anyone in a lab coat. And the user interfaces were all buttons and dials, punch cards, line printers. Today, the user isn’t an expert in using computers. And the user interfaces are a lot more friendly. But first of all, the internal complexity of software programs has exploded. So today there’s more of a disconnect between the UI and the internals than was the case forty years ago. Today, designing the UI is certainly a challenge, but of a completely different kind than designing the internals. So in a sense, the description of the internal structure for a system to be built can be seen as being indirectly about the needs of the user, in a time when the internal, invisible parts have been separated from the external, visible parts.

But perhaps in this separation, something has been lost. In this explosion of complexity, and the ensuing shift in emphasis from the software architect as advocate for the user, to the architect as the one who produces structural plans for the internals of software systems. From being about humans to being about technology. So much of software architecture today is about technology. But technology is only one of three pillars of software architecture. The other two concern the human things. They concern the social and the psychological. Collaboration and thinking. Groups and individuals.

This talk is about the other two, the forgotten pillars.

Let’s consider the social pillar. Imagine a software project. A project that’s about to start. There are a couple of people who will form a team; developers, UI designers, testers perhaps, and someone to manage the project. Someone has an idea for a system—either someone who will be on the team or someone external to it, perhaps another employee of the same company, or a client of the company. Imagine that these people have their first meeting. The point of this meeting is to pitch the idea to the others, who at this point don’t have a clue what it will be about. So the meeting begins and the person with the idea stands at a whiteboard and talks and scribbles, while the others listen and chip in with questions. And gradually during the course of the meeting, more and more of the vision is conveyed to the team. Each of those present begins to form a mental picture of this future system—and for the developers at least, this picture contains the components the system might have, and how signals are emitted from one to another, causing it to wake up and emit new signals. Already at this stage, order begins to emerge—the shape of these components and their interactions—a potential architecture. But at this early stage, there are as many potential architectures as people present. When the discussion continues, and each of the team members express his or her thoughts, these potential architectures will hopefully start to converge, such that the team eventually agrees upon what their architecture will be like.

Somewhere, filmmaker Francis Ford Coppola has said that the difference between a good and a bad movie, is getting everyone involved in making the same movie. I believe this is true for software as well, and yet there’s little discussion within software architecture about how to ensure that everyone on the team envisions the same system. The notion seems to be that if only one creates lots of diagrams, everyone will interpret them in the same way. But that’s not necessarily true. There are systems I participated in developing nearly ten years ago, which I can easily recall how they were structured. I can navigate them in my mind and I remember their pulse, how messages traveled from component to component, even in quite detail. Then there are other more recent systems which are a blur, that I can’t recall at all how they were organized.

Surely these more memorable systems must have been easier for the developers to understand? Newcomers to such a project surely must have found it easier to get a sense of the system’s entirety. And the mental pictures of those involved must have been more aligned, more similar to those of the others. How can we ensure that future architectures are more ‘memorable?’ That they have greater clarity, and are less open to interpretation?

The psychological pillar is closely related to the social one. Now imagine that you’re a developer on this project about to start up. As you listen to the presentation of ideas for the new system, and you begin to envision the interplay of components, your mental picture will fluctuate and evolve. New components come into play, and the shape of the interaction changes. Hopefully at some point, the mental picture begins to settle down. Depending on how well the ideas are conveyed by the person at the whiteboard—and also depending on how experienced you are—this might happen during this first meeting. Otherwise, it will stabilize later, in following meetings. Regardless of when, your mental picture will cover the system in its entirety. When you think about the system, it has an almost spatial quality. You can navigate the system in your mind. You can tag along with messages being sent from component to component.

This ability to navigate the system in the mind is essential. And in this regard, software development shares some traits with other creative activities, such as writing and filmmaking, but which it for instance doesn’t share with painting. And that is the fact that whatever it is that is being created must be retained in memory. The creation isn’t visible as a whole, in its entirety. Painters usually see the whole canvas all at once. Painters don’t have to retain in memory what they have painted so far. Each stroke of the brush is made based on what the painter sees, whereas the programmer has to decide where each addition should go based on his or her recollecting the structure of the system.

This brings us back to memorable systems. Where the social pillar of software architecture concerns architectures evoking similarly-organized mental pictures, the psychological pillar concerns architectures that stick in the minds of the individual team members. If these things were to be studied, one would probably find several factors that contribute to both aspects. So there’s some overlap. But the question is what makes some architectures more easily remembered, and more easily navigable? How can we ensure this in future architectures?

I think that focusing solely on the technological aspect is dangerous. The discussion within the field of software architecture, and in its wider application, enterprise architecture, misses its point when one neglects the human aspects. Software architecture is not just about the end result, but getting to that end result. Shouldn’t software architecture support that process, the creative process? Shouldn’t it facilitate collaboration among those involved? Shouldn’t it facilitate its internalizing by each person involved?

What follows is a couple of books that led me into thinking about the social and psychological aspects of software architecture. These are all books I found very inspiring, and which I hope I can inspire you to read.

The first book is one by Stewart Brand, titled How Buildings Learn. This is the best one of all the books I’ve read on ‘brick-and-mortar’ architecture to find out how relevant the analogy is for software development. If you believe that developing software is like building houses, you should read this book to get a clearer picture of what building houses is about. If you don’t believe it, you should also read this book, as it will throw you into doubt. Brand convincingly shows how buildings are ‘fluid,’ how they adapt to different uses, how they ‘learn’ from their inhabitants. Brand identifies the amenability to such change in ‘vernacular architecture,’ architecture without architects, in houses built by their the people living in them. There’s a great passage early in the book, where Brand quotes someone he talked to at a conference:

‘Porches fill in by stages, not all at once you know.’ The architect was responding to a talk I gave at a builder’s conference. ‘The family puts screens on the porch one summer because of the bugs. Then they see they could glass it in and make it part of the house. But it’s cold, so they add a duct from the furnace and some insulation, and now they realize they have to beef up the foundation and the roof. It happens that way because they can always visualize the next stage based on what’s already there.’

As a software developer, visualizing the next stage is easier when you have a strong and clear mental picture of the system you’re working with. And also, the image of vernacular architecture suggests that, like Jason Fried said yesterday, being close to the problem means you’re better off making decisions about the product, it suggests you should listen to the product itself and not the specifications.

The second one is the book The Image of the City by Kevin Lynch. Lynch, and his students at Yale, conducted a study of how we perceive cities. One thing he did was to let people draw maps of their city. He then compared the maps to each other, to see for which cities the maps were more similar to each other, and which parts of each city people found it easier or harder to visualize. Based on this he identified a number of elements that are important for the perception of a city: paths, edges, districts, nodes, and landmarks. Paths are most important, but all these combine to ease or make it more difficult for people to recognize and remember patterns as they navigate a city. Lynch introduces the concept of ‘imageability,’ which is defined as

that quality in a physical object which gives it a high probability of evoking a strong image in any given observer. It is that shape, color, or arrangement which facilitates the making of vividly identified, powerfully structured, highly useful mental images of the environment.

Applied to software development, imageability raises the question of what it is about an architecture that gives it ‘a high probability of evoking a strong image.’ How do we create high imageability software architectures?

The next book is actually one about software development, Kent Beck’s Extreme Programming Explained. But I wanted to mention one of the less talked about things—the system metaphor. This was one of the twelve practices Beck suggested for software development, and he presented it in the book as something that would guide the design of the system, in the absence of an established architecture. The idea was that you should pick something to serve as the metaphor for the system, and in the original XP project, which was about developing a payroll system, the metaphor was an assembly line, such as in a car factory. The metaphoric car (which I believe was a paycheck in the system) would start at the head of the assembly line as a chassis, and different parts (or items on the paycheck) would be fitted at different stations as it moved along the assembly line. Besides guiding the design, another of the motivations for using a system metaphor was to enable developers and non-developers to discuss the system. So this metaphor was a suitable one, as the project was for Chrysler.

But the system metaphor proved to be difficult. Kent Beck and the others talking about XP had it difficult explaining what the system metaphor was for, how it was intended to work, and perhaps most importantly, how you pick one. So as the second, revised edition of Extreme Programming Explained was published, the system metaphor was dropped. It isn’t mentioned at all. But I think there is a place for metaphoric architectures as a means for aligning people’s mental pictures, and to create more memorable systems. Let’s consider a metaphor that probably is more familiar than the assembly line.

The hamburger restaurant. Imagine being a newcomer to my project, and I say that the system is laid out and functions like a McDonald’s. This speaks volumes about the system’s architecture. The system probably services requests from a number of clients, which are queued and processed one in turn. The back-end of the system has a number of processes assembling objects upon request or anticipation, placing them in queues depending on their configuration, or in a special queue for odd requests. When an object is placed in an empty queue, this is announced by publishing a notification that this has happened. The processes servicing requests checks the queues to see if there’s an object matching what is requested. If there is, it is passed to the requester. If there isn’t, the request is broadcasted to the back-end processes. The front-end process can then begin servicing the next request until the back-end notifies that the previously requested object is now available.

Doc Searls said in his talk yesterday, that a metaphor is like a box of words, but it can also convey very vivid images. And Ben Cerveny also said many interesting things about metaphors and cultural knowledge. And it’s been mentioned in a couple of other talks as well. I would suggest that using a metaphor is like extracting the structure from something, and reusing it somewhere else—and using the thing from where it was extracted to boost the process of internalizing, learning, remembering this structure.

I would hope that despite it being dropped from XP, we could continue thinking about how metaphors can be used to infuse software architectures with vivid images from other areas with which we are familiar.

The fourth and final book is The Art of Memory by Frances Yates. I haven’t read it yet, but I sat down and scanned through it all in an hour. This book is about a time when a trained memory was of vital importance, about 2000 years ago in ancient Rome and Greece. As Yates writes, an ‘ancient world, devoid of printing, without paper for note-taking or on which to type lectures.’ Essentially, what people did to remember things was, in Yates’ words, to ‘imprint on the memory a series of loci or places.’ ‘The commonest,’ she continues, ‘though not the only, type of mnemonic place system used was the architectural type.’ So one would imagine a building, a large house with many rooms, each clearly distinguished from the next, and in these rooms one places objects that evoke whatever it is one wishes to remember. So to give a long speech from memory, one imagines entering the house, walking through room after room noticing objects corresponding to the things one wants to say.

It seems that this technique was used collectively as well. There’s an example in Yates’ book about a court case, where a man is charged for killing another man by poison. Here, Yates says that the defense is ‘forming a memory system about the whole case.’ Which is what software developers implicitly do as they shape the architecture.

I’m convinced that this book would be very interesting to read as a software developer. For instance, there are long passages about how metaphors were used to further vivify memories.

I just wanted to list a few more books I can recommend if you want to learn from other fields.

Christopher Alexander’s The Timeless Way of Building. This is the first book in a series of three about the patterns Alexander invented, the other two being A Pattern Language, and The Orgeon Experiment. Design patterns are familiar to most developers, but I feel Alexander’s original intention has been lost upon us. What Alexander wanted to do wasn’t to invent a handy format for capturing recurring solutions. That was more of a tool. Rather, he wanted to resurrect the lost knowledge of vernacular building. His pattern languages are intended as the vehicle to achieve this in the long run. And on the way to eliminating the architects from architecture, the pattern languages can serve as a tool for non-architects to make decisions about the house they’re having built. Great book.

Le Corbusier’s The City of Tomorrow and its Planning. This is a book that has been an immense influence on city planning. This is interesting to read in conjunction with Jane Jacobs’ The Death and Life of Great American Cities, where she is extremely critical of Le Corbusier’s approach to city planning. Jacobs book shows what happens when one organizes cities after a maxim of pure order, as Le Corbusier advocated in his book. When enterprise software development is compared to city planning, one is actually referring to Corbusian city planning, and not modern city planning trends, in which diversity and mixed-use, as Jacobs advocated, are valued. If you’re going to read one of these two, pick Jacobs’ book.

Thomas Kuhn’s The Structure of Scientific Revolutions. Another great book, which is about the way science progresses from initial hunches, to hypotheses, and to proven theories. This book shows how the community of scientists behave facing new and surprising evidence, which shatters established theories. It was a while since I read it, but I’m sure there are things in it that would say interesting things about the social and psychological aspects of software architecture. A book I haven’t read yet, but which I understand is a rejection of Kuhn’s ideas, is Paul Feyerabend’s Against Method.

My intention with this talk was to present a few books I’ve read that inspired me to think in other ways about software development and software architecture. The writing on software architecture is extremely dull and at many times, it completely denies that software development is about discovery, creativity, collaboration. I hope you have enjoyed this talk.


(Technorati tags: , .)

The above was posted to my personal weblog on June 14, 2005. My name is Peter Lindberg and I am a thirtysomething software developer and dad living in Stockholm, Sweden. Here, you’ll find posts in English and Swedish about whatever happens to interest me for the moment.


Related posts:

Posted around the same time:

The seven most recent posts:

  1. Tesugen Replaced (October 7)
  2. My Year of MacBook Troubles (May 16)
  3. Tesugen Turns Five (March 21)
  4. Gustaf Nordenskiöld om keramik kontra kläddesign (December 10, 2006)
  5. Se till att ha två buffertar för oförutsedda utgifter (October 30, 2006)
  6. Bra tips för den som vill börja fondspara (October 7, 2006)
  7. Light-Hearted Parenting Tips (September 16, 2006)