Ellen Isaacs: Designing collaboratively

Most software applications are difficult to use because the development process emphasizes the quality of the architecture and not the usability of the application. Programs can be both well engineered and usable if software engineers work with human interface engineers who not only design the user interface but advocate architectural decisions that help make tasks more natural for users. Our group used a process that reduced the workload of the software engineers and yet produced a particularly usable and well-engineered application. This process included designing the UI before coding, developing the applications user interface in a UI builder, and regularly testing the application with users under realistic conditions. This latter point was an important achievement because it is critical and yet particularly difficult to test collaborative applications. Carrying out these activities enabled the software engineers and human interface engineers to work closely together as a cooperative team because everyone was able to contribute valued components to the effort.

Introduction

The software market is dominated by products that are technically sophisticated but difficult to use. These products reflect the process by which software is developed. When developing a product, software engineers typically start by deciding what functionality should be provided, creating the feature list. The bulk of the time is then spent designing and implementing the software architecture to support the chosen features. The quality of the architecture is usually judged by criteria such as exibility efficiency and modularity. The user interface is seen as a front end, which usually exposes the features of the underlying architecture to the user. In contrast, human interface engineers recommend a user-centered design approach [Shn92]. HI engineers are people who are trained to understand user's needs and to design, but not necessarily code, software to meet those needs. HI engineers represent the needs of the user at all stages of the project. They interview and observe prospective users to gather input for the initial feature list. They ask how the product's functionality should be presented to the user, designing common tasks to be easier and more accessible than rare ones. They emphasize iterative design, during which early versions of the product design are tested by users and redesigned in response to feedback. They also do the detail work of laying out the user interface so that it is clear and attractive.

There is no inherent conflict between conducting user-centered design and producing quality software. Flexibility, efficiency, and modularity remain important concerns. The problem is that in many software teams they are the only concerns. Usability is seen as relatively unimportant, something that can be added on as needed. Perhaps part of the reason the user-centered approach is not widely adopted is that the HI community has not justied its process in terms that are compelling to the SE community.

Although most software engineers would no doubt prefer to develop more usable applications, they are concerned about the time and effort of doing so. When software engineers or managers hear about iterative design they worry about more effort and longer schedules. On the other hand, if the SE community came to understand that integrating HI principles into the process of developing a product would reduce their effort as well as improve the usability of the product, the reason to work closely with a human interface engineer would become more compelling.

As our group has developed an application for collaborative use, we have done four things that we believe not only improved the usability of the application, but also reduced the time and effort to produce it. Namely, we:

separated the role of user interface designer from that of implementer designed the bulk of the user interface before any coding began,
developed the user interface in a UI builder, and,
tested the application regularly in real usage situations, using the results to revise the design and code.

The last point is particularly important for collaborative applications because their success critically depends on designing them to be useful to and usable by many people, and yet they can be especially difficult to test [Gru88]. Many development efforts do not include these four components, although we believe most could benet from doing so. We found that carrying out these activities enabled our group to work together tightly as a team, even though we were not in the same organizational group. Each member of the team could not only spend more time working on aspects of the application they cared about most, but each member could also make contributions that were valued by everyone else. The following explains how we carried out each of these activities and how doing so improved the quality and efficiency of developing our application. First, however, we briefly explain the application we have been developing.

The Application: Forum

Forum is a UNIX-based distributed application that enables speakers to broadcast presentations over the network and audience members to participate in the presentations from their workstations. The audience receives live audio and video of the speaker as well as slides, see Figures 1 and 2. They can interact with the speaker in one of three ways: they can use audio to ask questions, they can send in written comments, and they can vote on multiple-choice questions raised by the speaker. The speaker can point to, write on or draw on her slides and her gestures or marks are seen by the audience. (See Isaacs, Morris, & Rodriguez, 1994 [Isa94] for a complete description of Forum.) Our development group consisted of two software engineers, one human interface engineer, and a graphic designer who worked part time on the project. The project itself was an advanced development effort, which meant that we did not have to undergo the usual processes associated with releasing a commercial product. This gave us more freedom because we did not have to remain consistent with other efforts around the company or meet predetermined schedules. However, we produced the application in about as much time as would be allotted to a product of this size. The following four sections describe the aspects of our development effort that we think contributed to its success.

Separating UI Design From Implementation

The majority of user interfaces are designed by software developers who double as user interface designers. The notion of hiring a person whose job is to design and test the user interface and not to write the code is still a novel idea in some parts of the software industry. At the 1994 workshop on Software Engineering and Human-Computer Interaction, conducted in conjunction with the 16th International Conference on Software Engineering, most of the software engineers thought of a user interface person as a person who wrote either UI code or toolkits and tools to help generate UI code. Only a few considered it a full-time job to develop a user model and design an application's look and feel. Our experience is that a person who is responsible for both designing and implementing the user interface will weigh implementation concerns more heavily than usability issues. They may settle on an obvious implementation without considering subtle usability issues. In cases when usability and implementation goals conflict, they may not work as hard to think of alternatives that satisfy the underlying user need and (??) in with the implementation approach. In contrast, a human interface engineer thinks only about the usability implications and, while knowledgable about implementation issues, isn't constrained by them. Many times during the development of Forum, the HI engineer pointed out problems that did not appear to be important but caused users great frustration. For example, Forum audience members can view any of the speaker's slides during the presentation. They simply click on a thumbnail representation and the slide appears in the main region (see Figure 2). However, the initial implementation made it difficult to tell when the system had finished drawing a slide. When a user tried to display a particularly complicated slide, the busy feedback often indicated that the new slide had been drawn long before it actually appeared. This feedback led users to believe that their command had been ignored and that they could take another action. This unresponsiveness is not only frustrating, but it can cause users to form incorrect theories about how to get the behavior they want. When the HI engineer pointed out the problem, the developers redesigned the code so that Forum could provide correct status information.

Figure 1. Forum's audience interface. Video of the speaker is shown on the left. On the right, a poll has been taken and an audience member is asking a question as others wait in the queue.

Our HI engineer did other tasks as well. She was responsible for understanding how users approach the task being supported in Forum, i.e. giving and attending presentations, and for prioritizing the range of possible features. She designed the organization and behavior of the application, and with the help of a graphic designer, laid out the controls. This is not to say that she had complete control over the look and feel. The software engineers not only provided implementation constraints but also gave their own opinions about the clarity of the design.

Figure 2. Audience's slide interface. Users may click on a thumbnail on the right to see that slide in the main region.

Designing Before Coding

When developing applications, the coding is most often done either in parallel with or even before the UI design has been completed. As a result, implementation decisions are often made without an understanding of how they will ultimately affect the UI. Later, what appear to be very simple UI demands cannot be met easily. Since it is too late to modify the underlying architecture, a fair amount of time is spent figuring out how to either modify the code or adjust the UI in other places to resolve the problem -- which can mean more coding work. Or the problem is left unsolved, reducing the usability of the application. When developing Forum, our group conducted a series of meetings to determine the functional requirements, after which the human interface engineer and graphic designer worked together for a little over a month to design most of the user interface. As various components were worked out, the software developers made sure not just that the design could be implemented but also that it remained true to the group's concept of the application. Once development began, the developers did not even consider implementations that would not accommodate the look and feel. Having a UI design in advance did not mean that the UI concerns always determined the implementation decisions. Instead, when the developers discovered implementation issues, the entire team discussed the implications not just for the code but also for the user's experience. The group jointly decided whether the amount of coding effort involved was worth the UI gain on any given decision.

For example, according to the original design, anyone watching a presentation from her desk could speak just by indicating the intention to do so. However, it quickly became apparent that having an open mic caused too much audio echo; the sound coming from an audience member's speakers fed back into her mic and reverberated through the system. The developers tried to automatically adjust audio levels at different parts of the system, but were unable to handle the many unpredictable variables that affect the volume perceived by users. We decided that the UI had to be adapted to require users to hold down a widget when they wanted to speak. The UI designers spent considerable effort trying to make this requirement as easy on the user as possible.

On the other hand, when designing the UI for the speaker, the developers wanted to put constraints on the UI to avoid the need to replicate code. However, after watching speakers give talks, it was obvious to everyone that just delivering a presentation puts a heavy cognitive load on speakers; it is unrealistic to expect them to spend more than minimal effort manipulating the UI. The software engineers changed the architecture of the code to accommodate speakers' needs.

We are confident we saved at least the month or so we spent designing the UI up front because we spent little time patching code that couldn't handle situations that emerged later in the process. The resulting application respected the fact that it was both driven by a vision of the users' tasks and built with a consistent underlying architecture. And, unlike many other projects, most of the originally planned features were ultimately included in the final implementation as well as some that we hadn't originally planned to provide. All of this was done without the contention that can emerge between software engineers and HI engineers when they have to argue about how to spend the limited amount of developers' time and effort.

Using a UI Builder

We made the decision early on to use the code generated by the Devguide user interface builder. This had many advantages as well as a few drawbacks. First, the UI designer could produce code, which meant not only less work for the developers but also that a larger percentage of their time could be spent on the more challenging, less tedious aspects of programming. Meanwhile, the UI designers, instead of being perceived as putting demands on the project, directly contributed to it, generating about 15% of the code. Second, the developers felt that by prototyping with Devguide, the UI designers tended to have more realistic demands than they would if they had used an unconstrained drawing tool; in fact, if anything, the software engineers could implement more than the tool could prototype. Third, it enabled the UI designers to spend more time refining design details than is usually available to developers who must code more than just the UI. Finally, as the design evolved and as use study data became available, the UI designers could make small, low priority adjustments to the UI that are often overlooked if the software engineers have to work it into their schedule. The result was a more visually and functionally polished application.

There were many ways in which Forum benetted from having the HI engineer spend time on the details. Layout is one obvious case. There are well-understood principles for using blank space and alignment to highlight only the relevant information in a display. For example, similar elements that are spaced irregularly suggest groupings that do not in fact correspond to functional distinctions. Such a layout is confusing and distracting. During the development, we frequently added, changed and removed widgets in response to user feedback. Each time, the HI engineer redesigned the layout so that the design continued to reinforce the functionality of the controls. This work was often time-consuming and required attention to details that, in the aggregate, determined the clarity of the design.

To give another example, the HI designer made sure that the speaker's and the audience's interfaces stayed in sync as we updated and changed them. At one point, we developed a metaphor of raising one's hand to indicate the desire to speak. Settings on the audience's interface allowed them to raise and lower their hand, which added or removed their name from a list of people waiting to speak (see Figure 1). The speaker's interface, however, showed only the names in the list and not the Raise hand and Lower hand settings. The UI designer made sure that the label of that list was updated so that the speaker understood the hand raising metaphor. This again is a small point, but a necessary one because it enables speakers and audiences to use the same terminology when referring to their actions.

We were able to use a UI builder so successfully because Devguide itself is among the few UI builders we know that is written from a non-programmer's point of view. With the exception of NextStep's Interface Builder, most such products, at least for the UNIX environment, require the user to know about the structure of the application and often require them to write code. For example, TeleUse, Xdesigner and UIMX require a signicant understanding of the Motif toolkit and many do not have tools to do certain basic layout operations. Devguide enables the user to concentrate on the look and behavior of the application without having to concern herself with the underlying implementation.

On the other hand, there were drawbacks to using Devguide. For example, the developers were not always happy with the code it generated. Although Devguide generates C++ code, it was not written in the same style as the rest of the application. They decided to write preprocessor macros that transformed the generated code into the style they wanted. This process required initial effort up front and some maintenance work throughout the development. There were other minor annoyances that made it more difficult to use Devguide to generate code. And certain behaviors that we expected the HI engineer to be able to congure easily turned out to be difficult or impossible to do in Devguide. In those cases, a developer had to write the code manually. Even with these disadvantages, the developers felt that using the UI builder saved them a signicant amount of coding time and allowed them to focus more energy on the more intellectually challenging aspects of implementing the application. The user interface designers felt empowered to spend time polishing the UI and so were satised with the look and feel of the application.

Testing Regularly and Realistically

The most signicant way in which developing Forum differed from our previous development experiences is that we tested it on a regular basis in a natural environment. Many applications, if they are tested with users at all, are tested near the end of the development cycle in relatively contrived ways. Users who are hoped to represent the target market are invited to a lab, where they carry out activities that are hoped to represent the typical user's tasks. They have no other distractions and they are often asked to imagine themselves in the situation that would require them to use the software. If fundamental problems are found, it is often too late to fix them for the upcoming release. This type of testing tends to be successful in finding the most glaring usability problems, but many HI professionals recommend combining the usability testing methodology with other techniques to find the full range of usability problems [NP93, JMWU91].

Testing applications for collaborative use can be even more difficult because doing so requires finding groups of people who work together and designing tasks that motivate them to use the application in a realistic way. There may also be logistical obstacles to setting up a network of test machines. Yet it is particularly important to test collaborative applications to make sure they are usable and useful because most such products cannot be successful unless a critical mass of people adopt them [Gru88]. Since our application was designed to enable people to participate in live presentations from their desktops, we decided to broadcast talks to a small set of people around the company, and observe how and even whether they used it.

At first, when the application had a minimal amount of functionality, one of the group members would give a talk about Forum itself. This enabled us to test whether the application would run with more than a few users as well as whether the UI was comprehensible. Once we were fairly sure Forum would run reliably, we scheduled a regular weekly series in which people around the company gave presentations on topics of their choice. Audience members signed up for the talks that interested them. Each time we conducted a presentation, we videotaped the speaker and at least one audience member, who was asked to do whatever they would normally do when watching. The HI engineer reviewed the tapes and showed highlights to the rest of the group. She also sent out email surveys to audiences and interviewed both speakers and audience members to learn about their reactions. This process of testing in a natural setting enabled us to get rich feedback about how people used Forum, and it helped us find bugs that were revealed only when it was used by large groups. The video tapes were an especially powerful tool because they provided concrete examples of user behavior. When we discussed design modications, we were able to avoid debates based on different intuitions about users' tendencies because everyone on the team had a common frame of reference.

Also, since the software engineers added features throughout this process, we could isolate the effect of each new feature, again both in terms of users' behavior and the quality of the code. For example, we had originally included an all-purpose mechanism for getting feedback from the audience. It was a dynamic line graph that showed the proportion of users who had pressed their Feedback button over time, much like a seismograph. We expected speakers to use it either to ask the audience a question, e.g. "How many of you have used Forum before?" or to monitor the audience's reaction, e.g. "Press your Feedback button whenever you don't understand what I'm saying." As it turned out, speakers mostly used it for two-option questions, which required asking the first part and watching the graph rise to its peak, and then asking the second option and estimating whether the graph peaked at a higher or lower point. Consequently, we changed the meter to a yes/no poll meter, which allowed the audience to vote on either option at once and for everyone to compare the vote counts of each option (see Figure 1). Subsequently, we found that people often did not unpress the button from their previous vote, so we enabled the speaker to clear the meter before asking another question. We would have not been able to anticipate how groups of people would use the feedback mechanism without using it under realistic circumstances.

There were some important drawbacks to testing regularly. Since the code had to be in working order on a weekly basis, the developers could not break large chunks of code as they implemented new features. Instead, changes were made incrementally, which required a certain amount of busywork. There was also a certain amount of stress associated with having weekly deadlines, which always generated a flurry of activity in the hours before. Although we now believe we could have avoided these problems had we better organized the process, we still think that even with these drawbacks, testing the application during development paid off handsomely. We essentially achieved in one release the kind of quality that normally is reached after a second release, which is usually the first to be informed by feedback from the market. And we did so without being constrained by concerns about backward compatibility or violating users' expectations.

Conclusion

In the software industry, it is probably fair to say that a product's functionality, performance and reliability are still valued more than its usability. However, in the consumer products industry, there has been a trend toward products that are more usable and less laden with fancy but rarely used features [BW91; Pet94]. This development suggests that, as the proportion of non-technical users become greater, we will begin to see an increase in the demand for usable products. As that happens, there will be a stronger need for software engineers to work closely with human interface professionals to design user-centered software. This can be a positive development if we recognize it as an opportunity for both types of professionals to focus on their strengths and gain from each others' expertise.

Our experience has been that it is helpful to design the UI first, use a UI builder and test the application regularly during development. Although we were part of an advanced development group, all of us have worked on product teams in the past, and we see no reason why this process couldn't work in a product development setting. Perhaps the biggest stumbling block would be justifying the time spent on design at the beginning, just when developers are eager to start coding. This problem could be avoided if the HI engineers started working on a new release while the developers fixed bugs from the current release. Or in cases of new products, developers could start coding those parts of the system that are least likely to affect the user. There is no doubt that we expended additional effort to accommodate the user-centered design process, but doing so ultimately saved us more time and effort and resulted in a better application from both the SE and HI perspectives. This process was successful because it enabled the group to work together as an integrated team, with each member able to make important contributions to the project.

References

[Gru88] Grudin, J., Why CSCW Applications Fail: Problems in the Design and Evaluation of Organizational Interfaces, Proceedings of the Conference on Computer-Supported Cooperative Work, Portland, OR, September 1988, pp. 85-93.

[BW91] I Can't Work This Thing! Business Week, April 29, 1991, pp. 58-66.

[Isa94] Isaacs, E.A., T. Morris, and T.K. Rodriguez, A Forum For Supporting Interactive Presentations to Distributed Audiences, Proceedings of the Conference on Computer-Supported Cooperative Work (CSCW `94), October, 1994, Chapel Hill, NC, pp. 405-416.

[Jef91] Jeffries, R., J. R. Miller, C. Wharton & K. M. Uyeda, User Interface Evaluation in the Real World: A Comparison of Four Techniques, Proceedings of Conference on Computer-Human Interaction, New Orleans, LA, April-May, 1991, pp. 119-124.

[NP93] Nielsen, J. and V. L. Phillips, Estimating the Relative Usability of Two Interfaces: Heuristic, Formal and Empirical Methods Compared, Proceedings of INTERCHI Conference, Amsterdam, The Netherlands, April, 1993, pp. 214-221.

[Pet94] Peters, Tom, Make Your Product Stand Out, Pay Close Attention to Design, San Jose Mercury News, May 16, 1994, p. 3D.

[Shn92] Shneiderman, B. Designing the User Interface: Strategies for Effective Human-Computer Interaction, Reading, MA, Addison-Wesley, 1992.

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specically the rights of translations, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfims or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer-Verlag. Violations are liable for prosecution under the German Copyright Law.