July 2008

If you read my other blog and you’re wondering why it’s offline, it’s because the webserver that hosted it (a very obsolete iMac in the corner of my bedroom) has finally died, probably thanks to the four consecutive power failures I had over the course of about three days (hooray, California power grid!)

It might be time to move that one over to WordPress, too, so I don’t have to worry about stuff like this. Blah. First, though, I’ll need to mount a hard drive rescue operation. (I do have backups but they’re at least a month old.)

Also, I may be out of cell phone reception this week, because from Monday through Friday I’ll be at the Mozilla Summit in British Columbia.


Now that I’ve blogged about Ubiquity, you should understand why I’ve been obsessing over the properties of a good linguistic UI. It’s not an academic problem: It’s one of the interfaces to the extension I’m working on right now!

Some commenters have asked me the question (if not in these exact words): Is a linguistic UI the right kind of UI for Ubiquity, and if so, why?

(“Because Jono is obsessed with linguistic UIs” isn’t a good enough reason.)

First, the really big picture of what Ubiquity is supposed to be all about: It’s a step towards a Web where verbs (i.e. functionality, i.e. commands, i.e. services) are first-class citizens. And that’s why I’m thinking it should be renamed from Ubiquity to something like “Mozilla Verbs”, maybe.


Last night was the monthly Mozilla Labs open house for July. Atul, Aza and I gave a presentation about our ongoing project, working title Ubiquity.

What is Ubiquity? It’s a Firefox extension which gives you an easy way to create, share, and use commands — tiny snippets of Javascript that do cool stuff with the Web. Commands can act on web content that you select with a mouse, or on information detected in a page in a microformat, or on arguments that you type in. A command that you create can be shared with others simply by giving them the URL to a Javascript file. Commands will be exposed in multiple ways: You’ll be able to choose a command from the context-menu (the one that pops up when you right-click on web content), but also, for more involved use-cases, there will be a linguistic UI that lets you type what you want to do. We’re still in the very early stage of development, but those are our plans, in a nutshell. Atul gives a much better explanation than I can.


What would the web be like if you could tell it what you want to do as easily as you currently tell it where you want to go?

Mozilla Labs is starting to experiment with linguistic interfaces. That is, we’re playing around with interfaces where you type commands and stuff happens — in much the same way that you can type a location into the address bar in order to go somewhere.

I think this is cool because, for one thing, I think language-based interfaces are seriously under-explored compared to pointing-based interfaces. For another thing, I used to work on a project called Enso. Enso’s a language-based interface, where you type commands in and stuff happens. I think we got certain things right and certain things wrong in Enso’s UI design, so I want to take another crack at doing it better.

What makes a good linguistic UI?

Here’s my current theory.

  1. It’s easy to learn.
  2. It’s efficient.
  3. It’s expressive.

Those are the three “E”s. Let’s unpack ’em a little.


First, watch this video and laugh:

Writing a Perl program with text-to-speech

Cue discussion of how voice-activated interfaces are a terrible idea, that real computers will never work like computers on Star Trek, that Microsoft’s UI designers were consumed with hubris if they ever thought this would work, etc. etc.

Too pessimistic! Thing is, there’s actually a very simple solution to this problem. It doesn’t even rely on any exotic technologies. We could do it today.

Things that a human says within a computer’s hearing might be:

  1. commands (“Do this”)
  2. content (“Put this text into my document”)
  3. noise (things not intended for the computer at all, such as conversations between humans)

The very same words can be any of these three categories, depending on the intent behind them. Human language is ambiguous! That’s why humans rely on so many non-verbal cues, like tone of voice and facial expression, to interpret what other humans are saying.

Computers can’t interpret like that. But they wouldn’t have to if we just had a microphone with a couple of buttons on it.

  • Hold button 1 and talk: The software interprets your speech as commands.
  • Hold button 2 and talk: The software interprets your speech as content.
  • Hold neither button: The software ignores anything you say.

I’m gonna go out on a limb here and predict that we’ll see a decent voice-activated system within the next few years, that will rely on a non-verbal communication channel (such as a few buttons) to help resolve the ambiguity of speech.

There will still be plenty of applications where people would rather type than talk (think about all the reasons why people would use text messaging on cell phones instead calling up and talking to someone), but once the novelty wears off I think speech-based interfaces will soon be seen as one more useful tool among many.

These things I believe about software development and user-interface design.

1. Why write code?

Software is for humans, not for computers.

Software is only as good as the improvement it makes to a human being’s life.

Are we making someone’s job easier? Letting them have more fun? Helping them learn? Helping them keep in touch with friends and family?

Are we making the world a better place?

2. What do people want?

Most people do not want a computer.

They don’t even want software.

For us software developers, this is a painful truth.

If people don’t want a computer, why do they use one?

  • Email — for writing to other people.
  • Instant messaging — for talking to other people.
  • The web browser — for reading what other people have written.
  • Word processing — for writing something you’re going to print out and show to other people.
  • Graphics — for creating artwork. To show to other people.
  • Presentation — for communicating your brilliant plan. To other people.
  • Games — especially games that you can play online. With other people.
  • Social networking websites — Enough said.

The computer is merely an intermediary. A poor and frustrating one. It is a necessary evil that people put up with in order to get what they want.

What they want is a better way to talk to each other.

3. Why does software succeed or fail?

We software developers, being not exactly social creatures by nature, must work extra hard to understand the social impact our software will have. If the social effect is not what people want, the software goes unused.

We software developers, being not exactly average users, must work extra hard to understand how average users will relate to our software. We see the trees, they see the forest.

We software developers have often been confused and frustrated when a clearly superior technology fails, while a clearly inferior technology spreads like wildfire and takes over the world.

We were surprised because we want each technology to be judged only by its cleverness, its raw power, the cleanliness of its architecture, the purity of its ideas. We were blind to the user experience, to what each technology meant in the bigger picture of a person’s life.

To the people buying and using the “clearly inferior” technology, exactly the opposite was true.

To the user, the interface is the product.

4. Why is there not more Linux on the desktop?

For open source software to take over the world, we’re going to have to do a lot better at user interfaces than we have been doing.

How do I know?

Open source has already taken over the invisible parts of the world: the servers, the infrastructure, the things users need not touch directly.

Mozilla, the most user-experience-focused of open-source companies, has the most adoption by end-users.

People say things to me like, “Linux is only free if the value of my time is zero.”

These are not coincidences.

At one time, the way of open-source software development was thought impossible. But the techniques were invented. The way became possible; then it became successful. Now the techniques are becoming widely known.

The way to make open-source UI design successful is still unclear. We must invent the techniques.

5. Are users dumb?

User interface design is not about dumbing things down for the poor stupid user.

We software developers, understanding the software as we do, find it easy to look down upon those who lack our understanding.

This is wrong.

Users aren’t dumb. They just have better things to do with their lives than memorizing the internal data model of our screwy software.

When software is hard to use, don’t make excuses for it. Improve it.

When a user makes a mistake, don’t blame the user. Ask how the software misled them. Then fix it.

The user’s time is more valuable than ours. Respect it.

Good UI design is humble.

6. Is UI design marketing?

User interface design is not marketing.

Software developers loathe marketing, so if they think that UI design is marketing, then they will loathe UI design.

The qualities of software that make for a good advertisement or computer-store demo are not the same qualities that make software usable and pleasant to work with long-term, day-in day-out. Often these qualities are opposites.

A shopper may choose the microwave with more buttons, because it seems “more powerful”. However, the shopper will soon find out that it does the same thing as any other microwave, you just have to spend longer figuring out which button to push.

It is easy to fool people into buying something that is against their own best interest.

Don’t do that.

7. What is the task of the UI designer?

Let us talk about that microwave some more.

The microwave with the most buttons may be most popular, but it is not the best microwave.

The best microwave has no buttons at all.

It doesn’t need any buttons because it already knows how long you want your food cooked and how hot. You never need to set the clock, either: it’s just always right.

The no-button microwave may not be reachable, but like a guiding star it shows us the direction we should travel.

Users do not know what interface they want. Users do not know what features they want.

Users know the tasks they want to do, and the problems they have.

We learn more by watching the user work than by asking the user.

The job of the UI designer is to provide what the users need, not what the users say they need.

It is to make tasks easier, not to provide features.

8. Where is the science?

User interface design can be approached scientifically. But usually isn’t.

Until we observe people using our software for real, our design is guesswork and superstition.

These things can be measured and given numbers:

  • What program features are being used most frequently, and least.
  • The number of mouse/keyboard interactions required to perform a task.
  • The time it takes a user to figure out how to do a task.
  • Rates of error.
  • How quickly task-completion-time and error-frequency decrease as a user gains experience.

An interface’s efficiency and learnability are empirically determinable quantities.

They are not matters of opinion.

Every user is different, but that’s why we have statistical methods.

The science of design can tell us that interface foo is X% more efficient than interface bar, but bar is Y% more learnable than foo.

Choosing between foo and bar — that’s where the science ends and the art begins.

9. Is change good or bad?

Change has a cost. Change disrupts the user’s habits. Change forces the user to learn something new.

Sometimes the new UI is so much better than the old one that the change is worth the cost.

Sometimes it isn’t.

The trick is knowing when change is worth it.

10. What is the evil of the bad interface?

It is a sin to waste the user’s time, break the user’s train of thought, or lose the user’s work.

Bad user interfaces do all three. Frequently.

Most interfaces are bad.

I do not use the word “sin” lightly.

Because of bad user interfaces, an action taken based on a reasonable assumption or out of habit often results in broken trains of thought, wasted time, and lost work. This is called “user error”, but it isn’t. It is programmer or designer error.

When we blame the user, we teach them that technology is perfect and that the errors are their own. Because technology is hard to use, we are teaching a generation to be afraid of technology. We are teaching a generation to believe in their own stupidity. This is a sin, too.

It’s not the user’s fault.