Not The User's Fault

We talk a lot about Ubiquity being easy to extend, what with the ease of writing your own commands and so on.

But did you know that Weave is also built to be extended?

Weave uses a generic syncing algorithm which uses plug-ins called Engines to handle different data types. The following engines are built into Weave:

• Bookmarks (fully functional)
• History (fully functional)
• Stored Passwords (currently being revamped for 0.3)
• Tabs (being revamped)
• Form Field Entries (being revamped)
• Cookies (being revamped)

But since the syncing algorithm is fully generic, it will work with any data you give it. Therefore, you can write a sync engine to synchronize any data you want, as long as that data is accessible from Mozilla extension javascript. Your sync engine just has to extend some existing Javascript prototypes and add some logic for reading, writing, and updating your data type. The Weave core will do the rest.

It’s more work than writing a Ubiquity command, but not as much as writing a whole new extension.

Last night I finished writing the Weave client-side API documentation that describes, in excruciating detail, how to write your own sync engine.

I hope having this API documented will inspire some people to try their hand at weaving together their own data types!

(If there’s any part of the documentation that doesn’t make sense, leave a comment or visit #weave on irc.mozilla.org and let us know how it could be made clearer.)

I saw a great presentation yesterday by Patrick Dubroy, who presented some preliminary results from his research on how people use tabs in the real world.

[UPDATED: You can now read Patrick’s whole presentation on his blog.]

The sample size was small (22 people) so we should be cautious about any conclusions we draw from this study, but it’s pretty exciting to have any real scientific data at all on this question (as opposed to anecdotes, personal observations, etc.) The study also points in some exciting directions for further research.

The big thing I took away from Patrick’s presentation was that among heavy web users, tabs are enabling new styles of browsing behavior that rely less on bookmarking and less on the back button. According to Patrick, the back button is getting used less and less as the years go by, and for all but two of his test subjects, switching tabs was a more common action than hitting “back”.

This change seems to coincide with the rise of web applications, where a user might spend a long time interacting with data on a single page. Web applications live comfortably in tabs (I always keep gmail open in my leftmost tab, for instance) but did tabbed browsers help popularize web applications, or did web applications help popularize tabbed browsing, or neither?

One anecdote from the presentation really stuck with me: apparently some users love opening multiple links in separate tabs, but they use a laborious manual workaround to do so, because they don’t know about command (or control) -clicking a link to open it in a new tab. This tells me that the feature needs to be more discoverable somehow.

Aza interviews me about Ubiquity and Weave. I do a bad Frank Zappa impersonation and some air guitar.

This is a tutorial I made to walk new command developers through the process of writing a Ubiquity command and sharing it on the Web.

Ubiquity Command Development Tutorial 1 (on Vimeo).

You’ll notice that I break my own rule here and give the command a hyphenated name. That’s because I did the recording several weeks ago, before I started working on the naming conventions stuff, and only today got around to finishing the video editing. I’ll fix the command’s name when I make part 2 of the tutorial, which will explain how to use noun-types, asynchronous requests, and other advanced stuff.

EDIT: If the vimeo link above doesn’t work for you, try this one instead.

Many commenters on my previous posts have already guessed what sort of naming standard I’m going to propose, or have suggested very similar ideas of their own. (I’ll take that as a sign that my idea is a natural evolution of the Ubiquity interface and not a controversial mutation.) Here’s the standard I propose:

1. Multiple words are allowed, separated by spaces.
2. The first word must be an actual verb.
3. It’s OK for more than one command to use the same initial verb.
4. The name should describe what the command actually does as specifically as possible.
5. The name should not be longer than needed to fullfil these requirements.

To make this possible will require several changes in the way Ubiquity parses input and in the way it subscribes to command feeds.

I’ll get to those changes in a minute, but first, here’s a representative sample of what commands from the standard feeds would look like using this standard:

(more…)

To review my previous two posts:

Ubiquity needs a naming standard.

The standard must provide an attractive alternative to hyphenated-phrase-command-names. It must allow unambiguous command names, which means it must support multiple words in a command name. It must provide namespacing, and reduce competition for common verbs. The parser must be able to split up the words so you can enter them in any order that makes sense to you.

As if those weren’t tricky enough requirements, I’m adding one more: It must not increase the number of keystrokes that the user has to type to invoke a command.

For example, if part of the standard is “must start with a verb” — so that the “weather” command was renamed to “check weather” — it would be very bad if the user had to actually type “c-h-e-c-k–w-e …”. That would be forcing the user to do more typing for little or no gain. We need to make sure that the user can still type just “w-e-a” and have it complete to “check weather”.

Yes, I want a natural language interface, but that doesn’t mean I think we should be enforcing a strict grammar on people at the cost of efficiency. In real life people speak in fragments all the time, or start and stop in the middle of sentences, so a humane interface should allow the same.

In my previous post I pointed out the problem that is the lack of standardization in command names.

My idea when originally designing the Ubiquity parser was that each command would be named with a single word, always a verb, and that this would allow user input in something approximating natural language, without being too difficult to parse.

However, now that several months have passed and a healthy library of commands has been created, I think my original assumption was wrong. Looking over the list of built-in commands, I find that:

23 are verbs: bookmark, calculate, close, convert, define, delete, email, exit, help, hilight, map, print, redo, refresh, restart, save, search, tag, translate, undelete, underline, undo, zoom

9 are nouns, adverbs, or adjectives: back, bold, forward, fullscreen, home, italic, sparkline, tab, weather

10 are names of websites and services: bugzilla, digg, flickr, google, imdb, tinyurl, twitter, wikipedia, yelp, youtube (A few of these, especially Google, Digg, and Twitter, are starting to become verbs by common usage these days, so you could count them as verbs if you were being generous.)

32 are hyphenated phrases: add-to-calendar, amazon-search, answers-search, ask-search, check-calendar, close-related-tabs, close-tab, command-editor, command-list, desaturate-image, detect-email-provider, ebay-search, edge-detect-image, edit-page, escape-html-entities, flip-page, get-email-address, google-image-search, invert-image, last-email, link-to-wikipedia, map-these, msn-search, remove-annotations, rotate-image, share-on-delicious, skin-list, stop-editing-page, syntax-hilight, view-source, word-count, yahoo-search

(Pie chart thanks to http://chartpart.com/)

The great thing about standards is there are so many to choose from!

The lack of consistency is bad in itself, especially for new users. Say I’m new to Ubiquity, and I want to upload a selected image to my Flickr account. I don’t know whether or not there’s a command to do that. I’m going to make some guesses and type them into the command line to see if they match commands. But what shall I type?

• “flickr”?
• “upload”?
• “photo”?
• “upload-photo-to-flickr”?

If ubiquity stuck to one style of naming, I could immediately eliminate all but one of these. But as-is, I have to try all four before I can conclude that there’s no command matching what I want.

Currently, using Ubiquity is like programming in PHP — where some standard library functions are_like_this() and other functions areLikeThis(). I have to look up the names of functions almost every time I use them in PHP, because there’s no consistency.

Hyphenated-phrases-are-awkward

Beyond the lack of consistency, the fact that so many commands — nearly half! — are named with hyphenated phrases is really bad. Hyphenated phrases have the following drawbacks:

1. They have the DOS nature. Remember the bad old days when you couldn’t have spaces in your filenames on most operating systems? You had to mash words together or put hyphens or underscores in there because the OS used spaces to separate arguments? Yeah. I’m not proud of recreating that environment.
2. They’re hard to type. The hyphen key is off the home row. You have to reach for it; you have to think about it. And the parser isn’t smart enough to do disjoint matches inside a verb name, so you can’t type “add to cal” and have it recognize “add-to-c alendar”. You can type “add” or “cal” and arrow-key down to the correct completion, or you can type hyphens. Either way, your fingers must leave the home row.
3. They’re hard to learn. A long series of words (which must go in a certain order) is harder to remember than a single word.
4. They make a mockery of the idea of natural-language input. “add-to-calendar this” is anything but natural, but that’s what the system wants you to type. The natural input would be “add this to calendar”, but Ubiquity can’t parse that because it doesn’t know how to break up the hyphenated phrase.
5. They’re just plain ugly.

So-why-do-we-use-so-many-of-them?

First of all, they’re hyphenated phrases instead of multi-word phrases because of a limitation in the parser: it assumes that the verb ends at the first space, and therefore verbs can’t contain spaces. This could be fixed, but the question then shifts to “why do we use so many multi-word phrases”?

To clearly spell out what a command does. Commands named with a single verb are usually ambiguous as to function: Without looking at the documentation or preview strings, can you tell me what “convert” is supposed to do, what “exit” exits, and what “zoom” zooms? Often the verb by itself has little meaning. If we were to enforce a rule that command names must start with verbs, think of how many dozens of commands would start with “add”, “check”, “set”, “search”, or some variant thereof. If you want to “check the weather”, most of the meaning of that phrase is in the word “weather”, not the word “check”. That leads to commands getting named either with the noun (“weather”) or with an entire phrase like “check-calendar”.

The other reason commands get named with phrases is because the verb alone is already taken. If you want to write a command that does something with email, you can’t call it “email”, because that word is already taken by a built-in command, as are many other common and widely applicable verbs. So you’re forced to go with something like “last-email” or “get-email-address”. Simple, meaningful, memorable, single-word verbs are a limited resource. That resource is already running low, and the problem will only get worse the more commands are written.

Finally, related to this, hyphenated phrases are used as a primitive version of namespacing. There is no better example than the Feedly commands:

• feedly-calais
• feedly-email
• feedly-explore
• feedly-mark-as-read
• feedly-save-for-later
• feedly-share
• feedly-tweet
• feedly-view

Feed-reader service Feedly is now building a whole Ubiquity-command-based interface to their software. As you can see, they’re prefixing their commands with “feedly” to distinguish them from the built-in commands like “email”, “twitter”, etc. They’re using a hyphenated prefix to define a “Feedly namespace”.

As one of the Ubiquity developers, I find this incredibly cool, not to mention very flattering. However, at the same time, I’m cringing a little bit inside, because the limitations of the system that I helped build are forcing Feedly to choose sub-optimal names. Command names like “feedly-email” are bad for all the reasons I mentioned above: they require very un-natural language, and worse than that, they’re hard to type. You can type “email” and be offered a choice between “email” and “feedly-email”. You can type “feedly” and be offered a choice between the top five “feedly” commands. But to unambiguosly specify “feedly-email” under the current parse, you have to either type the whole thing including the hyphen or you have to use the arrow keys.

I’m not happy about this. (Can you tell?)

I don’t believe it would work to simply tell command developers “Hey! Stop using hyphens in your names!”. Instead, we need to address the limitations of the system that are currently pushing developers to choose hyphenated names. The solution will have to satisfy the need for namespacing, alleviate the scarcity of good single-verb names, and allow for more descriptive and meaningful command names, without requiring the user to enter un-natural language or to do lots of extra typing.

In the next and last post in this series, I’ll make a proposal for such a solution.

Hello, Ubiquity command developer community?

We need to talk.

Here is an assortment of Ubiquity verbs, that I chose semi-randomly as a representative sample. Some were developed by Mozilla Labs, and others are third-party verbs I found via the Herd. What do they all have in common?

• google
• yahoo-search
• add-to-calendar
• command-editor
• remove-annotations
• flickr
• view-source
• foxytunes
• stumbleupon
• validate
• search
• lyrics
• link-to-wikipedia

(Answer below the fold.)

(more…)