Game On 2010, Example Code: Multitouch Gestures

While my previous code example showed how to recreate the past, this next example points out the possibilities of the future — Multitouch gesture interfaces, on the web!

The UI designer in me is chafing at the bit to start giving people web-apps with multitouch-based interfaces. The possibilities are enormous, and the advantages of multitouch UI – rich input, direct manipulation, natural mappings, fewer widgets, instant feedback, and a certain fun factor – are many.

But unfortunately, since this is still an emerging technology, it’s not exactly a standard yet. It’s not available on all hardware. On hardware where it is available, it’s not always made available through the browser. Different browsers expose multitouch events to web content in different ways, if they expose them at all. For this reason, the demo I’m about to show you won’t work on all browsers, sadly. So far, I’ve only been able to make it work using the Firefox 4 beta and only on multitouch-enabled computers running Windows 7 – which drastically limits the audience that will be able to try out this demo, I’m afraid.

So, I’m sorry to give you a demo that’s not fully cross-browser-compatible or standards-compliant. But! I think multitouch will become standardized, and soon. It won’t be too many more years before multitouch input becomes an everyday part of the Web experience on all platforms. So why not start experimenting with it now? We can start trying out new interface possibilities now, and work towards full cross-browser compatibility as the standards emerge.

And what better forum for experimenting with interface possibilities than the Game On 2010 contest?

Anyway, here’s the demo:

Multitouch Gesture Interpretation Demo

 

Game On 2010, Example Code: Old-School RPG Map

Mozilla Labs has announced Game On 2010, our first-ever game development contest.

To support Game On 2010, I’ve been working on some sample code that demonstrates how to accomplish common game programming techniques using only open Web standards and no plug-ins. For those who are thinking about entering but don’t know where to get started developing a game engine in HTML and Javascript, read on.

My first example is a set of classes for implementing a top-down, tile-based map screen that scrolls to follow the player’s movement. If you, like me, wasted your youth playing 8-bit and 16-bit console RPGs (like Dragon Warrior, Ultima, and my favorite under-appreciated classic: Phantasy Star) then this ought to look a bit familiar:

Old-School RPG Map Demo

Move the character around with the arrow keys, and notice how the screen scrolls when you get close to the edge.

 

Web-based piano made possible by <Audio> tag

For a lark, I hacked together a quick web-based piano keyboard using the <Audio> tag. The Audio tag (along with the Video tag) is part of HTML version 5, and it’s pretty cool, since it makes the audio file just another part of the HTML page rather than something that is stuck in a plug-in ghetto. That means that javascript running in the page can interact with the audio file, which is what makes this piano possible to do without Flash.

The audio files are pure sine waves (e.g. the A above middle C is 440 Hz) generated by Audacity and saved in .ogg format (an open-source encoding).

Unfortunately it will only work in a browser that supports HTML 5 the Audio tag, which as of right now is pretty much only (shameless plug) Firefox 3.5.

August 28, 2009: EDITED TO ADD: The web-based piano has been moved to evilbrainjono.net/music, where I am working on integrating it into a full web-based music composition lab.

 

Pie Menu evolves into Grid Menu!

Response to my post about pie menus was overwhelming! There were way too many comments for me to reply to each one individually, so instead I’ll respond to the general ideas in follow-up posts.

I’ve put up a new and improved pie menu demo that solves some of the usability problems with the previous version. Although, given the shape, perhaps “Grid Menu” might be a more descriptive name for this new style.

This version solves the following problems that people pointed out in the previous version:

1. It’s difficult to lay out text in pie wedges: this version uses squares. It’s still the case that every item is just a short movement in a unique direction from the starting point, so the benefits of the circular menu are still preserved.
2. The number of items can grow without bound, but usability suffers with more than eight choices: this version can handle hierachical sub-menus (and sub-sub-menus). Try mousing over the item in the bottom-right of the menu to see this in action.
3. Some items are unreachable if the pie menu is invoked near the edge of the screen: this version pushes the menu grid inward from the edge so that all items stay visible.
4. Many people aren’t used to holding the mouse button down while navigating the menu: this version respects click-move-click style interaction in addition to click-drag-release style interaction.

I’ve also added the names and icons of some actual Ubiquity commands, just to make the demo look more like what you might see in an upcoming version of Ubiquity — if we decide to include it.

Still a problem: if the original menu is invoked too close to the bottom-right of the area, there is sometimes not room for the sub-menu or sub-sub-menu. I’ll need to come up with a smarter positioning algorithm in order to deal with this problem.

You can try out new demo here.

 

Pie In The Sky

This blog post is a hands-on experiment. If you have Firefox 3, before you read any further, please go to this page and try out the demo.

(Unfortunately, the demo will only work in Firefox 3.0 and later, because it uses some cutting-edge features of the Canvas object that aren’t supported in Internet Explorer, Safari, or older versions of Firefox. I believe strongly in presenting web content in a cross-browser compatible way, but sometimes for feature demos, requiring Firefox is unavoidable. This is Mozilla labs, after all.)

Anyway, the demo presented two different styles of pop-up menus and asked you to repeatedly select the same menu item.

Which menu style made the task easier? Why do you think that was?