I've had a great time working with The House Theatre of Chicago and it's been exciting to wtch the company grow and evolve. I just wanted to point out that the House's first television commercial for their Broadway on Chicago production of "The Sparrow" is now online. Give it a peek.
(Credit for the video work goes to Lucas Merino and the music is composed by Kevin O'Donnell. You can get the Sparrow album on iTunes or your favorite online digital music store.)
In addition to Shion, I've been working on another new research application this summer. I call it "Do Not Disturb" (a working title) and it is an application that sits in the background and observes users in order to determine users' various contexts based upon readings collected from sensors. The long term goal for this application is to provide a system-wide service that other applications can consult when interacting with the local user. I've been inspired by the success that Growl's been in the Mac community and my goal is that this tool will ultimately provide a similarly useful service that makes applications smarter and users happier.
Before I get into the details about how this works, let me describe two use cases.
Instant messenger clients: Imagine that in your local IM client, there are settings that control the notification details of various events on the basis of your context. When you're surfing the 'Net in a recreational context, you may configure the client to be pretty obtrusive when a new message arrives. Sounds play, icons bounce, and things flash. When you're in a context where you are busy working, you may configure the client to be a bit less attention-hogging. A subtle sound plays, no icons bounce, and nothing flashes.
Secondly, imagine that you map the contexts "Working", "On the Phone", "Reading" to the instant messenger's "away" presence indicator. You map "Gaming", and "Surfing the Web" to the "available" indicator. "Away from the Computer" gets mapped to "idle". Rather than worry about setting your presence in the IM client throughout the day, the computer updates that for you on the basis of your observed context.
Smart environments / home automation: Imagine that your computer is smart enough to recognize when you're chilling out listening to music while playing a game. It instructs the lamps in the room to dim a few steps to create a relaxing atmosphere. Imagine that it can detect when you're busy writing a paper or programming. It boosts the level of the lights to a bright attentive level and powers an air freshener to begin emitting an orange scent (for example) that makes you more alert.
About Do Not Disturb
The missing component necessary to making these things possible is a centralized context engine that applications can consult to determine the user's context and act accordingly. Applications like IM clients sport their own simplified context engines, but these generally are not accessible to other applications and depend upon users manually setting their own context or simple timers that change the "available" context to "idle". Do Not Disturb (DND) attempts to remedy this deficiency by acting as a central service that other applications can message to obtain users' self-defined contexts.
Do Not Disturb is roughly based on the architecture developed by James Fogarty and Scott Hudson for the Subtle toolkit. DND includes a collection of independent sensors that take readings of things such as the currently running applications, the local AirPort networks, local Bonjour services, and what Bluetooth devices are visible to the machine. These sensors generate readings that are recorded and used for training to recognize context or to describe the current environment so that a probable context may be computed.
When the system is in training mode, the user is periodically prompted to specify their current context and this is fed into a machine learner (with a snapshot of the current environment). When the system is not in training mode, it periodically provides the machine learner a snapshot of the environment and the learner returns the most probable context. This is the same basic technology that underlies junk mail filtering and other text classification tools.
Where DND diverges from Subtle is that it is a native Cocoa application instead of a Java system service. This greatly eases the installation of the system because everything is self-contained and easily configured using standard GUI preference panes. The system may be started and stopped at any time by the user, but the context model generated by the learner is persistent between running instances of the application.
Current Status
Earlier this weekend, I finished grafting a simple naive Bayes learner into the application. This was the last major missing piece of the software and the application can now complete the full set of tasks of collecting sensor readings, training the machine learner, and using the machine learner to predict contexts. I've been using Google Code as my source repository and the current Xcode snapshots can be obtained using Subversion.
The current version is functionally still an alpha product. I'll be testing and tweaking the machine learner to measure and improve its performance over the next couple of weeks. I also have not created the interface for communicating with outside applications yet. This should be coming in the next several weeks and I plan on prototyping it using a custom Growl plugin, Shion, and an Adium plugin. It also goes without saying that nothing is currently documented, so that'll be something to do too.
In addition to the external integration, I have a full list of additional sensors to create and some of these sensors bring along their own set of issues. For example, observing that Safari is open may not be that informative. Observing that Safari is open and the frontmost page is Apple Developer Connection suggests a much different context than observing that the user is at Desktop Tower Defense. Since the context model is persistent across application instances, a use shouldn't have to reset their model to keep their browsing history private. Addressing this will involve some one-way hashing in the loggers and sensors. (This has the additional benefit of making the saved context model more compact.)
Another challenge is dealing with continuous values in the context of the current machine learner. From the learner's perspective, a local audio volume of 0.1 is as different from 0.2 as 0.9. I need to think about this problem some more and determine if generating discrete ordered values is sufficient or whether I need to supplement the machine learner in some way.
In the end, however, progress is being made at a nice pace. Once the application is ready for end-users, I'll probably throw up a blog for the application and begin distributing it to interested folk. I'm currently a couple of months away from that point, so stay tuned.
I've been looking forward to posting here on Vox with the new MarsEdit. The software looks slick and even talks with Flickr:
As you can see from the image above, Adenadar now has a pimpin' ride to go with his pimpin' hat.
