The Age of Assistants?

Reading Norman Winarsky's post on The Age of Assistants reminded me of the scene from the movie Dancing with the Wolves where the Sioux Chief asks Lieutenant Dunbar played by Kevin Costner:

You always ask about the white people. You always want to know how many more are coming. There will be a lot, my friend. More than can be counted.

How many?

Like the stars.

In a similar vein, Winarsky says: "And we likely won't have just one assistant - we'll have two or three or maybe even 10, a scalable, distributed cadre, an army, even - of Virtual Personal Assistants (VPAs) at our service." I believed that in 1993 when we shipped the world's first desktop assistant Open Sesame! for the Mac. Open Sesame! was a learning agent that observed a user’s interaction with the operating system GUI, found repetitive patterns and user preferences, and offered to automate repetitive tasks for the user:

Open Sesame Box.jpg Open Sesame Dialog Box.jpg

Open Sesame! was a relative success on the Mac. It was localized in Japan, shipped with every PowerPC in Taiwan, and got positive reviews in US, Italy, Germany, and beyond. In A Review and Analysis of Commercial User Modeling Servers for Personalization on the World Wide Web, Fink and Kobsa state: “Open Sesame can be considered an early pioneer of personalization, both in research and commercial environments. Despite its early market entry and its sophisticated features, there is, to the best of our knowledge, no commercial system on the market that is comparable to Learn Sesame.” In spite of the positive reviews, Open Sesame! could not escape the criticism of generating a new category of software - nagware. Alas Open Sesame! cannot even get credit for generating this category in Winarsky's article, which bestows this honor to Microsoft's brain dead Clippy.

We presented our analysis of the rich database of user feedback collected with Open Sesame! in our Applied Artificial Intelligence paper Learn Sesame - a Learning Agent Engine. While the users found event based learning useful, they found the monitored events and offered actions limited in scope, and stated their desire for improved agent communication and social skills. In the intervening 15 years, a lot changed in personal computing to make the conditions ripe for software assistants:

  • Most personal computer users have embraced direct manipulation in user experience. The idea of delegating a task to a software assistant, and waiting a couple of seconds, minutes, hours, ... etc. will only work for boring periodic maintenance type of tasks. Both personal and server computers have now become so powerful, enabling the opportunity for offering instant responses to delegated tasks. That will change everything in that delegated tasks to software agents will perform like direct manipulation to a user, thus increasing adoption.
  • Thanks to mobile computing, users have become used to notifications on their smartphones. Agent notifications that seem like intrusive spam on the desktop are now welcomed by users on their phones. In other words, we are more open to intrusion on our smartphones as they entertain us while waiting for a 3-hour flight delay. So the mobile computing platforms will be more welcoming to software agents that notify their users on the delegated tasks for status updates, additional task clarification requests, and the like.
  • Social computing helped users embrace the notifications of changes in user's social networks. The new generation of users cannot go a couple of seconds without clicking on a Facebook notification on their mobile app, which is essentially a notification agent for Facebook. In other words, Facebook is teaching users the value of notification, which was considered nagware intrusion on the desktop.
  • There is a decent amount of content for personalization. In 1998, using Open Sesame learning engine, we built eGenie - a personalized Web site that learned user interests, built user profiles and presented personalized content for new books, movies, TV shows, concerts, etc. Frankly, how personalized can be a movie preference that you share with millions of others? Not much. In contrast, the social media is now generating truly personal content like your friend's Facebook updates, Delicious annotations ... Personalization that can be performed by software assistants has a lot more value for the Long Tail.
  • Semantic technology is coming on strong. As your friends and colleagues generate more semantically tagged content using tags, forms, etc., it will make the job of personal assistants easier in filtering knowledge of import to users. Similarly, web services APIs , linked data, etc. are becoming mainstream, thus making it easy for your personal assistant to interact with these data and services in the cloud programmatically on your behalf.
  • When I first showed Open Sesame! to Don Norman at Apple, he asked: "What enabled this product to be built? Why now?" I replied: Apple Events enabled us to monitor user actions reliably, and instruct the OS to perform tasks with ease. My answer had some element of truth as trying to build the same assistant for Windows 95 proved to be an insurmountable task due the lack of support for high level recordable and scriptable events on this platform. Now that the Web browser is becoming the GUI of the operating system as we move more towards cloud computing, it is relieving personal assistants from the necessity of learning the legacy of desktop operating systems, and putting up with their changes.

At milcord, we are keeping the learning agent flame burning in our Commander's Learning Agent project. Stay tuned.