mSpace: Improving Information Access to Multimedia Domains with Multimodal Exploratory Search

If you do not know much about classical music, how do you discover what you might like? The first port of call for most people is to Google “classical music.” This returns a list of links to sites that provide textual descriptions of terms or times in the classical music domain, or sometimes sites that sell classical music, or review particular pieces, or feature beginner’s guides that list “must hear” works [1]. While these are all useful sources of information, they do not enable direct assessment by or access to the content of the domain itself. mSpace (mspace.fm) is an interaction model [2] and software framework [6] that brings together a variety of mechanisms to improve access to information by supporting multiple ways of exploring the information itself.

The mSpace model conceptualizes information as a set of dimensions within a domain. In order to manage these high-dimensional spaces, we represent only a subset at a time, called a “slice.” These are arranged from left to right, in columns, creating a hierarchy, where the left-most column is the top level of the hierarchy and the right-most is at the bottom. Instances associated with a dimension are populated into a column. The accompanying figure shows the selection of Baroque in the Era column has restricted the instances that appear in the Composer column to those composers of the Baroque era.

The slice is dynamic: it can be altered by rearranging, adding, or subtracting dimensions, enabling individuals to determine how they wish to organize the domain to support their interest. For instance, someone interested in piano pieces can replace the Era dimension (default first column) with Arrangement. Selecting Piano will then mean that the following column—Composers—will show only those composers who created works for Piano. Providing these kinds of manipulations is what Marchionini refers to as support for “slicing and dicing” [3] information for user-determined exploration.

mSpace represents slices within a multicolumn “faceted” (see Hearst in this section) spatial browser that presents persistent contextual information around items of interest (see the figure). This approach facilitates “information triage” [4]. People can quickly browse the domain, see information about a selected item, see which other items are in the same area, and easily add the selected item to a collection space for later attention.

The terms in a given dimension (Baroque, symphony, serenade), nicely organized though they are, may mean nothing to a neophyte explorer. mSpace provides preview cues [5] to help address this problem. A preview cue is a lightweight mechanism that provides a multimedia preview of the area currently being triaged in the interface. In the classical music example, hovering over cues associated with Baroque will trigger pieces that are from that period. In this way, preview cues are distinct from current music store software, which provides audio samples only with a given piece. As well, preview cues foreground the exploration of content through examples of the content itself—in this case, the music—rather than the information about it, like Composer, Period, and so on.

Our research with pr views cues has shown, perhaps not surprisingly, that by providing cues at the category rather than the instance level alone, people are significantly better able to explore a domain and make selections within it [5]. At the system level, there are many methods to select the cues to associate with a category: expert recommendations, collaborative filtering, feature detection, and even random selection. Our own research suggests that even if the system uses only simple random selection to select a preview cue within the set of available pieces for that category it is both highly effective and scalable. In other words, any valid preview cue is significantly better than no cue at all. To date, we have explored the use of preview cues for music and video; audio cues and video cues respectively. One of the advantages of preview cues is they provide an additional dimension for exploration with low impact for screen real estate requirements. This low screen usage for high value assessment has been particularly helpful when porting the mSpace interface to mobile devices (www.mspace.fm/mobile), where screen real estate is highly constrained.

mSpace is an interaction model and software framework that brings together a variety of mechanisms to improve access to information by supporting multiple ways of exploring the information itself.

The mSpace framework itself is built on Semantic Web technologies. The Semantic Web enables diverse, heterogeneous (that is, Web-type) resources to be reused in new contexts. Thus, an mSpace can pull together a variety of resources into new contexts by leveraging their associated semantics. Increasing numbers of Web resources, such as the Open Guides, blogs, and news RSS streams, publish their data as RDF. Other open sources such as Wikipedia can be easily translated into RDF—the W3C standard for mark-up language for semantic description of resources. With Semantic Web tools like mSpace, a source such as Wikipedia, when presenting information on Beethoven, can use either an mSpace about historical figures or a classical music mSpace featuring composers. These semantics also suggest that multiple mSpaces can be interconnected, to enable moving between domains as readily as moving between dimensions within a domain. The heterogeneity of domains will, in the near future, enable people to choose information sources they trust to populate the interface.

We also are integrating lightweight annotation/publishing methods to enable people to make comments on their discoveries and publish these comments so they too can add to information that can be explored about attributes of a domain. In the architecture, we are also investigating models from peer-to-peer to trackback for automatic publishing to an mSpace about relevant resources or for it to discover appropriate resources.

The purpose of mSpace is to improve access to information by supporting exploration of domains. mSpace does this by providing a variety of mechanisms—slice rearrangement, preview cues, collection building, information views, and contextual spatial displays—to let people determine how they wish to organize the space to best support their focus of interest. The long game plan for mSpace is to evolve into one of those interfaces that can both help discover these resources, bring them together dynamically, and enable a rich palette of mechanisms to explore and share the discoveries made on such expeditions.

Back to Top

Back to Top

Back to Top

Back to top