Citizen Sensing, Social Signals, and Enriching Human Experience

As I watch a media outlet such as CNN playing a video from YouTube based on a link their reporter/producer sees on a twitter feed [1], I am able to find the same content directly, reported by citizen sensors. The next small step is being able to study social signals through analysis of growing sources of social media and citizen (human-in-the-loop) sensors--to help us study the pattern of discourse, and variations along spatial, temporal and thematic dimensions. For example, we can ask, what are the important topics of discussions and concerns in different parts of the world on a particular day, or how different cultures or countries are reacting to the same event or situation (eg Mumbai Attack), or how a situation such as financial crisis is evolving over a period of time in terms of key topics of discussion and issues of concern (eg subprime mortgages and foreclosures, followed by troubled banks and credit freeze, followed by massive government intervention and borrowing, and so on). Combining the citizen sensors with fixed and mobile sensors (also called multilayer sensing) will lead to even more compresensive information on events and insights into situations. A paper [4] gives more details while a talk [2] and an article [3] give overview.

From computer science research perspective, it is exciting to see that a number of things Kno.e.sis researchers are working on are coming together -- especially the topics of semantics-enabled services, sensor and social computing:

• aggregation and integration of social data (coordinated/led by Karthik Gomadam, and leading to Twitris)
• analysis of user-generated content (by Meena Nagarajan [5])
• extraction/creation of a domain model from Wikipedia or similar community authored content (by Christopher Thomas)
• semantic sensor web (by Cory Henson)

From social perspective, here is a thought that comes to my mind. Countries may have authoritative (or dictatorial) regime, but it is becoming increasingly impossible to contain democratic aspiration and expressions of their citizens when they have access to social media. Only a couple of countries have effectively managed to be exceptions-- how long will they remain exceptions? Social scientists have started to ask such questions. Lately I have seen lively discussion on journalism and new media on TV news programs. I see rewarding opportunities for collaborations between computer scientists and social scientists.


Amit

[1] e.g., http://twitter.com/#search?q=%23iranelection
[2] Semantic Integration of Citizen Sensor Data and Multilevel Sensing: A comprehensive path towards event monitoring and situational awareness, February 17, 2009.
[3] Citizen Sensing, Social Signals, and Enriching Human Experience- IEEE Internet Computing, July/August 2009.
[4] M. Nagarajan et al., Spatio-Temporal-Thematic Analysis of Citizen-Sensor Data - Challenges and Experiences, Tenth International Conference on Web Information Systems Engineering, Oct 5-7, 2009, Poland, to appear.
[5] What are people talking about, Why people write, How people write: Meena Najarajan's research

Maturing semantic technologies for information systems

In 2002, I organized one of the earliest workshops on Semantic Web. It was nice to see that Reasoning Web 2009 [1] has chosen to focus on something I wrote in the Amicalola Report [2].

"Researchers and practitioners in the database, information systems and internet fields over the years have made significant progress towards the building of solutions that involve such systems for a wide range of application domains. In doing this, solutions necessarily concentrated mainly on syntax as the readily available unifying formalism for representation and structure, rather more than on the broad variety of semantics involved. One of the recent unifying visions is that of Semantic Web, which proposed semantic annotation of data, so that programs can understand it, and help in making decisions. Researchers have subsequently seen the value of using semantics to understand information and decision making needs of humans, so that data and human's needs can be semantically intermediated. The scope of semantics-based solutions has also moved from data and information to services and processes. Semantics has not been new to the database and information systems community. Semantics in data models was studied intensively in the 1980s, and applied to problems such as query processing, view management, schema transformation, schema integration and transaction processing. Semantic heterogeneity and interoperability have been studied as part of all major information systems architectures during the last three decades, including federated, mediator, and information brokering architectures. Many projects in information interoperability and integration have addressed semantic heterogeneity. In addition to the study of semantics, we believe there are several important areas of expertise within the database and information systems community developed as part of successful database management, information interoperability, information retrieval and workflow management systems that will be important to build large scale, high performance and practical Semantic Web and Enterprise solutions. A partial list of relevant technology for, e.g., semantic web services includes transaction management, query planning and optimization, distributed scheduling, exception handling, dynamic changes and adaptation, and security."

The corollary is that much more still needs to be done in Semantic Web to make its technologies as relevant and ubiquitous as a database management system, a workflow management system, or an IR system.

[1] http://reasoningweb.org/2009/Objectives.html

[2] http://knoesis.wright.edu/library/resource.php?id=00175

CI Fellows

For students who have completed their PhD between 5/1/08 and
8/31/09 and are interested in a postdoc position at Kno.e.sis
can check out my posting at: http://bit.ly/U4R72

Relationship Web

Relationships are at the heart of semantics and we envision a Web of relationships
to relate multimodal content across the Web. Following the first
generation of Web content access characterized by keyword driven document-retrieval, and the more recent process in entity awareness, we believe this third generation of relationship centric framework will support insight elicitation, semantic analytics and knowledge discovery over Web resources not possible so far. Relationship web incorporate the vision of trail blazing outlined by Dr. Bush in 1945!

More in our Internet Computing article: Relationship Web Blazing Semantic Trails between Web Resources (also available here).

What is Semantic Computing?

I have seen broadening of the scope of where "semantics" is applied. In mid-80s to early 90s, we had explored semantics for data(base) modeling, interoperability and integration (for examples of our work: see [1]). Others also looked at similar problems as well as use semantics in AI, programming language and several other areas in Computer Science. As Web started to become the global information system, some of us explored semantics (esp use of ontologies) for searching and integrating Web accessible information (see [2] for our efforts that predated coining of the term Semantic Web). All these activities then came under the umbrella of Semantic Web after TBL coined the term. When Philip Sheu organized the Semantic Computing conference last year, it seems the scope of semantics as it related to computing further expanded. And the question was asked: how do we define Semantic Computing?

Phil Sheu in ICSC2007 cfp described it as:

"The field Semantic Computing applies technologies in natural languageprocessing, data and knowledge engineering, software engineering, computer systems and networks, signal processing and pattern recognition, and anycombination of the above to extract, access, transform and synthesize the semantics (contents) of multimedia, texts, services and structured data."

Here is my take:

Semantic computing is a vision of computing based on semantics shared between machines and people. It supports and exploits intrinsic, intended, and emergent meanings (content) in all aspects of computing, encompassing programming, algorithms, information management, and human interactions within devices, as part of communications, and across the Web. Semantics involves the use of formal descriptions, languages, and models, often encoded in metadata, knowledge, and representation of agreements (as in ontologies) to capture the content of multimedia, texts, services, and structured data so that it may be extracted, shared, synthesized and transformed. Semantic techniques foster the development emerging forms of computing, such as semantic Web, and entirely new forms, such as bio-inspired computing, as well as enhance traditional techniques of information retrieval, management of data (including multimedia and multimodal) and artificial intelligence (e.g., natural language processing machine learning, and computational intelligence), leading to more efficient and scalable information processing and higher-quality computer-human interaction.

Perhaps there is a better way to state what I intend to say in the above description of Semantic Computing. I am sure there will better attempts to define this.


[1] mid 80s to early 90s: So far yet so Near, Schematic and Semantic Similarities between Database Objects
[2] early 90s to about 1998: Semantic Information Brokering, InfoHarness, InfoQuilt, OBSERVER

SAWSDL becomes a W3C recommendation

August 28, 2007 is a special day for our research group (METEOR-S project, LSDIS lab and Kno.e.sis Center). On this day, we saw the culmination of an activity I started a little over three years ago. In July 2004, we wrote a white paper"WSDL-S: Adding Semantics to WSDL - White Paper". In November 2005, in collaboration with IBM, we made a W3C member submission Web Service Semantics - WSDL-S. Our presentation at a W3C workshop held at Innsbruck received support from a number of key members of Semantic Web Services community that includes the OWL-S, WSMO and SWSF groups, which then led to the Semantic Annotations for WSDL (SAWSDL) Working Group in which Kno.e.sis (at Wright State University - a W3C member) and LSDIS (at the University of Georgia) also participated. After timely and extensive contributions of working group members (and especially the groups chair and editors- Jacek, Joel and Holger), the W3C declared Semantic Annotations for WSDL and XML Schema as a recommendation.

Interested readers can find more details in the Internet Computing column I wrote with my former advisee and a key early technical contributor to our work in this area, Kunal Verma (Semantically Annotating a Web Service), by giving this tutorial at Semantic Technology conference (Using SAWSDL for Semantic Service Interoperability), or by playing with implementations and test suites.

So what's next? My own group has defined SA-REST (more on this soon), and Charles Petrie and I have started (with healthy dose of encouragement from Dieter Fensel) a W3C SWS Testbed Incubator Group to develop the awareness and further agreements on post-SAWSDL issues that we have to address before more developers will find Semantic Web Services ready for the prime time.

Marrying Social Media with Semantic Media

This is a recreation of the original blog at LSDIS lab, UGA, before I moved to Kno.e.sis Center, Wright State University posted on 12/03/05.

A lot has been written about social networking and we can see a number of success stories around this phenomenon. Flickr exemplified taking social networking to media (photos in this case), and hence the emerging focus on social media (see The Flickrization of Yahoo). The power of users playing the roles of authors and editors is undeniable, but the lack of organization it creates is an antithesis to how Yahoo! organized the early web around a directory (that we can now replace by ontologies) and human cataloging (which we can to a good extent replace by automatic and ontology-assisted semi-automatic semantic annotation). Is it possible to get such organization back into naturally unconstrained and entropic collection of stuff (in this case media and associated human given tags without the supporting nomenclature)? I think it quite is, and the current approaches for and technologies behind the Semantic Web provide the most promising paths.

Indeed there is already an independent set of activities in developing semantic multimedia and semantic media. What can we attain by marrying semantic media with social media? Quite a lot. What we get is a multiplicative outcome that combines social media organized to make it much easier to search, integrate and exploit, with semantic media enhanced by the power of people. This will also make it much easier to integrate all shared information whether in text or in digital media of any format. It will be easier to serve up multimedia and multimodal applications. And it will unlock a lot of untapped potential with targeted advertisements and refined personalization that underlying semantic infrastructure could provide, significantly enhancing the business potential associated with shared media.

Here is just one enticing example. You are looking a photo of a church in Innsbruck. And you want to sell an advertisement that takes you on a landing page on Priceline that has a flight from US gateways to Munich. (I had a chance to checked out what types of Google ads are servied today in this context of Innsbruck, and it did not seem to me that anyone would likely close a sale, and especially a sale with a high value). What will tell you that the nearest airport with transatlantic flights is Munich, do not bother to try to sell a ticket to Innsbruck especially from a non European distination, and how to get the specific page on Priceline for North America to Munich flights? Context, semantic metadata, ontologies that have a load of real world and factual knowledge, and a set of rules.

Some of the lovers of information retrieval and dumb keyword search would tell you all this is too complex, not scalable or not maintainable. It is not. Contact me if you want to see a demonstration of how we are working to realize this promise.