Computing for Human Expereince: Semantics empowered Sensors, Services, and Social Computing on ubiquitous Web

When planning for my keynote at 2008 Asian Semantic Web Conference, I decided to think about and focus on what might drive my future research. Besides mainstream work on Semantic Web, we at Kno.e.sis are doing a good bit of work on Semantics-enhanced Services and Cloud Computing, Semantics-empowered Social Web, Semantic Sensor Web-- the talk leverages these research thrusts. However, more than these technological perspectives, what excites me more is the evolution along the lines of sensory-perception-semantics dimensions and and disappearance of physical-digital-conceptual divide. As with most of my research, applications drove the thought process-- in this case trying to envisage applications. More recently, I was invited to align my Web Semantics and Services Column with the Jan/Feb 2010's IEEE Internet Computing special issue on "Internet Predictions" that is guest-edited by Vint Cerf and Munindar Singh.

Version of the article on Computing for Human Experience
Version of the keynote talk and video on Computing for Human Experience

January 1, 2010

Two interesting issues of IJSWIS

Those interested in this blog are likely to be interested in:

IJSWIS Special Issue on Linked Data (with free access to lead article by Bizer/Heath/Berners-Lee)

IJSWIS Special Issue on Scalability and Performance of Semantic Web Systems

Amit

Citizen Sensing, Social Signals, and Enriching Human Experience

As I watch a media outlet such as CNN playing a video from YouTube based on a link their reporter/producer sees on a twitter feed [1], I am able to find the same content directly, reported by citizen sensors. The next small step is being able to study social signals through analysis of growing sources of social media and citizen (human-in-the-loop) sensors--to help us study the pattern of discourse, and variations along spatial, temporal and thematic dimensions. For example, we can ask, what are the important topics of discussions and concerns in different parts of the world on a particular day, or how different cultures or countries are reacting to the same event or situation (eg Mumbai Attack), or how a situation such as financial crisis is evolving over a period of time in terms of key topics of discussion and issues of concern (eg subprime mortgages and foreclosures, followed by troubled banks and credit freeze, followed by massive government intervention and borrowing, and so on). Combining the citizen sensors with fixed and mobile sensors (also called multilayer sensing) will lead to even more compresensive information on events and insights into situations. A paper [4] gives more details while a talk [2] and an article [3] give overview.

From computer science research perspective, it is exciting to see that a number of things Kno.e.sis researchers are working on are coming together -- especially the topics of semantics-enabled services, sensor and social computing:

• aggregation and integration of social data (coordinated/led by Karthik Gomadam, and leading to Twitris)
• analysis of user-generated content (by Meena Nagarajan [5])
• extraction/creation of a domain model from Wikipedia or similar community authored content (by Christopher Thomas)
• semantic sensor web (by Cory Henson)

From social perspective, here is a thought that comes to my mind. Countries may have authoritative (or dictatorial) regime, but it is becoming increasingly impossible to contain democratic aspiration and expressions of their citizens when they have access to social media. Only a couple of countries have effectively managed to be exceptions-- how long will they remain exceptions? Social scientists have started to ask such questions. Lately I have seen lively discussion on journalism and new media on TV news programs. I see rewarding opportunities for collaborations between computer scientists and social scientists.


Amit

[1] e.g., http://twitter.com/#search?q=%23iranelection
[2] Semantic Integration of Citizen Sensor Data and Multilevel Sensing: A comprehensive path towards event monitoring and situational awareness, February 17, 2009.
[3] Citizen Sensing, Social Signals, and Enriching Human Experience- IEEE Internet Computing, July/August 2009.
[4] M. Nagarajan et al., Spatio-Temporal-Thematic Analysis of Citizen-Sensor Data - Challenges and Experiences, Tenth International Conference on Web Information Systems Engineering, Oct 5-7, 2009, Poland, to appear.
[5] What are people talking about, Why people write, How people write: Meena Najarajan's research

Maturing semantic technologies for information systems

In 2002, I organized one of the earliest workshops on Semantic Web. It was nice to see that Reasoning Web 2009 [1] has chosen to focus on something I wrote in the Amicalola Report [2].

"Researchers and practitioners in the database, information systems and internet fields over the years have made significant progress towards the building of solutions that involve such systems for a wide range of application domains. In doing this, solutions necessarily concentrated mainly on syntax as the readily available unifying formalism for representation and structure, rather more than on the broad variety of semantics involved. One of the recent unifying visions is that of Semantic Web, which proposed semantic annotation of data, so that programs can understand it, and help in making decisions. Researchers have subsequently seen the value of using semantics to understand information and decision making needs of humans, so that data and human's needs can be semantically intermediated. The scope of semantics-based solutions has also moved from data and information to services and processes. Semantics has not been new to the database and information systems community. Semantics in data models was studied intensively in the 1980s, and applied to problems such as query processing, view management, schema transformation, schema integration and transaction processing. Semantic heterogeneity and interoperability have been studied as part of all major information systems architectures during the last three decades, including federated, mediator, and information brokering architectures. Many projects in information interoperability and integration have addressed semantic heterogeneity. In addition to the study of semantics, we believe there are several important areas of expertise within the database and information systems community developed as part of successful database management, information interoperability, information retrieval and workflow management systems that will be important to build large scale, high performance and practical Semantic Web and Enterprise solutions. A partial list of relevant technology for, e.g., semantic web services includes transaction management, query planning and optimization, distributed scheduling, exception handling, dynamic changes and adaptation, and security."

The corollary is that much more still needs to be done in Semantic Web to make its technologies as relevant and ubiquitous as a database management system, a workflow management system, or an IR system.

[1] http://reasoningweb.org/2009/Objectives.html

[2] http://knoesis.wright.edu/library/resource.php?id=00175

CI Fellows

For students who have completed their PhD between 5/1/08 and
8/31/09 and are interested in a postdoc position at Kno.e.sis
can check out my posting at: http://bit.ly/U4R72

Relationship Web

Relationships are at the heart of semantics and we envision a Web of relationships
to relate multimodal content across the Web. Following the first
generation of Web content access characterized by keyword driven document-retrieval, and the more recent process in entity awareness, we believe this third generation of relationship centric framework will support insight elicitation, semantic analytics and knowledge discovery over Web resources not possible so far. Relationship web incorporate the vision of trail blazing outlined by Dr. Bush in 1945!

More in our Internet Computing article: Relationship Web Blazing Semantic Trails between Web Resources (also available here).

What is Semantic Computing?

I have seen broadening of the scope of where "semantics" is applied. In mid-80s to early 90s, we had explored semantics for data(base) modeling, interoperability and integration (for examples of our work: see [1]). Others also looked at similar problems as well as use semantics in AI, programming language and several other areas in Computer Science. As Web started to become the global information system, some of us explored semantics (esp use of ontologies) for searching and integrating Web accessible information (see [2] for our efforts that predated coining of the term Semantic Web). All these activities then came under the umbrella of Semantic Web after TBL coined the term. When Philip Sheu organized the Semantic Computing conference last year, it seems the scope of semantics as it related to computing further expanded. And the question was asked: how do we define Semantic Computing?

Phil Sheu in ICSC2007 cfp described it as:

"The field Semantic Computing applies technologies in natural languageprocessing, data and knowledge engineering, software engineering, computer systems and networks, signal processing and pattern recognition, and anycombination of the above to extract, access, transform and synthesize the semantics (contents) of multimedia, texts, services and structured data."

Here is my take:

Semantic computing is a vision of computing based on semantics shared between machines and people. It supports and exploits intrinsic, intended, and emergent meanings (content) in all aspects of computing, encompassing programming, algorithms, information management, and human interactions within devices, as part of communications, and across the Web. Semantics involves the use of formal descriptions, languages, and models, often encoded in metadata, knowledge, and representation of agreements (as in ontologies) to capture the content of multimedia, texts, services, and structured data so that it may be extracted, shared, synthesized and transformed. Semantic techniques foster the development emerging forms of computing, such as semantic Web, and entirely new forms, such as bio-inspired computing, as well as enhance traditional techniques of information retrieval, management of data (including multimedia and multimodal) and artificial intelligence (e.g., natural language processing machine learning, and computational intelligence), leading to more efficient and scalable information processing and higher-quality computer-human interaction.

Perhaps there is a better way to state what I intend to say in the above description of Semantic Computing. I am sure there will better attempts to define this.


[1] mid 80s to early 90s: So far yet so Near, Schematic and Semantic Similarities between Database Objects
[2] early 90s to about 1998: Semantic Information Brokering, InfoHarness, InfoQuilt, OBSERVER