Metadata Management in the Semantic Grid

(1)

L3S Institute, Hanover, Germany October 18th, 2006

http://www.cs.man.ac.uk/~ocorcho/InvitedTalks/L3S_October2006.zip

S-OGSA v2.0.

Metadata Management in the Semantic Grid

www.ontogrid.eu

Oscar Corcho

2 Oscar Corcho. L3S, Hanover, 18 Oct 2006

Outline

Background

The Grid and its characteristics

Open Grid Services Architecture-OGSA

Grid Standardization Activities

Semantic Grid

OntoGrid and Semantic-OGSA (S-OGSA)

The S-OGSA model

S-OGSA capabilities and mechanisms

Lifetime specification

S-OGSA scenarios of use

Semantic Provisioning Services

Conclusions and Future Work

(2)

What is the Grid?

Heavily overloaded term

Analogy to On-Demand Access to Electricity

Vision and definition

“When the network is as fast as the computer's internal links, the machine

disintegrates across the net into a set of special purpose appliances” [Gilder]

Infrastructure that will enable

“coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations” [Foster et al.]

°

Virtual Organizations

Dynamic confederations organized around common goals

Diverse membership & capabilities

• People, compute resources, data resources, etc.

Diverse geographic distribution

Sharing is well-controlled

Minimum knowledge about physical characteristics of resources

Construction of higher level capabilities via composition of existing ones similar to SOA

From http://www.globus.org

°

(3)

CAUTION!

ACRONYM SPILL!!

Open Grid Service Architecture - OGSA

Cross cutting requirements

Interoperable

VO level

Optimized

Reliable

Certain QoS Guarantee

Scalable

Available

Extensible

Characteristics

Service Orientation

Management operations

• Resource Representation/

State

• Lifetime create

stop destroy queryProperties

rewind Get/ set Properties

Total capacity

Service Resource Available space

Used space

WS-RF WS-Management WS-I+

WS-GAF

(4)

Grid Reality

“Virtual Homogeneity”

Realizations OGSA - Vision

. . . .

Heavy use of XML

Requires experts to install, configure and maintain

Not near the ambitious OGSA landscape in terms of cross-cut requirements

Metadata Matters

Particularly for the following activities:

Provenance

Systems Configuration

Policy representation and reconciliation

Resource discovery

Using:

Open, flexible and extensible self describing schemas that don’t have to be nailed down

• “Let’s describe my data set, or the output format of this tool”

• Lightweight, referable, non-flat schemas

• Decoupled, interoperable systems, which resist to syntactic changes

Data integration

• Resource models

• Policy models

Formalization & Reasoning support

(5)

“The Semantic Grid is an extension of the current Grid in which information and services are given well-defined and explicitly represented meaning, better enabling computers and people to work in cooperation”

D. De Roure, et. al

The Semantic Grid

Semantics in and on the Grid

The Semantic Grid Roadmap

Pioneering Phase

• myGrid, Geodise, UniGrids…..

Systematic Investigation Phase

• Bridge Building:

Dagstuhl Seminar on Semantic Grid (July 2005)

Semantic Grid Workshops in OGF

Semantic Grid 101

• OntoGrid and others:

Synergetic (Grid ÅÆSW) approaches, tackling problems in middleware and application scenarios

Demonstration phase (1-5 years)

Deployment of semantic grid systems

(6)

Outline

Background

Semantic Grid

OntoGrid and Semantic-OGSA (S-OGSA)

The S-OGSA model

S-OGSA scenarios of use

Semantic Provisioning Services

Conclusions and Future Work

EU-STREP Project OntoGrid

Middleware for the Semantic Grid

Metadata Storage & Querying

Ontology Access

Annotation

• Data and provenance

• Services

Business Process Monitoring

Negotiation and Coordination

Applications

Insurance Settlement

Satellite Image Quality Analysis

SEMANTIC OGSA

Capabilites & Behaviors for Semantic Grids

Principled way of realization

And other applications being analysed

(7)

S-OGSA Design Principles

• Conceptual: reference architecture that can be applied to any grounding (WSRF, WS-Man, WS-I+, etc.)

• Parsimony: Architecture as lightweight as possible: minimise the impact on tooling, not dictate content

• Extensibility: Extensible and customisable as opposed to complete and generic architecture

• Diversity : Mixed ecosystem of Grid and Semantic Grid services. Semantics Ignorant, Semantics aware but incapable, Semantics aware and capable

• Uniformity: Everything is OGSA compliant. Our services are Grid services, knowledge and Metadata are Grid Resources.

• Multiform-Multiplicity: Any resource can have multiple descriptions and any description can be in different formalisms

• Enlightenment: Straightforward migration path

S-OGSA

Semantic-OGSA (S-OGSA) is...

Our proposed Semantic Grid reference architecture

A low-impact extension of OGSA

• Mixed ecosystem of Grid and Semantic Grid services

Services ignorantof semantics

Services awareof semantics but unableto process them

Services awareof semantics and ableto process (part of) them

• Everything is OGSA compliant

Defined by

• Information model

New entities

• Capabilites

New functionalities

• Mechanisms

How it is delivered

Model

Capabilities Mechanisms

provide/

consume expose

use

(8)

S-OGSA Model. Semantic Bindings

Model Mechanisms Capabilities

METADATA as Semantic Annotations

S-OGSA Model Example

(9)

Optimization

Execution Management

Resource management

Data Security

Information Management Infrastructure Services

Application 1 Application N

OGSA Semantic- O GSA

Semantic Provisioning

Services

From OGSA to the S-OGSA

Ontology

Reasoning

Knowledge

Metadata

Annotation

Semantic binding

Semantic Provisioning Services

18 Oscar Corcho. L3S, Hanover, 18 Oct 2006 Semantic Provisioning

Service

Knowledge Resource

Grid Entity

Semantic Binding

Grid Service

Is-a

0..m 0..m

1..m 1..m

Semantic aware Grid Service

consume

produce

0..m 0..m

1..m 1..m

uses

WebMDS

SAML file

DFDL file

JSDL file

Is-a

Knowledge Entity

Is-a

Ontology Service

Is-a

Reasoning Service

Semantic Binding Provisioning Service Annotation

Service

Metadata Service

Grid Resource

OGSA-DAI

CAS

Is-a Is-a

Knowledge Service

Is-a

Ontology

Rule set

Knowledge Semantic Grid Grid

S-OGSA Model and Capabilities. The complete picture

(10)

S-OGSA Patterns. Semantic-ignorant service

Lifetime

Metadata Service Ontology

Service

Service Resource Metadata

Seeking Client

Properties

Others….

Access/Query Metadata Refers to

Resource props

S-OGSA Patterns. Semantic Aware but Incapable Service

Lifetime

Metadata Service Ontology

Service

Properties

Others…

Access/Query Semantic

Bindings ^{Refers to}

Get Semantic Binding Pointers 2

1 Resource

properties

(11)

S-OGSA Patterns. Semantic Aware and Capable Service

Lifetime

Metadata Service

Properties

Others…

Access/Query Semantic Bindings

1 Semantics

1.1 Farm out

request

Semantic aware interface Ontology

Service

S-OGSA Grounding. Grid Ontology and S-OGSA Ontology

Grid Ontology

Common set of ontologies to describe Grid entities (resources and services)

Based on work from UniGrids

Effort to be continued by OntoGrid

Available in OntoGrid’s CVS

(12)

Semantic Binding Service. Lifetime Specification

What happens if...

...any or all of the Grid entities it refers to disappears?

• Instrument and planning files for satellites do not disappear

• Insurance contracts, cars, repair companies, etc., may disappear

...the Knowledge entities disappear or evolve?

• Ontologies may change

... a SB is no longer available (its content is not useful any more)?

• Damage claims: add witness reports, improve info about location, create new hypothesis...

When do/should SBs become invalid? How often should this be checked?

What is the status of the content of a SB (e.g., content checked, stable, unchecked, etc.)?

Is a SB always active or can it be archived after a period of time?

Satellite data that is not used after some time

Semantic Binding Service. WS-SBResourceLifetime

Lifetime specification based on WS-ResourceLifetime

Extension with

Resource properties (state)

Updates

From entities to Semantic Binding

SemanticBindingService EPR key

...

key

...

Lifetime

Service Resource

Properties

Others….

Resource props

Polling of resource property [lastModificationTime]

Grid Entity

Knowledge Entity

Lifetime

Service Resource

Properties

Others….

Resource props

Polling of resource property [lastModificationTime]

(14)

Update Notifications

Semantic Binding Update

Description: Updates in the content or in the state of a Semantic Binding

Message content:

• updateTime

• updateType [stateChange,contentChange]

• newState [any of the ones defined in the state machine]

• updateReason

SemanticBindingService

MetadataService EPR

key

...

key

...

SBHouseKeepingService

Check that SB content is still valid

Outline

Background

Semantic Grid

OntoGrid and Semantic-OGSA (S-OGSA)

The S-OGSA model

S-OGSA scenarios of use

Semantic Provisioning Services

Conclusions and Future Work

(15)

S-OGSA Scenario. Satellite Image Quality Analysis

Scenes:

Routine operations

Metadata generation

Report retrieving

Satellite Routine Operations

Satellite LifeCycle:

Launch and Early Orbit Phase (~ 3 days)

Calibration and Validation campaign (~ 6-9 months)

Routine operations (~ 5-9 years)

Satellite de-orbiting.

Product processing continues

S-OGSA Scenario. Satellite Image Quality Analysis

WebDAV

WS-DAIOnt

SatelliteDomain Ontology

Grid-KP XML Summary

WebDAV client File e.g. MS Windows

Explorer HTTP PUT

Atlas Metadata Service QUARC-SG client

JSP

2

UTC2Seconds Soaplab

3

4

7

2 1

1

3

6 Convert time to

canonical representation

Annotate file

Obtain ontology

Type metadata Store

Query Convert time to canonical representation Input

criteria Copy satellite XML summary file

Metadata generation process Metadata querying process

RDFRDF RDFRDF

(16)

S-OGSA Scenario. Insurance settlement

Data and resources scenarios

Register Repair Co. contract at CarRepairGrid.

Select Repair Companies for negotiation

Metadata scenarios

Calculate offer by a Repair Company (damage report)

Judge Invoice sent by Repair Company

Process management scenarios

Multi issue negotiation between Repair Companies (repair)

Multi issue negotiation between >3 insurance companies (Recovery)

Services scenarios

Provide Policy Information

Check coverage

Security scenarios

Check client registration at insurance companies

Check Car Theft - automatic check on car identity i.e. frame numbers and parts

S-OGSA Scenario. Insurance settlement

WS-DAIOnt Negotitation

Service (Manager)

Job Negotiation client

1 2

Do Negotiation

Atlas RDFRDF RDFRDF

RDFRDF RDFRDF RDFRDF

InsurranceCo DB

Motor Vahicles

Car Parts

Repair CO. 1 (Nego. Srvc. Contractor)

Job + Contractor List Job

Job Cfp

Cfp

Cfp propose Offer Refuse

propose Offer 2

2 4 4

4

accept

5

Reject 5

WS-DAIOnt

Car Repair DB RDFRDF RDFRDF RDFRDF Car Repair

DB 3 calculatePrice

3 calculatePrice

Retrieve public Job desc.

Legacy databases Legacy

databases

(17)

International Insurance Settlement Scenario

Insurance Security scenario cast as role based Grid Access Control Scenario.

Role based Access Control Policy is:

Good Reputation Drivers are allowed to ask for an insurance policy. Bad Reputation ones are not.

VO ontology based on

• KaOS ontologies (Actors, Groups and Actions)

Role definitons

Extend ontology with domain-specific classes and properties

Define roles wrt these extensions

)E.g., a blacklistedDriver is a driver that has had at least 3 accident claims in the past

)E.g., a goodReputationDriver is a driver that has been insured at least by one trusted company and that has had at most 2 accident claims

The Access Control Function uses a DL classifier to obtain roles of a Subject.

WS-DAIOnt

XACML_AuthZService (PDP) CarFraudService (PEP)

XACML AuthZ Request getInsurancePolicy

VO Ontology Class Hierarchy -RDFS

RDFRDF

John Doe has had 2 distinct accidents

Role Op Mapping

Pellet Reasoner Obtain Semantic

Bindings of John Doe

Obtain all classes that are subclass of ROLE

Classify John Doe wrt VO ont

Lookup whether the ROLE that is inferred permits or not XACML

AuthZ Response 1

2

3

4

5 6 7

Semantic Binding Service

PIP Proxy

PDP Proxy

VO Ontology OWL

S-OGSA Scenario. Authorisation

8 Result or Exception

/C=GB/O=PERMIS/CN=User0

(18)

WS-DAIOnt

RDFRDF

Role Op Mapping

AuthZ Response 1

2

3

4

5 6 7

PIP Proxy

PDP Proxy

VO Ontology OWL

WS-DAIOnt

RDFRDF

Role Op Mapping

AuthZ Response 1

2

3

4

5 6 7

PIP Proxy

PDP Proxy

VO Ontology OWL

(19)

WS-DAIOnt

RDFRDF

Role Op Mapping

AuthZ Response 1

2

3

4

5 6 7

PIP Proxy

PDP Proxy

VO Ontology OWL

WS-DAIOnt

RDFRDF

Role Op Mapping

AuthZ Response 1

2

3

4

5 6 7

PIP Proxy

PDP Proxy

VO Ontology OWL

(20)

WS-DAIOnt

RDFRDF

Role Op Mapping

AuthZ Response 1

2

3

4

5 6 7

PIP Proxy

PDP Proxy

VO Ontology OWL

Ignorant of semantics

Semantic aware and capable of processing semantics Semantic provisioning services

Semantic aware but incapable of processing semantics

Outline

Background

Semantic Grid

OntoGrid and Semantic-OGSA (S-OGSA)

The S-OGSA model

S-OGSA scenarios of use

Semantic Provisioning Services

Conclusions and Future Work

(21)

Motivation for a Grid Compliant Ontology Service

Node A

RDF(S) 3store

Node C OWL Jena Node B

RDF(S) Kowaki

Current Semantic Grid Applications access ontologies using Semantic Web technologies

No Grid compliant ontology access mechanisms for RDF(S) and OWL ontologies that exploit resource sharing capabilities

No specialized ontology registries

No Grid compliant instance access mechanisms in RDF

Node D

Ol1l

W

WS-DAIOnt-RDF(S) Implementation Architecture

Web Services Data Access and Integration – The Ontology Realization

First implementation focused on RDF(S):

WS-DAIOnt-RDF(S)

Sesame and Jena

• Future: Oracle RDF-store

Query languages: SPARQL, SeRQL, RDQL, RQL

Two-tier architecture

Web Service tier

• Upper service layer

• Intermediate service layer

• Lower service layer

RDF(S) access tier

• RDFSConnector

• Storage layer

Sesame RDFSConnector

Upper service layer

Intermadiate service layer

Lower service layer

Storage layer RDFSRepository

SelectorService

RDFSRepository Service

RDFSClass Service

Web Service TierRDF(S) AccessTier

(22)

Outline

Background

Semantic Grid

OntoGrid and Semantic-OGSA (S-OGSA)

The S-OGSA model

S-OGSA scenarios of use

Semantic Provisioning Services

Conclusions and Future Work

Conclusions

A principled Semantic Grid reference architecture

Low-impact extension of OGSA

Mixed ecosystem of Grid and Semantic Grid services

Ontology and metadata technology...

... can be used in Grid applications

... has to be adapted for its use in Grid environments

• Grid-compliant (provide Grid protocols, interfaces, etc.)

• Grid-aware (use of Grid technology)

First use cases being deployed

Still far from large-scale (production) deployment

(23)

Our future work

Semantic Binding Service

Lifetime management

Fine-grained AuthZ

Prototypes demonstrating Knowledge-Aware Grid Services

OGSA-DAI semantic extensions

EGEE information service consolidating heterogeneous information sources

Meta-scheduler using semantic technologies

Enlightenment

Guidelines about how and when to apply semantic technologies in Grid systems

More information

Publications

An overview of S-OGSA: a Reference Semantic Grid Architecture.

Corcho O, Alper P, Kotsiopoulos I, Missier P, Bechhofer S, Goble C.

Journal of Web Semantics 4(2):102-115. June 2006

http://www.ontogrid.eu/. Deliverable D1.2v2

Source code

http://www.ontogrid.eu/,For Downloading Distributions

Access to CVS

Connection type: pserver user: ontogrid

password: not needed Host: rpc262.cs.man.ac.uk Port: 2401

Repository path: /local/ontogrid/cvsroot module: prototype

(24)

Questions

Thank you for your attention!

Questions?

Acknowledgements

OntoGrid Consortium

• Pinar Alper, Ioannis Kotsiopoulos, Paolo Missier, Sean Bechhofer, Carole Goble

Miguel Esteban and Asunción Gómez- Pérez (WS-DAIOnt)

L3S Institute, Hanover, Germany October 18th, 2006

http://www.cs.man.ac.uk/~ocorcho/InvitedTalks/L3S_October2006.zip

S-OGSA v2.0.

Metadata Management in the Semantic Grid

www.ontogrid.eu

Oscar Corcho

Metadata Management in the Semantic Grid

S-OGSA v2.0.

Metadata Management in the Semantic Grid

Outline

 Background

 OntoGrid and Semantic-OGSA (S-OGSA)

 S-OGSA scenarios of use

 Semantic Provisioning Services

 Conclusions and Future Work

What is the Grid?

°

Virtual Organizations

°

CAUTION!

ACRONYM SPILL!!

Open Grid Service Architecture - OGSA

Grid Reality

Metadata Matters

D. De Roure, et. al

The Semantic Grid

Semantics in and on the Grid

 Pioneering Phase

 Systematic Investigation Phase

 Demonstration phase (1-5 years)

Outline

 Background

 OntoGrid and Semantic-OGSA (S-OGSA)

 S-OGSA scenarios of use

 Semantic Provisioning Services

 Conclusions and Future Work

EU-STREP Project OntoGrid

Metadata Storage & Querying

Ontology Access

Annotation

 SEMANTIC OGSA

S-OGSA Design Principles

 Semantic-OGSA (S-OGSA) is...

Our proposed Semantic Grid reference architecture

A low-impact extension of OGSA

Defined by

METADATA as Semantic Annotations

OGSA Semantic- O GSA

 Grid Ontology

 Effort to be continued by OntoGrid

 From entities to Semantic Binding

Outline

 Background

 OntoGrid and Semantic-OGSA (S-OGSA)

 S-OGSA scenarios of use

 Semantic Provisioning Services

 Conclusions and Future Work

 Insurance Security scenario cast as role based Grid Access Control Scenario.

 Role based Access Control Policy is:

 VO ontology based on

 The Access Control Function uses a DL classifier to obtain roles of a Subject.

Outline

 Background

 OntoGrid and Semantic-OGSA (S-OGSA)

 S-OGSA scenarios of use

 Semantic Provisioning Services

 Conclusions and Future Work

WS-DAIOnt-RDF(S) Implementation Architecture

Outline

 Background

 OntoGrid and Semantic-OGSA (S-OGSA)

 S-OGSA scenarios of use

 Semantic Provisioning Services

 Conclusions and Future Work

 A principled Semantic Grid reference architecture

 Ontology and metadata technology...

 First use cases being deployed

 Semantic Binding Service

Lifetime management

Fine-grained AuthZ

 Prototypes demonstrating Knowledge-Aware Grid Services

OGSA-DAI semantic extensions

EGEE information service consolidating heterogeneous information sources

Meta-scheduler using semantic technologies

 Enlightenment

 Guidelines about how and when to apply semantic technologies in Grid systems

Background

OntoGrid and Semantic-OGSA (S-OGSA)

S-OGSA scenarios of use

Semantic Provisioning Services

Conclusions and Future Work

Pioneering Phase

Systematic Investigation Phase

Demonstration phase (1-5 years)

Background

OntoGrid and Semantic-OGSA (S-OGSA)

S-OGSA scenarios of use

Semantic Provisioning Services

Conclusions and Future Work

Metadata Storage & Querying

Ontology Access

Annotation

SEMANTIC OGSA

Semantic-OGSA (S-OGSA) is...

Our proposed Semantic Grid reference architecture

A low-impact extension of OGSA

Defined by

Grid Ontology

Effort to be continued by OntoGrid

From entities to Semantic Binding

Background

OntoGrid and Semantic-OGSA (S-OGSA)

S-OGSA scenarios of use

Semantic Provisioning Services

Conclusions and Future Work

Insurance Security scenario cast as role based Grid Access Control Scenario.

Role based Access Control Policy is:

VO ontology based on

The Access Control Function uses a DL classifier to obtain roles of a Subject.

Background

OntoGrid and Semantic-OGSA (S-OGSA)

S-OGSA scenarios of use

Semantic Provisioning Services

Conclusions and Future Work

Background

OntoGrid and Semantic-OGSA (S-OGSA)

S-OGSA scenarios of use

Semantic Provisioning Services

Conclusions and Future Work

A principled Semantic Grid reference architecture

Ontology and metadata technology...

First use cases being deployed

Semantic Binding Service

Lifetime management

Fine-grained AuthZ

Prototypes demonstrating Knowledge-Aware Grid Services

OGSA-DAI semantic extensions

EGEE information service consolidating heterogeneous information sources

Meta-scheduler using semantic technologies

Enlightenment

Guidelines about how and when to apply semantic technologies in Grid systems

Thank you for your attention!

Questions?