Preserving Electronic Records in the Sciences - InterPARES

(1)

InterPARES Project

Evelyn Peters McLellan, VanMap Case Study leader 1

Preserving Electronic Records in the

Sciences

VanMap GIS

(2)

The VanMap case study

Glenn Dingwall, City of Vancouver Archives

Richard Marciano, San Diego Supercomputer Center Reagan Moore, San Diego

Supercomputer Center Evelyn Peters McLellan,

Insurance Corporation of BC

(3)

What is a GIS?

• Geographic Information System

• Database system containing data linked to geospatial coordinates

• Typically presented to the viewer in the form of interactive maps

• May incorporate files such as CAD drawings,

satellite imagery and photographs that are not

geospatially referenced

(4)

What is VanMap?

The cross-corporate GIS created by the City of Vancouver and used by staff in

– Engineering – Planning – Permits and

Licenses – By-law

Enforcement

– Social Planning – Police

– Fire and Rescue – Parks and

Recreation

(5)

(6)

© C I T Y O F V A N C O U V E R

D ata quality not guaranteed

1 5 0 D I C L 1 9 8 3

1 5 0 D I C L 1 9 8 2 3

0 0 C I 1 9 5 4

2 0 0 C I 1 9 5 4

150 DI CL 300CI

1958 300 CI CL 1961 2

0 0 C I 1 9 5 6

150 CI

20 0

C I

19 55 2 0 0 C I 1 9 5 4

200 CI

1953

2 0 0 C I

1 9 5 5 150 CI 1948

300CI 1958 DI C

L2001

2 0 0 C I 1 9 5 4

150 CI 1948

150 DI CL 1976 200 DI 1975

0 DI 1975

150 DI CL 1983 1

5 0 C I 1 9 4

1 5 0 C I

1 0 0 D I

5 0 C

O P

1 9 7

0

20 COP 1970

0 DI CL 1985 2

0 0

D I C

L 1

9 7

6

300D I CL1993

300CI 1942 2

0 0 D I C L 1 9 8

4 0 C O P

150 CI

1 5 0 C I

3 0 0 D I C L 1 9 8 3

1 5 0 C I 1 9 4 8

300D I CL1983

300 DI CL1975

300 DI CL1975 2

0 0 D I 1 9 7 5

200 DI CL 1975 300

DI 1975

300DICL1983

200DICL1985300 CI

1951 300DICL1983

300 DI CL 1984 3

0 0 D I C L 1 9 7 5

150 DI CL1975 200

DI C L19

75

200 DI CL 1985 2

0 0 D I C L 1 9 7 5

300 DI CL 1975

300 DI CL1975

200 DI CL 1985 300DI C L1975

3 0 0 C I 1 9 5 1

300DICL1983

300 D I CL1975

CL 1976 150 DI CL 1975

300 DI CL 1975

1 5 0 D I C L 1 9 8 4

150 DI CL 1975

150 DI CL1975 300 DI CL 1975

150 DI CL 1975

300 DI CL 1975

300 CI 1951

150 DI CL 1983

150 DI CL 1983 300 CI 1951

150 DI CL 1976 150 DI CL 1976

300 CI 1951

150 DI CL 1985 150 DI CL 1974

2 0

0 D

I C

L 1

9 8 2 0 0 D I C L 1 9 8 5 7 5 0 S

100 WALTER HARDW 100 ATHLETES

300W2NDAV

300 W 6TH AV 700 MILLBANK

700 CHARLESON

2 4 0 0 O A K S T

RS-1

CD-1 (186)

BCPED

I-1 FCCDD

CD-1 (324)

M-2

C-3A

RM-3A CD-1 (297)

(7)

© C I T Y O F V A N C O U V E R

C A M B I E B R I D G E

<

- - -

<

- - -

<

- - -

<

- - -

<

- - -

C A M B I E B R I D G E - - -

>

2 1 0 0 C A M B I E S T - - -

>

2 1 0 0 C A M B I E S T - - -

>

2 1 0 0 C A M B I E S T - - -

>

2 1 0 0 C A M B I E S T - - -

>

2 1 0 0 C A M B I E S T - - -

>

100 W 1ST A 100 W 1ST A100 W 1ST A100 W 1ST A100 W 1ST A 100 WALTER HAR 100 WALTER HAR 100 WALTER HAR 100 WALTER HAR 100 WALTER HAR 100 ATHLETES 100 ATHLETES100 ATHLETES100 ATHLETES100 ATHLETES

100 W 3RD 100 W 3RD 100 W 3RD 100 W 3RD 100 W 3RD 100 W 2ND 100 W 2ND 100 W 2ND 100 W 2ND 100 W 2ND

2 2 0 0 A L B E R T A S T

2 0 0 0 A L B E R T A S T

200 W 5TH AV 200 W 5TH AV200 W 5TH AV200 W 5TH AV200 W 5TH AV

2 2 0 0 C O L U M B I A S T

100 W 4TH 100 W 4TH A100 W 4TH 100 W 4TH 100 W 4TH 300 W 4TH AV

300 W 4TH AV 300 W 4TH AV 300 W 4TH AV 300 W 4TH AV 200W2NDAV 200W2NDAV 200W2NDAV 200W2NDAV 200W2NDAV 17

00 CO

OK ST 17

00 CO

OK ST 17

00 CO

OK ST 17

00 CO

OK ST 17

00 CO

OK ST 200W1STAV

200W1STAV 200W1STAV 200W1STAV 200W1STAV

1 9 0 0 M O B E R L Y R O A D

300W1STAV 300W1STAV 300W1STAV 300W1STAV 300W1STAV

19 00

W YL

IEST 19

00 W

YL IEST 19

00 W

YL IEST 19

00 W

YL IEST 19

00 W

YL IEST 1

8 0 0 S P Y G L A S

1 8 0 0 S P Y G L A S

600 W 6TH AV 600 W 6TH AV 600 W 6TH AV 600 W 6TH AV 600 W 6TH AV 2 2 0 0 A S H S T

2 2 0 0 A S H S T

500 W 6TH AV 500 W 6TH AV 500 W 6TH AV 500 W 6TH AV 500 W 6TH AV

2 4 0 0 W I L L O W S T

500 W 7TH AV 500 W 7TH AV 500 W 7TH AV 500 W 7TH AV 500 W 7TH AV 1900GREENCHAIN

1900GREENCHAIN 1900GREENCHAIN 1900GREENCHAIN 1900GREENCHAIN

700 MILLBANK 700 MILLBANK700 MILLBANK700 MILLBANK700 MILLBANK

2 0

0 0

M IL L

Y A R

D 2

0 0

0 M

IL L Y

A R D 2

0 0

0 M

IL L Y

A R D 2

0 0

0 M

IL L Y

A R D 2

0 0

0 M

IL L Y

A R D

700 MILLBANK 700 MILLBANK700 MILLBANK700 MILLBANK700 MILLBANK 1 9 0 0 M I L L B A N K

1 9 0 0 M I L L B A N K

1 9 0 0 M I L L B A N K 100

DR AK

ES T 100

DR AK

ES T 100

DR AK

ES T 100

DR AK

EST 100

DR AK

ES T

CHARLESON CHARLESONCHARLESONCHARLESONCHARLESON

2 2 0 0 L A U R E L S T

800 W 6TH AV 800 W 6TH AV 800 W 6TH AV 800 W 6TH AV 800 W 6TH AV 800 CHARLESON 800 CHARLESON 800 CHARLESON 800 CHARLESON 800 CHARLESON

2 2 0 0 W I L L O W S T

700 W 6TH AV 700 W 6TH AV700 W 6TH AV700 W 6TH AV700 W 6TH AV EA

CH CR

ES CEN

T EA

CH CR

ES CEN

T EA

CH CR

ES CE

NT EA

CH CR

ES CEN

T EA

CH CR

ES CEN

T

00 SCHO

OL GREEN 0 SCHO

OL GREEN

00 W 8TH AV0 W 8TH AV0 W 8TH AV0 W 8TH AV0 W 8TH AV 00 W 7TH AV0 W 7TH AV 00 W 7TH AV 00 W 7TH AV 00 W 7TH AV

2 4 0 0 O A K S T

2 4 0 0 C O L U M B I A S T

2 4 0 0 A L B E R T A S T

2 3 0 0 Y U K O N S T

2 4 0 0 C A M B I E S T

2 2 0 0 C A M B I E S T

200W1ST AV

400W2NDAV C

A M B I E B

500 W 8TH AV 00 W 6TH AV

900 W 7TH AV

(8)

© C I T Y O F V A N C O U V E R

518 518518518518 609

609 609609 609

525 525525 525 525

500 500500500500

1859 1859185918591859

633

633633633633 527527527527527

619 619619619619

1873 1873187318731873 456

456456 456 456 601

601601601601

(9)

© C I T Y O F V A N C O U V E R

1 9 0 0 M O B E R L Y R O A D

500STARBOARDSQUARE

6 0 0 M I L L B A N K

600 BUCKETWHEEL 1

9 0 0 M I L L B A N K

STAMP'S LANDING

1900 MO

BE RLY

RO AD

518 609

525

500

1859

633 527

619

1873 456

601

(10)

(11)

[Vancouver Marijuana Grow-op Slide

omitted for privacy and security reasons.]

(12)

VanMap technical components

• Oracle Spatial database

• Other databases

• CAD drawings, satellite imagery, photographs, html pages

• Autodesk MapGuide

• Autodesk ActiveX Viewer

• Application servers

• Web server

(13)

A dynamic system

• Some data are overwritten without being saved

• The data are viewed as maps but these views are not saved

• New layers are being added all the time

(14)

Is VanMap preservable?

• Yes, if we introduce fixed form and stable content

• We need to configure the system so that as each layer is updated it is saved rather than overwritten

• Then we need to develop a means of

reproducing VanMap as it was on any given

date

(15)

Building a preservation environment…

• Step 1: save the layers

• Step 2: add metadata to the layers

• Step 3: store the data in a secure environment

• Step 4: create infrastructure independence

• Step 5: migrate to new/neutral technology platforms

• Step 6: reproduce VanMap

(16)

…using data grid technology

• Software developed by San Diego

Supercomputer Center to manage large volumes of data

• Implemented as the Storage Resource Broker (SRB) which manages several large data

repositories

(17)

Data grid technology

• Manages data and their associated metadata

(18)

Data grid technology

• Manages data and their associated metadata

• Separates the data from dependence on

original creating infrastructure

(19)

Data grid technology

• Manages data and their associated metadata

• Separates the data from dependence on original creating infrastructure

• Maintains audit trails of all operations

performed on the data

(20)

Data grid technology

• Manages data and their associated metadata

• Separates the data from dependence on original creating infrastructure

• Maintains audit trails of all operations performed on the data

• Manages access and retrieval

(21)

Data grid technology

• Manages data and their associated metadata

• Separates the data from dependence on original creating infrastructure

• Maintains audit trails of all operations performed on the data

• Manages access and retrieval

• Supports migration of data to new platforms

(22)

Data grids and VanMap

The scenario:

• Data grid is inserted between the data

storage systems and the access applications

• Each saved layer within the GIS is

independently registered in the data grid

• Date-based queries are used to reproduce

VanMap layers

(23)

Testing the data grid

The test:

• Selected data transferred from Vancouver to San Diego Supercomputer Center

• Data stored in technological environment similar to original environment

• Data registered in an SRB data grid

• Data queried for specific dates

• Queried data loaded into a different GIS product

(24)

Testing the data grid

VanMap Background Data:

• Majorstreets 5/1/04

• Webcam 3/21/04

• Can_wat 1/1/05

• Parcels 1/1/06

• Cityline 1/1/06

• Cityhall 1/1/06

• Sealine 1/1/06

• Shoreline 1/1/06

(25)

Testing the data grid

VanMap Temporal Data:

• ParkingMeterLines – 1/1/2003

– 1/1/2004

• ParkingMeterPoints – 2/1/2002

– 2/1/2004

• Streets

– 6/1/2003 – 6/1/2004

• Zoning

– 11/4/2004

– 11/30/2005

(26)

(27)

(28)

What gets preserved?

• The data themselves must be preserved

• The ability to render the data as interactive maps must be preserved

• Presentation elements such as colours and

fonts do not necessarily have to be preserved

(29)

(30)