Control de calidad en R Estándares de control posicional

(1)

Francisco Javier Ariza López

fjariza@ujaen.es

Universidad de Jaén (España)

Estándares de control posicional

Quito, Noviembre de 2016

(2)

Índice

• Introduction

• Prerequisites

• General View

• Some PAAMs

i i

i

t m

y y y

e  

i i

i

t m

x x x

e  

z z

e  

(3)

Francisco Javier Ariza‐Lópezfjariza@ujaen.es/ Universidad de Jaén (España)

Prereq Check 

Good analysis

Cartographic prerequisites  Positional interoperability of data Control work prerequisites  Quality of the source of reference Statistical prerequisites  Confidence on the statistical procedures

Prerrequisitos

(4)

Positional interoperability between the assesses product and the reference dataset.

Mathematical Cartography:

Ellipsoid, datum and projection This is called CRS (Coordinate Reference System)

CRS = Datum + Coordinate System

CRS = a CS related to the Earth by a Datum CS = set of mathematical rules for specifying Cartographic prerequisites

Prerrequisitos

(5)

Independence of the assessment work  no correlation, no same bias.

Accuracy  The accuracy of the assessment work (at least × 2‐ × 3, better ×5) the assessment work does not affect calculations.

There is no agreement (× 2, × 3, ×5, × 10). More usual case × 3. (at least) There are methods that consider "safety factors“.

Control work prerequisites

Product QC Method Estimation Introduced Error

1 1,000 1,414 41,42%

1 0,800 1,281 28,06%

1 0,600 1,166 16,62%

1 0,500 1,118 11,80%

1 0,400 1,077 7,70%

1 0,333 1,054 5,41%

1 0,250 1,031 3,08%

1 0,200 1,020 1,98%

1 0,167 1,014 1,38%

1 0,143 1,010 1,02%

1 0,125 1,008 0,78%

1 0,111 1,006 0,62%

1 0,100 1,005 0,50%

1 0,001 1,000 0,00%

0,800 0,900 1,000 1,100 1,200 1,300 1,400

0,000 0,200 0,400 0,600 0,800 1,000

ߪ _{ா௦௧௜௠௔௧௜௢௡} ൌ ߪ _௉ ^ଶ ൅ ߪ _ொ஼ ^ଶ ߪ _{ா௦௧௜௠௔௧௜௢௡}

ߪ _ொ஼

ߪ ߪ ߪ

Prerrequisitos

(6)

Prerequisites (measuring method accuracy 3x, etc.).

Bias = 1/3 σ Bias = 1/3 σ

Population

Sampling Sampling

% of the sample within [‐1σ, +1 σ] = 97,72%

% of the sample within [‐1σ, +1 σ] = 65.46%

Prerrequisitos

Control work prerequisites

(7)

Visión general de los MECP

(8)

Perspectives

How the product is?

How the production processes are?

How I can improve it?

How I can be more efficient?

How I can reduce liability of my products?

Is this product suitable for my application?

Is this product good enough for me?

Is this product integrable with my other datasets?

User’s point of view. Producer’s point of view.

Objectives

• Help to the users and producers to take better decisions.

• Standardize the positional accuracy assessment method.

• Transparency in market relations and supplies.

Visión general de los MECP

(9)

Perspectives

User’s point of view. Producer’s point of view.

When they want assess the quality, they can be interested in:

• To know, as good as possible, the exact level of uncertainty to assure the quality or, if the quality is not adequate, in order to improve the

production process and product characteristics.

• Estimation

• To decide if the product satisfies the

requirements. In this case we do not mind the true quality parameter, but only if it is adequate or not for our purposes.

• Hypothesis test (control)

μ, σ, ξ, α

Valor medio, desviación, precisión, significación, tamaño muestra

Tol, α, β

Tolerancia, significación, potencia (riesgos tipo I y II),

Visión general de los MECP

(10)

• Planimetry and altimetry independently, or altimetry adjustable by planimetry.

• Point-based methods (points, well-defined, well-distributed)

• Reduced sample (sample size: “at least 20”)

• Reference source of higher accuracy (accuracy “at least x3”)

• Majority of PAAM are based on the Normality of errors

• Weak definition of statistical prerequisites and absence of specifications on the control of statistical prerequisites.

• Report  Scarce information about the entirely process.

Main common facts

Visión general de los MECP

(11)

Points:

• Well-defined points: A well‐defined point represents a feature for which the horizontal position is known to a high degree of accuracy and position with respect to the geodetic datum. For the purpose of accuracy testing, well‐defined points must be easily visible or recoverable on the ground, on the independent source of higher accuracy, and on the product itself. The selected points will differ depending on the type of dataset and output scale of the dataset. For graphic maps and vector data, suitable well‐defined points represent right‐angle intersections of roads, railroads, or other linear mapped features, such as canals, ditches, trails, fence lines, and pipelines. For orthoimagery, suitable well‐defined points may represent features such as small isolated shrubs or bushes, in addition to right‐

angle intersections of linear features. For map products at scales of 1:5,000 or larger, such as engineering plats or property maps, suitable well‐defined points may represent additional features such as utility access covers and intersections of sidewalks, curbs, or gutters.

Visión general de los MECP

Control elements

(12)

Points:

• Well-defined points:

• Well-distributed points

Visión general de los MECP

Control elements

(13)

Sample size:

• “at least 20” is a repeated sentence.

• In general the sample size is not related with the size of the population of controlled elements or area.

• The value 20 is related with the central limit theorem in statistics.

• The value 20 is related to the approximation to normal distribution.

• This value may be adequate for hypothesis testing but not for parameters estimation.

• No guidance for substitution.

• The more control points  the more $ cost

Visión general de los MECP

Sample

(14)

Reference source:

• of higher accuracy (“at least x3”)

• Usually, the accuracy of the reference source is not taken into account in the computations, it is only a condition.

Visión general de los MECP

Source

(15)

Weak definition of statistical prerequisites and absence of specifications on the control of statistical prerequisites.

Some common prerequisites (hypothesis) are:

 randomness

 absence of outliers

 independency between error components

 variational behavior is similar for X and Y (σx≈ σy)

 Normality of error data

PAAMs do not provide guidance for they control.

In general, nobody test these hypotheses!!!

Visión general de los MECP

Statistical prerequisites

(16)

Limited reporting about the entirely process

Report centered on results, no information about the process

PAAMs propose a “sentence” to inform about the result but the don't inform about many important issues (prerequisites, quality of the reference,

management of bias, management of outliers, etc.). No graphic results are

ASPRS

Visión general de los MECP

Report

(17)

• Weak definition of statistical prerequisites and absence of specifications on the control of statistical prerequisites.

• Weak definition of conditions (e.g. RMSEx=RMSEy in the NSSDA).

• Significance level: Simultaneously application of several statistical test without considering Bonferroni, or other similar techniques.

• Scarce information about the entirely process.

• Based on the normality of errors.

• They consider different probabilities for limiting errors (90%, 95%...)

• They consider different graphical thresholds for limiting errors (0.2mm, 0.25mm, 0.30mm...)

• Majority of them are not adequate for continuous supplies.

Summary of problems

Visión general de los MECP

(18)

Common problems of people applying the standards

• Combination of radial (horizontal) accuracies with linear accuracies.

• Confusion between planimetic (radial, 2D) values) and X/Y values (1D).

• Apply RMSE instead of SIGMA.

• Apply normal‐based expansion factors to the RMSE (when μ≠0).

• Apply normal‐based expansion factors to non‐normal/Chi2 data.

• Apply 1D expansion factors to the 2D case.

• Apply 2D expansion factors to the 1D case.

• Elimination of outliers from the report.

• Confusion when considering the tail of a distribution function (right or left?).

• Poor report.

Visión general de los MECP

Summary of problems

(19)

There are a lot of PAAMs:

• EMAS (Engineering Map Accuracy Standard) (1983), By The American Society of Civil Engineers

• ASPRS‐ASLSM (ASPRS Accuracy Standards for large‐scale Maps) (1989), By the American Society of Photogrammetry and Remote Sensing.

• AS4P (Accuracy Standards for positioning V 1.0) (1996), By Geomatics Canada.

• NSSDA (National Standard for Spatial Data Accuracy) (1998), By The Federal Geographic Data Committee.

• STANAG 2215 (Evaluation of maps, aeronatical charts and digital topographic data) (2002), By NATO.

• CPATT (France 2003 Arrêté du 16 septembre 2003 portant sur les classes de précision applicables aux catégories de travaux topographiques réalisés par l'Etat, les collectivités locales et leurs

établissements publics ou exécutés pour leur compte) (2003).

• MVMASG (Model Virginia Map Accuracy Standards Guideline)(2009), By Virginia Information Technologies Agency.

• AMSDHAS (Australian Map and Spatial Data Horizontal Accuracy Standard) (2009), By ICSM.

• Kontroll av Geodata (2013), By Norska standarden.

• ASPRS‐PAS4DGD (ASPRS Positional Accuracy Standards for Digital Geospatial Data) (2015), By the American Society of Photogrammetry and Remote Sensing.

Algunos MECP

(20)

• NMAS (National Map Accuracy Standard) (1947), by the United States Bureau of the Budget.

• NIST‐ASM4BP (An Acceptance Sampling Method for 2D/3D Building Plans) (2009), by National Institute of Standards and Technology.

• ISO 2859‐1. Sampling procedures for inspection by attributes — Part 1: Sampling schemes indexed by acceptance quality limit (AQL) for lot‐by‐lot inspection (1999), by ISO.

• IPGH Especificaciones topograficas (1978), By the Instituto Panamericano de Geografía e Historia.

• Brasil (1984). Normas Técnicas da Cartografia Nacional, Decreto nº 89.817, de 20 de junho de 1984.

• ISO 3951‐1 Sampling procedures for inspection by variables ‐‐ Part 1: Specification for single sampling plans indexed by acceptance quality limit (AQL) for lot‐by‐lot inspection for a single quality characteristic and a single AQL, (2005), By ISO.

• UNE 148002 Metodología de evaluación de la exactitud posicional de la información geográfica (2016), by AENOR.

• Positional control by two tolerances (2016), by Ariza‐López & Rodríguez‐Avi, 2016.

…and many others…¡¡¡

Many of them are measured‐based methods

Algunos MECP

There are a lot of PAAMs:

(21)

NMAS: H & V. Very simple calculations. Ease of understanding (pass / fail). Values on the "paper“

and fixed tolerances. Statistics unexplained. {n≥20, ×?}

EMAS: X, Y, Z. Complex calculations. 4 hypothesis test for planimetry + 2 hypothesis test for altimetry. Values on the "ground". The error limits can be user‐defined. It allows to know the process and its problems (bias, precision). Significance levels with problems. It assumes that bias has been eliminated and that errors are normally distributed but no control process is proposed.

{n≥?, ×3}

ASPRS‐1990: H & V. Very simple calculations. Ease of understanding. Values on the “ground” and fixed tolerances related with scales. Proposes 3 classes of accuracy. Assumes that blunders have been corrected previously but no control process is proposed. {n≥20, ×?}

NSSDA: H & V. Simple calculations. Values on the “ground”. No classes of accuracy or tolerances.

The user must interpret the result. It assumes that bias has been eliminated, errors are normally distributed and independent but no control process is proposed. {n≥20, ×?}

Control de calidad en R Estándares de control posicional

Francisco Javier Ariza López

fjariza@ujaen.es

Universidad de Jaén (España)

Estándares de control posicional

Quito, Noviembre de 2016

Índice

• Introduction

• Prerequisites

• General View

• Some PAAMs

t m

y y y

e  

t m

x x x

e  

z z

e  

Prereq Check 

Good analysis

Cartographic prerequisites  Positional interoperability of data Control work prerequisites  Quality of the source of reference Statistical prerequisites  Confidence on the statistical procedures

Prerrequisitos

Positional interoperability between the assesses product and the reference dataset.

Mathematical Cartography:

Ellipsoid, datum and projection This is called CRS (Coordinate Reference System)

CRS = Datum + Coordinate System

CRS = a CS related to the Earth by a Datum CS = set of mathematical rules for specifying Cartographic prerequisites

Prerrequisitos

Independence of the assessment work  no correlation, no same bias.

Accuracy  The accuracy of the assessment work (at least × 2‐ × 3, better ×5) the assessment work does not affect calculations.

There is no agreement (× 2, × 3, ×5, × 10). More usual case × 3. (at least) There are methods that consider "safety factors“.

Control work prerequisites

ߪ ா௦௧௜௠௔௧௜௢௡ ൌ ߪ ௉ ଶ ൅ ߪ ொ஼ ଶ ߪ ா௦௧௜௠௔௧௜௢௡

ߪ ொ஼

ߪ ߪ ߪ

Prerrequisitos

Prerequisites (measuring method accuracy 3x, etc.).

Bias = 1/3 σ Bias = 1/3 σ

Population

Sampling Sampling

% of the sample within [‐1σ, +1 σ] = 97,72%

% of the sample within [‐1σ, +1 σ] = 65.46%

Prerrequisitos

Control work prerequisites

Visión general de los MECP

Perspectives

How the product is?

How the production processes are?

How I can improve it?

How I can be more efficient?

How I can reduce liability of my products?

Is this product suitable for my application?

Is this product good enough for me?

Is this product integrable with my other datasets?

User’s point of view. Producer’s point of view.

Objectives

• Help to the users and producers to take better decisions.

• Standardize the positional accuracy assessment method.

• Transparency in market relations and supplies.

Visión general de los MECP

Perspectives

User’s point of view. Producer’s point of view.

When they want assess the quality, they can be interested in:

• To know, as good as possible, the exact level of uncertainty to assure the quality or, if the quality is not adequate, in order to improve the

production process and product characteristics.

• Estimation

• To decide if the product satisfies the

requirements. In this case we do not mind the true quality parameter, but only if it is adequate or not for our purposes.

• Hypothesis test (control)

μ, σ, ξ, α

Tol, α, β

Visión general de los MECP

• Planimetry and altimetry independently, or altimetry adjustable by planimetry.

• Point-based methods (points, well-defined, well-distributed)

• Reduced sample (sample size: “at least 20”)

• Reference source of higher accuracy (accuracy “at least x3”)

• Majority of PAAM are based on the Normality of errors

• Weak definition of statistical prerequisites and absence of specifications on the control of statistical prerequisites.

• Report  Scarce information about the entirely process.

ߪ _{ா௦௧௜௠௔௧௜௢௡} ൌ ߪ _௉ ^ଶ ൅ ߪ _ொ஼ ^ଶ ߪ _{ா௦௧௜௠௔௧௜௢௡}

ߪ _ொ஼