Merge branch 'chore/remove-annex-d' into 'master'

# Annex D [(informative)]{.inform}: Transformation algorithms

# Annex D [(informative)]{.inform}: RDF-compatible specification of NGSI-LD meta-model

## D.1 Introduction

These algorithms are informative but NGSI-LD implementations should aim at

either implementing them as they are described here or devising similar

algorithms which take exactly the same input and provides exactly the same

output (or an equivalent one as per the JSON-LD specification [n.8]).

## D.2 Algorithm for transforming an NGSI-LD Entity into a JSON-LD document (ALG1)

This algorithm takes as input an NGSI-LD graph which top level node is a

particular Entity and returns as output a JSON-LD document which represents all

the data associated to the entity. The JSON-LD document (and its associated

_`@context`_) corresponds to a representation of the Entity in JSON-LD as per

the NGSI-LD Information Model.

<!-- prettier-ignore-start -->

>>> [!note] NOTE:

An early implementation of this algorithm can be found at [i.21].

>>>

<!-- prettier-ignore-end -->

Let:

- **G** be a graph defined as follows:

  - Let **N** be G's top level node.

  - N is an Entity instance of type **T** or types **Ts**. Type name is "AliasT"

    or there is an Array of Type names ["AliasT1", ...,"AliasTn"], N's

    identifier is **I**.

  - N has 0 or more associated _Property_. Each _Property_ (**Psi**) is defined

    as follows:

    - _Property_ type identifier is **Pi**.

    - _Property_ name is "AliasPi".

    - _Property_ Value is **Vi**.

    - _Property_ Value's associated data type is **Di**.

  - N is the subject of 0 or more _Relationship_. Each _Relationship_ is defined

    as follows:

    - _Relationship_ type identifier is **Ri**.

    - _Relationship_ name is "AliasRi".

    - _Relationship_ target object identifier is **Robji**.

- **O** be a JSON object initialized to the empty object ({}).

- **C** be a JSON-LD _`@context`_ initialized as described by annex B.

The algorithm should run as follows, provided all the preconditions defined

above are satisfied:

1.  Add to C a new member <"AliasT", T> or new members <"AliasT1", T1> ...

    <"AliasTn", Tn>.

2.  Add to O two new members:

<!-- prettier-ignore-start -->

    a. <"id", I>.

    b. <"type", "AliasT"> or <"type", "AliasT1", ...,"AliasTn"> .">.

<!-- prettier-ignore-end -->

3.  For each _Property_ Psi (Pi, "AliasP", Vi, Di) associated to N:

<!-- prettier-ignore-start -->

    a.  Run Algorithm ALG1.1 taking the following inputs:

        - Ps → Psi.

        - O → O.

        - C → C.

<!-- prettier-ignore-end -->

4.  For each _Relationship_ Rs (Ri, AliasRi, Robji) associated to N:

<!-- prettier-ignore-start -->

    a.  Run Algorithm ALG1.2 taking the following inputs:

        - Rs → Rsi.

        - O → O.

        - C → C.

<!-- prettier-ignore-end -->

5.  Return (O, C) and end of the algorithm.

## D.3 Algorithm for transforming an NGSI-LD Property into JSON-LD (ALG1.1)

Let **Ps** be the _Property_ that has to be transformed. It is defined by (P,

"AliasP", V, D), where **P** denotes a _Property_ Type Id, "**AliasP**" is the

_Property_ name, **V** is the _Property_ Value and **D** is the _Property_

Value's data type.

Ps might be associated to extra _Properties_ or _Relationships_.

Let O be the output JSON-LD object and C the associated JSON-LD context:

1.  Execute the following steps:

<!-- prettier-ignore-start -->

    a.  If no member with "AliasP" is present in O, add a new member to O with

        key "AliasP" and value an object structure, let it be named **Op**

        <"type", "Property">.

        - If D is not a native JSON data type add a new member to Op with name

          "value" and which value has to be an object structure as follows:

          1.  <"&#64;type", D>.

          2.  <"&#64;value", V>.

        - Else If D is a native JSON data type add a new member to Op as follows:

          1.  <"value", V>.

    b.  Add a new member to C as follows:

        - <"AliasP", P>.

    c.  For each _Property_ associated to Ps (Pss)

        recursively run the present algorithm (ALG1.1) taking the following

        inputs:

        - Ps → Pss.

        - O → Op.

        - C → C.

    d.  For each  _Relationship_ associated to Ps (Rss) run

        algorithm ALG1.2 taking the following inputs:

        - Rs → Rss.

        - O → Op.

        - C → C.

<!-- prettier-ignore-end -->

2.  Return (O,C) and end of the algorithm.

## D.4 Algorithm for transforming an NGSI-LD Relationship into JSON-LD (ALG1.2)

Let **Rs** be the _Relationship_ that has to be transformed. It is defined by

(R, "AliasR", Robj), where **R** denotes a _Relationship_ Type Id, "**AliasR**"

is the _Relationship_'s name and **Robj** is the identifier of the target object

of the _Relationship_.

Rs might be associated to extra _Properties_ or _Relationships_.

Let O be the output JSON-LD object and C the current JSON-LD context:

1.  Execute the following statements:

<!-- prettier-ignore-start -->

    a.  If no member with "AliasR" is present in O, add a new member to O with

        key "AliasR" and value an object structure, let it be named **Or**

        - <"object", Robj>.

        - <"type", "Relationship">.

    b.  For each _Property_ associated to Rs (Pss) run the

        algorithm ALG1.1 taking the following inputs:

        - Ps → Pss.

        - O → Or.

        - C → C.

    c.  For each  _Relationship_ associated to Rs (Rss)

        recursively run the present algorithm ALG1.2 taking the following inputs:

        - Rs → Rss.

        - O → Or.

        - C → C.

<!-- prettier-ignore-end -->

2.  Return (O,C) and end of the algorithm.

The content of this annex is now in ETSI TS 104 178 [i.6].

# Annex E [(informative)]{.inform}: RDF-compatible specification of NGSI-LD meta-model

# Annex E [(informative)]{.inform}: Conventions and syntax guidelines

The content of this annex is now in ETSI TS 104 178 [i.6].

When new terms are defined, they are marked in bold, and terms are capitalized

thereafter.

<!-- prettier-ignore-start -->

>>> [!tip] EXAMPLE 1:

**NGSI-LD Linked Entity**, [Linked Entity]{.HTML-Keyboard}.

>>>

<!-- prettier-ignore-end -->

API Parameter names are always in lowercase.

<!-- prettier-ignore-start -->

>>> [!tip] EXAMPLE 2:

_`options`_.

>>>

<!-- prettier-ignore-end -->

Entity Types are defined using lowercase but with a starting capital letter.

<!-- prettier-ignore-start -->

>>> [!tip] EXAMPLE 3:

Vehicle, Building, ParkingSpace.

>>>

<!-- prettier-ignore-end -->

JSON-LD nodes and terms are always defined using camel case notation starting

with lower case.

<!-- prettier-ignore-start -->

>>> [!tip] EXAMPLE 4:

_`createdAt`_, _`value`_, _`unitCode`_.

>>>

<!-- prettier-ignore-end -->

When referring to special terms, data types or words defined previously in the

present document or by other referenced specifications, italics format is used.

<!-- prettier-ignore-start -->

>>> [!tip] EXAMPLE 5:

_ListRelationship_, _GeoProperty_, _Geometry_, _Second_, _Number_.

>>>

<!-- prettier-ignore-end -->

When referring to literal strings double quotes are used.

<!-- prettier-ignore-start -->

>>> [!tip] EXAMPLE 6:

["application/json"]{.HTML-Code}, ["Subscription"]{.HTML-Code}.

>>>

<!-- prettier-ignore-end -->

When referring to the JSON-LD Context the mnemonic text string _`@context`_ is

used as a placeholder.

All the dates and times are given in UTC format.

<!-- prettier-ignore-start -->

>>> [!tip] EXAMPLE 7:

[2018-02-09T11:00:00Z]{.HTML-Code}.

>>>

<!-- prettier-ignore-end -->

The measurement units used in the API are those defined by the International

System of Units.

<!-- prettier-ignore-start -->

>>> [!tip] EXAMPLE 8:

The distance in geo-queries is provided in meters.

>>>

<!-- prettier-ignore-end -->

When defining application-specific elements or API extensions the same

conventions and syntax guidelines should be followed.

# Annex F [(informative)]{.inform}: Conventions and syntax guidelines

# Annex F [(informative)]{.inform}: Localization and internationalization support

## F.1 Introduction

These algorithms described below are informative, but NGSI-LD implementations

should aim at either implementing them as they are described here or providing

equivalent _`@context`_ elements for their payloads to provide interoperability

with their internationalized context entities.

## F.2 Localization and internationalization

### F.2.1 Introduction

Since Internationalization is not core to context information management, any

direct support within NGSI-LD systems is limited. Annex+++root proposes a series

of best practices for maintaining, querying and displaying interoperable

internationalized data.

The content of the _`@context`_ utilized for the referred Entities within these

examples uses pre-existing URNs used for internationalization and is as follows:

```json

{

  "inLanguage": "http://schema.org/inLanguage",

  "sameAs": "http://schema.org/sameAs"

}

```

### F.2.2 Associating an Entity with a natural language

Where a context Entity is associated with a single natural language, include a

well-defined _Property_ indicating the natural language of the content. For

example an Event taking place in French may be defined as follows:

```json

{

  "type": "Event",

  "id": "urn:ngsi-ld:Event:bonjourLeMonde",

  "name": {

    "type": "Property",

    "value": "Bonjour le Monde"

  },

  "description": {

    "type": "Property",

    "value": "«Bonjour le monde» sont les mots traditionnellement écrits par un programme informatique simple"

  },

  "inLanguage": {

    "type": "Property",

    "value": "fr"

  }

}

```

### F.2.3 Associating a Property with a natural language

Where a _Property_ of a context entity can be associated to one more natural

language, include additional metadata as a sub-Property of that _Property_. For

example, a Hotel with booking forms available in English, French and German may

be defined as follows:

```json

{

  "type": "Hotel",

  "id": "urn:ngsi-ld:Hotel:XXXXX",

  "name": {

    "type": "Property",

    "value": "Grand Hotel"

  },

  "bookingUrl": {

    "type": "Property",

    "value": [

      "http://example.com/booking-in-french/",

      "http://example.com/booking-in-english/",

      "http://example.com/booking-in-german/"

    ],

    "inLanguage": {

      "type": "Property",

      "value": ["fr", "en", "de"]

    }

  }

}

```

### F.2.4 Associating as equivalent Entity

Where equivalent context entities in multiple natural languages exist, they may

be associated with each other through the use of a one-to-many _Relationships_,

where each _Relationships_ holds an additional sub-Property indicating the

natural language of the equivalent entities.

For example, three Events (such as a walking tour which is available in English,

French and German) may be associated to each other as follows:

```json

{

  "type": "Event",

  "id": "urn:ngsi-ld:Event:bonjourLeMonde",

  "name": {

    "type": "Property",

    "value": "Bonjour le Monde"

  },

  "sameAs": [

    {

      "type": "Relationship",

      "datasetId": "urn:ngsi-ld:Relationship:1",

      "object": "urn:ngsi-ld:Event:helloWorld",

      "inLanguage": {

        "type": "Property",

        "value": "en"

      }

    },

    {

      "type": "Relationship",

      "object": "urn:ngsi-ld:Event:halloWelt",

      "inLanguage": {

        "type": "Property",

        "value": "de"

      }

    }

  ]

}

```

## F.3 Natural language collation support

### F.3.1 Introduction

All strings within an NGSI-LD system are defined and sorted as a sequence of

Unicode characters. As such there is no simple collation mechanism to query

entities ignoring case, diacritic marks or matching diphthong single letters

such as the German ["ö"]{.HTML-Code} to also match with ["oe"]{.HTML-Code}.

Many databases support a degree of natural language support, in general

collation support will always depend upon the underlying database and as such

will vary from implementation to implementation. This therefore and cannot be

standardized and exposed as part of the context information management API.

Furthermore, collation is slow and processor intensive, and for massive systems

is better achieved using a separate index.

For systems that require it, this clause proposes a mechanism as an extension to

a NGSI-LD [Context Broker]{.HTML-Keyboard} which can be modified and used to

offer collation support to the natural language attributes found within context

entities where necessary through creating, querying and maintaining an

additional property of a property for collated attributes.

### F.3.2 Maintain collations as metadata

- Create a subscription on the attribute (e.g. _`name`_)

- Create a simple microservice to add/upsert a _`name.collate`_

  property-of-a-property using a simple function to strip all diacritic marks -

  for example: [str.normalize("NFD").replace(/\[\\u0300-\\u036f\]/g,

  "").toLower()]{.HTML-Sample}

Other substitutions could be made where local spelling rules vary (for example

different for German ö = oe).

### F.3.3 Route language sensitive queries via a proxy

Create a simple forwarding proxy around the NGSI-LD system. For any urls with a

_`q`_ param (and a _`collate`_ flag) run a clean-up of the _`q`_ param and amend

the query string:

The following request on the proxy:

- **GET**

  _`/ngsi-ld/v1/entities/?type=Building&q=name==%22Schöne%20Grüße%22&collate=name`_

is altered on the fly and is sent to the NGSI-LD system as shown:

- **GET**

  _`/ngsi-ld/v1/entities/?type=Building&q=name.collate==%22schoene%20gruesse%22`_

Once again, the substitutions to make to the query string will depend on the

rules of the natural language to be supported.

## F.4 Localization of dates, currency formats, etc.

### F.4.1 Introduction

Context data entities are designed to be interoperable and therefore all dates

are held as UTC dates, all currency amounts are held as JSON numbers (with the

unitCode property-of-a-property available to hold the currency), etc.

Localization should not occur within the context data entities themselves.

Offering fully localized responses is not a concern of the NGSI-LD API.

If localization support is necessary, a simple proxying a conversion mechanism

could be used to amend the context data received from the NGSI-LD system before

being passed to a third party system for display.

### F.4.2 Localizing dates

For example, if a system needs to display _DateTime_ data in Islamic Date format

The following request on the proxy:

- **GET**

  _`/ngsi-ld/v1/entities/urn:ngsi-ld:Event:XXX?attrs=date&format=simplified`_

is forwarded unaltered and is sent to the NGSI-LD system as shown:

- **GET**

  _`/ngsi-ld/v1/entities/urn:ngsi-ld:Event:XXX?attrs=date&format=simplified`_

When new terms are defined, they are marked in bold, and terms are capitalized

thereafter.

The response from the NGSI-LD system is always in UTC format:

<!-- prettier-ignore-start -->

```json

{

  "date": "2020-09-28T17:13:39+02:00"

}

```

>>> [!tip] EXAMPLE 1:

And the proxy can be used to update this to the desired format:

**NGSI-LD Linked Entity**, [Linked Entity]{.HTML-Keyboard}.

```json

{

  "date": "11 Safar, 1442 1:13:39PM"

}

```

>>>

Using an internationalization script such as the following:

<!-- prettier-ignore-end -->

```js

new Intl.DateTimeFormat("en-u-ca-islamic", {

  day: "numeric",

  month: "long",

  weekday: "long",

  year: "numeric",

}).format(date);

```

API Parameter names are always in lowercase.

<!-- prettier-ignore-start -->

>>> [!tip] EXAMPLE 2:

_`options`_.

>>>

<!-- prettier-ignore-end -->

Entity Types are defined using lowercase but with a starting capital letter.

<!-- prettier-ignore-start -->

>>> [!tip] EXAMPLE 3:

Vehicle, Building, ParkingSpace.

>>>

<!-- prettier-ignore-end -->

JSON-LD nodes and terms are always defined using camel case notation starting

with lower case.

<!-- prettier-ignore-start -->

>>> [!tip] EXAMPLE 4:

_`createdAt`_, _`value`_, _`unitCode`_.

>>>

<!-- prettier-ignore-end -->

When referring to special terms, data types or words defined previously in the

present document or by other referenced specifications, italics format is used.

<!-- prettier-ignore-start -->

>>> [!tip] EXAMPLE 5:

_ListRelationship_, _GeoProperty_, _Geometry_, _Second_, _Number_.

>>>

<!-- prettier-ignore-end -->

When referring to literal strings double quotes are used.

<!-- prettier-ignore-start -->

>>> [!tip] EXAMPLE 6:

["application/json"]{.HTML-Code}, ["Subscription"]{.HTML-Code}.

>>>

<!-- prettier-ignore-end -->

When referring to the JSON-LD Context the mnemonic text string _`@context`_ is

used as a placeholder.

All the dates and times are given in UTC format.

<!-- prettier-ignore-start -->

>>> [!tip] EXAMPLE 7:

[2018-02-09T11:00:00Z]{.HTML-Code}.

>>>

<!-- prettier-ignore-end -->

The measurement units used in the API are those defined by the International

System of Units.

<!-- prettier-ignore-start -->

>>> [!tip] EXAMPLE 8:

The distance in geo-queries is provided in meters.

>>>

<!-- prettier-ignore-end -->

When defining application-specific elements or API extensions the same

conventions and syntax guidelines should be followed.

It should be noted that post-localization, the transformed date is no longer

valid NGSI-LD.