fix(020_13, 020_14): temporal pagination follows TS 104 176 clause 6.4.7.3

- Mqtt tests (058) : 

  - Mqtt tests launch a Mosquitto container with docker, thus it requires docker to be installed and running

- Temporal pagination (020_13, 020_14, 021_15 & 021_16) :

  - The pagination tests assume that it will be triggered by 1 or 2 entities with 2 attributes with 20 instances each

    (40 instances)

  - Non-pagination tests assume that it will **not** be triggered by 2 entities with 2 attributes with 5 instances each

    (20 instances)

  - You should configure your broker accordingly

- Interoperability tests (IOP_CNF_*) :

  - The Interoperability tests assume that it will manage multiple brokers

  - Multiple NGSI-LD brokers will be running simultaneously on different ports

*** Settings ***

Documentation       Check temporal pagination is applied when querying the temporal evolution of entities

Documentation       Check that temporal pagination (TS 104-176, clause 6.4.7.3) is applied when querying the temporal evolution of entities

Library             DateTime

Resource            ${EXECDIR}/resources/ApiUtils/Common.resource

Suite Setup         Setup Initial Entities

Suite Teardown      Delete Initial Entities

Test Template       Retrieve Temporal Entities

Test Template       Query Temporal Entities With Pagination

*** Variables ***

${second_vehicle_payload_file}=     2020-09-vehicle-temporal-representation.jsonld

${timeBefore}=                      2019-01-01T01:01:00Z

${timeAfter}=                       2021-01-01T01:01:00Z

${lrt}=                             least recent timestamp

${mrt}=                             most recent timestamp

*** Test Cases ***

021_15_01 Retrieve The Entity With lastN And Timerel Before

    [Tags]    te-retrieve    core_11_3_3    http_6_4_7_3    since_v1.5.1

    lastN=${20}    expectedSize=20    timerel=before    timeAt=${timeAfter}    expectedRangeStart=${timeAfter}    expectedRangeEnd=${lrt}

021_15_02 Retrieve The Entity With lastN And Timerel Between

    [Tags]    te-retrieve    core_11_3_3    http_6_4_7_3    since_v1.5.1

    lastN=${20}    timerel=between    timeAt=${timeBefore}    endTimeAt=${timeAfter}    expectedRangeStart=${timeAfter}    expectedRangeEnd=${lrt}

021_15_03 Retrieve The Entity With lastN And Timerel After

    [Tags]    te-retrieve    core_11_3_3    http_6_4_7_3    since_v1.5.1

    lastN=${20}    expectedSize=20    timerel=after    timeAt=${timeBefore}    expectedRangeStart=${mrt}    expectedRangeEnd=${lrt}

021_15_04 Retrieve The Entity With Timerel Before

    [Tags]    te-retrieve    core_11_3_3    http_6_4_7_3    since_v1.5.1

    timerel=before    expectedSize=*    timeAt=${timeAfter}    expectedRangeEnd=${mrt}

021_15_05 Retrieve The Entity With Timerel Between

    [Tags]    te-retrieve    core_11_3_3    http_6_4_7_3    since_v1.5.1

    timerel=between    timeAt=${timeBefore}    endTimeAt=${timeAfter}    expectedRangeStart=${timeBefore}    expectedRangeEnd=${mrt}

021_15_06 Retrieve The Entity With Timerel After

    [Tags]    te-retrieve    core_11_3_3    http_6_4_7_3    since_v1.5.1

    timerel=after    timeAt=${timeBefore}    expectedRangeStart=${timeBefore}    expectedRangeEnd=${mrt}

021_15_07 Retrieve The Entity With temporalValues And Timerel After

    [Tags]    te-retrieve    core_11_3_3    http_6_4_7_3    since_v1.5.1

    representation=temporalValues    timerel=after    timeAt=${timeBefore}    expectedRangeStart=${timeBefore}    expectedRangeEnd=${mrt}

021_15_08 Retrieve The Entity With temporalValues, lastN And Timerel Between

    [Tags]    te-retrieve    core_11_3_3    http_6_4_7_3    since_v1.5.1

    representation=temporalValues    lastN=${20}    timerel=between    timeAt=${timeBefore}    endTimeAt=${timeAfter}    expectedRangeStart=${timeAfter}    expectedRangeEnd=${lrt}

021_15_01 Query With lastN Smaller Than The Available Instances

    [Documentation]    lastN paginates in descending order; the deeper page holds EARLIER timestamps → rel="intervalbefore"

    [Tags]    te-query    core_11_3_3    http_6_4_7_3    since_ts104_v1.1.1

    lastN=${5}    timerel=between    timeAt=${timeBefore}    endTimeAt=${timeAfter}

    ...    maxInstances=5    expectBeforeOffset=5

021_15_02 Query The Second Page With lastN And offsetN

    [Documentation]    A middle page carries both pointers

    [Tags]    te-query    core_11_3_3    http_6_4_7_3    since_ts104_v1.1.1

    lastN=${5}    offsetN=${5}    timerel=between    timeAt=${timeBefore}    endTimeAt=${timeAfter}

    ...    maxInstances=5    expectBeforeOffset=10    expectAfterOffset=0

021_15_03 Query With firstN

    [Documentation]    firstN paginates in ascending order; the deeper page holds LATER timestamps → rel="intervalafter"

    [Tags]    te-query    core_11_3_3    http_6_4_7_3    since_ts104_v1.1.1

    firstN=${5}    timerel=after    timeAt=${timeBefore}

    ...    maxInstances=5    expectAfterOffset=5

021_15_04 Query The Last Page With firstN And offsetN

    [Documentation]    Past the largest attribute's instance count only the backward pointer remains

    [Tags]    te-query    core_11_3_3    http_6_4_7_3    since_ts104_v1.1.1

    firstN=${5}    offsetN=${18}    timerel=after    timeAt=${timeBefore}

    ...    maxInstances=2    expectBeforeOffset=13

021_15_05 Query With lastN Covering All Instances

    [Documentation]    Nothing remains beyond the page — no pagination pointers

    [Tags]    te-query    core_11_3_3    http_6_4_7_3    since_ts104_v1.1.1

    lastN=${20}    timerel=after    timeAt=${timeBefore}

    ...    maxInstances=20

021_15_06 Query With temporalValues And lastN

    [Documentation]    Pagination pointers are representation-independent

    [Tags]    te-query    core_11_3_3    http_6_4_7_3    since_ts104_v1.1.1

    representation=temporalValues    lastN=${5}    timerel=after    timeAt=${timeBefore}

    ...    expectBeforeOffset=5

021_15_07 Query With Both firstN And lastN

    [Documentation]    Ascending and descending pagination at once is contradictory

    [Tags]    te-query    core_11_3_3    http_6_4_7_3    since_ts104_v1.1.1

    firstN=${5}    lastN=${5}    timerel=after    timeAt=${timeBefore}

    ...    expectedStatus=400

021_15_08 Query With limit And firstN

    [Documentation]    Both levels of pagination in one response: one Entity per page, so a pointer to the next page of

    ...    Entities (rel="next", clause 6.4.7.2), and two instances per Attribute, so a pointer to the next page of

    ...    instances (rel="intervalafter", clause 6.4.7.3) - both vehicles have more than two

    [Tags]    te-query    core_11_3_3    http_6_4_7_2    http_6_4_7_3    since_ts104_v1.1.1

    firstN=${2}    limit=${1}    timerel=after    timeAt=${timeBefore}

    ...    maxInstances=2    expectAfterOffset=2    expectNext=${True}

*** Keywords ***

Retrieve Temporal Entities

    [Documentation]    Check temporal pagination behavior on multiple entity endpoint

Query Temporal Entities With Pagination

    [Documentation]    Query the temporal evolution of entities and check the § 6.4.7.3 pagination behaviour

    [Arguments]

    ...    ${representation}=${EMPTY}

    ...    ${timerel}=${EMPTY}

    ...    ${timeAt}=${EMPTY}

    ...    ${endTimeAt}=${EMPTY}

    ...    ${lastN}=${EMPTY}

    ...    ${expectedRangeStart}=${EMPTY}

    ...    ${expectedRangeEnd}=${EMPTY}

    ...    ${expectedSize}=${EMPTY}

    ...    ${firstN}=${EMPTY}

    ...    ${offsetN}=${EMPTY}

    ...    ${limit}=${EMPTY}

    ...    ${maxInstances}=${EMPTY}

    ...    ${expectAfterOffset}=${EMPTY}

    ...    ${expectBeforeOffset}=${EMPTY}

    ...    ${expectNext}=${False}

    ...    ${expectedStatus}=200

    ${entity_types_to_be_retrieved}=    Catenate    SEPARATOR=,    Vehicle

    ${response}=    Query Temporal Representation Of Entities

    ...    timeAt=${timeAt}

    ...    endTimeAt=${endTimeAt}

    ...    lastN=${lastN}

    Check Response Status Code    206    ${response.status_code}

    ${contentRange}=    Get Regexp Matches

    ...    ${response.headers}[Content-Range]

    ...    ([a-zA-Z\-]+) (.*-.*-.*)-(.*-.*-.*)\/(.*)

    ...    1

    ...    2

    ...    3

    ...    4

    ${unit}=    Set Variable    ${contentRange}[0][0]

    ${rangeStart}=    Convert Date    ${contentRange}[0][1]

    ${rangeEnd}=    Convert Date    ${contentRange}[0][2]

    ${size}=    Set Variable    ${contentRange}[0][3]

    Check Content Range Part Equal    ${unit}    date-time

    IF    $expectedRangeStart != ''

        IF    $expectedRangeStart == $lrt

            ${expectedRangeStart}=    Get Least Recent Timestamp From Vehicles    ${response.json()}

        ELSE IF    $expectedRangeStart == $mrt

            ${expectedRangeStart}=    Get Most Recent Timestamp From Vehicles    ${response.json()}

    ...    firstN=${firstN}

    ...    offsetN=${offsetN}

    ...    limit=${limit}

    Check Response Status Code    ${expectedStatus}    ${response.status_code}

    IF    '${expectedStatus}' != '200'    RETURN

    IF    '${limit}' != ''    Length Should Be    ${response.json()}    ${limit}

    IF    ${expectNext}

        Check Pagination Link    ${response.headers}    next

    ELSE

            ${expectedRangeStart}=    Convert Date    ${expectedRangeStart}

        Check Pagination Link Absent    ${response.headers}    next

    END

        Check Content Range Part Equal    ${expectedRangeStart}    ${rangeStart}

    IF    '${maxInstances}' != '' and '${representation}' == ''

        Check Temporal Page Size    ${response.json()}    ${maxInstances}

    END

    IF    $expectedRangeEnd != ''

        IF    $expectedRangeEnd == $lrt

            ${expectedRangeEnd}=    Get Least Recent Timestamp From Vehicles

            ...    ${response.json()}

            ...    ${representation}

        ELSE IF    $expectedRangeEnd == $mrt

            ${expectedRangeEnd}=    Get Most Recent Timestamp From Vehicles

            ...    ${response.json()}

            ...    ${representation}

    IF    '${expectAfterOffset}' != ''

        Check Pagination Link    ${response.headers}    intervalafter    ${expectAfterOffset}

    ELSE

            ${expectedRangeEnd}=    Convert Date    ${expectedRangeEnd}

        END

        Check Content Range Part Equal    ${expectedRangeEnd}    ${rangeEnd}

        Check Pagination Link Absent    ${response.headers}    intervalafter

    END

    IF    $expectedSize != ''

        Check Content Range Part Equal    ${expectedSize}    ${size}

    IF    '${expectBeforeOffset}' != ''

        Check Pagination Link    ${response.headers}    intervalbefore    ${expectBeforeOffset}

    ELSE

        Check Pagination Link Absent    ${response.headers}    intervalbefore

    END

Setup Initial Entities

Delete Initial Entities

    Delete Temporal Representation Of Entity    ${first_temporal_entity_representation_id}

    Delete Temporal Representation Of Entity    ${second_temporal_entity_representation_id}

Get Least Recent Timestamp From Vehicles

    [Arguments]    ${body}    ${representation}=${EMPTY}

    ${attributeList}=    Get Attribute List From Vehicle Body    ${body}    ${representation}

    ${leastRecentTimestamp}=    Convert Date    ${timeAfter}

    FOR    ${attribute}    IN    @{attributeList}

        IF    $representation == 'temporalValues'

            ${attributeTime}=    Convert Date    ${attribute}[1]

        ELSE

            ${attributeTime}=    Convert Date    ${attribute}[observedAt]

        END

        IF    $leastRecentTimestamp >= $attributeTime

            ${leastRecentTimestamp}=    Set Variable    ${attributeTime}

        END

    END

    RETURN    ${leastRecentTimestamp}

Get Most Recent Timestamp From Vehicles

    [Arguments]    ${body}    ${representation}=${EMPTY}

    ${attributeList}=    Get Attribute List From Vehicle Body    ${body}    ${representation}

    ${mostRecentTimestamp}=    Convert Date    ${timeBefore}

    FOR    ${attribute}    IN    @{attributeList}

        IF    $representation == 'temporalValues'

            ${attributeTime}=    Convert Date    ${attribute}[1]

        ELSE

            ${attributeTime}=    Convert Date    ${attribute}[observedAt]

        END

        IF    $mostRecentTimestamp <= $attributeTime

            ${mostRecentTimestamp}=    Set Variable    ${attributeTime}

        END

    END

    RETURN    ${mostRecentTimestamp}

Get Attribute List From Vehicle Body

    [Arguments]    ${body}    ${representation}=${EMPTY}

    IF    $representation == 'temporalValues'

        ${attributeList}=    Combine Lists

        ...    ${body}[0][speed][values]

        ...    ${body}[0][fuelLevel][values]

        ...    ${body}[1][speed][values]

        ...    ${body}[1][fuelLevel][values]

    ELSE

        ${attributeList}=    Combine Lists

        ...    ${body}[0][speed]

        ...    ${body}[0][fuelLevel]

        ...    ${body}[1][speed]

        ...    ${body}[1][fuelLevel]

    END

    RETURN    ${attributeList}

*** Settings ***

Documentation       Check temporal pagination is applied when querying the temporal evolution of entities via POST

Documentation       Check that temporal pagination (TS 104-176, clause 6.4.7.3) is applied when querying the temporal evolution of entities via POST

Resource            ${EXECDIR}/resources/ApiUtils/Common.resource

Resource            ${EXECDIR}/resources/ApiUtils/TemporalContextInformationConsumption.resource

*** Test Cases ***    PAYLOAD_FILE

021_16_01 Retrieve The Entities Via Post

    [Tags]    te-retrieve    core_11_3_3    http_6_4_7_3    http_8_7_3_1    since_v1.5.1

    entity-operations-before-query.jsonld

    [Tags]    te-retrieve    core_11_3_3    http_6_4_7_3    http_8_7_3_1    since_ts104_v1.1.1

    entity-operations-paginated-query.jsonld

*** Keywords ***

Retrieve Temporal Entities Via Post

    [Documentation]    Check that temporal pagination is triggered on the post temporal query

    [Documentation]    Check that temporal pagination (TS 104-176, clause 6.4.7.3) is triggered on the post temporal query

    [Arguments]    ${payload_file}

    ${response}=    Query Temporal Representation Of Entities Via Post

    ...    query_file_name=${payload_file}

    ...    context=${ngsild_test_suite_context}

    Check Response Status Code    206    ${response.status_code}

    Check Response Status Code    200    ${response.status_code}

    ${contentRange}=    Get Regexp Matches

    ...    ${response.headers}[Content-Range]

    ...    ([a-zA-Z\-]+) (.*-.*-.*)-(.*-.*-.*)\/(.*)

    ...    1

    ...    2

    ...    3

    ...    4

    ${unit}=    Set Variable    ${contentRange}[0][0]

    Check Content Range Part Equal    ${unit}    date-time

    # temporalQ.lastN=5 against a 20-instance fixture: instances remain

    # beyond the page in descending order → rel="intervalbefore" pointer.

    Check Pagination Link    ${response.headers}    intervalbefore    5

Setup Initial Entities

    ${first_temporal_entity_representation_id}=    Generate Random Vehicle Entity Id

*** Settings ***

Documentation       Check that the time range cut before the second attribute to avoid missing data in content-range

Documentation       Check that unsynchronized attributes paginate independently (TS 104-176, clause 6.4.7.3: the page limit applies per Attribute)

Resource            ${EXECDIR}/resources/ApiUtils/Common.resource

Resource            ${EXECDIR}/resources/ApiUtils/TemporalContextInformationConsumption.resource

*** Test Cases ***

020_14_01 Retrieve An Entity With 60 Instances Of Unsynchronized Attributes

    [Tags]    te-retrieve    core_11_3_2    http_6_4_7_3    since_v1.5.1

    timerel=after    timeAt=${timeBefore}    emptyAttr=fuelLevel

020_14_02 Retrieve The Entity With lastN

    [Tags]    te-retrieve    core_11_3_2    http_6_4_7_3    since_v1.5.1

    lastN=${100}    timerel=before    timeAt=${timeAfter}    emptyAttr=speed

020_14_01 Retrieve An Entity With Unsynchronized Attributes And firstN

    [Documentation]    Both attributes return their own first page even though their time ranges do not overlap

    [Tags]    te-retrieve    core_11_3_2    http_6_4_7_3    since_ts104_v1.1.1

    firstN=${30}    timerel=after    timeAt=${timeBefore}    expectedInstances=30

020_14_02 Retrieve An Entity With Unsynchronized Attributes And lastN

    [Documentation]    Both attributes return their own last page even though their time ranges do not overlap

    [Tags]    te-retrieve    core_11_3_2    http_6_4_7_3    since_ts104_v1.1.1

    lastN=${30}    timerel=before    timeAt=${timeAfter}    expectedInstances=30

*** Keywords ***

Retrieve Temporal Entity

    [Documentation]    Check that the time range cut before the second attribute

    [Documentation]    Check that the per-attribute page limit applies to each unsynchronized attribute independently

    [Arguments]

    ...    ${timerel}

    ...    ${timeAt}

    ...    ${emptyAttr}

    ...    ${expectedInstances}

    ...    ${lastN}=${EMPTY}

    ...    ${firstN}=${EMPTY}

    ${response}=    Retrieve Temporal Representation Of Entity

    ...    temporal_entity_representation_id=${temporal_entity_representation_id}

    ...    context=${ngsild_test_suite_context}

    ...    timerel=${timerel}

    ...    timeAt=${timeAt}

    ...    lastN=${lastN}

    Check Data Is Empty    ${response.json()}[${emptyAttr}]

    ...    firstN=${firstN}

    Check Response Status Code    200    ${response.status_code}

    ${speedCount}=    Get Length    ${response.json()}[speed]

    ${fuelCount}=    Get Length    ${response.json()}[fuelLevel]

    Should Be Equal As Integers    ${speedCount}    ${expectedInstances}

    Should Be Equal As Integers    ${fuelCount}    ${expectedInstances}

Create Temporal Entity

    ${temporal_entity_representation_id}=    Generate Random Vehicle Entity Id