Loading README.md +10 −6 Changes for README.md: 10 added lines, 6 removed lines. Original line number Diff line number Diff line Loading @@ -156,12 +156,6 @@ test launch command followed by the file name. - Mqtt tests (058) : - Mqtt tests launch a Mosquitto container with docker, thus it requires docker to be installed and running - Temporal pagination (020_13, 020_14, 021_15 & 021_16) : - The pagination tests assume that it will be triggered by 1 or 2 entities with 2 attributes with 20 instances each (40 instances) - Non-pagination tests assume that it will **not** be triggered by 2 entities with 2 attributes with 5 instances each (20 instances) - You should configure your broker accordingly - Interoperability tests (IOP_CNF_*) : - The Interoperability tests assume that it will manage multiple brokers - Multiple NGSI-LD brokers will be running simultaneously on different ports Loading Loading @@ -191,6 +185,16 @@ execute the following command: It will generate a specific `errors.log` file into the `results` folder with the description of the different steps developed and the mismatched observed on them. ### Keep the requests and responses out of the terminal Every request and response is written to the terminal as well as to `log.html`. On a full run that is thousands of them. With `ETSI_QUIET_IO=1` they go to `log.html` and `output.xml` only, and the terminal shows one line per Test Case: ```ETSI_QUIET_IO=1 robot --outputdir ./results ./TP/NGSI-LD``` The `Output` and `Output Notification` keywords also take `console=${False}` for a single call. ### Measure the API coverage of a run [scripts/api_coverage](./scripts/api_coverage) counts, from a run's `output.xml`, which operations of the NGSI-LD Loading TP/NGSI-LD/ContextInformation/Consumption/Entity/QueryEntities/019_02_07.robot +3 −3 Changes for TP/NGSI-LD/ContextInformation/Consumption/Entity/QueryEntities/019_02_07.robot: 3 added lines, 3 removed lines. Original line number Diff line number Diff line Loading @@ -14,13 +14,13 @@ Test Template Query entities via POST based on q *** Test Cases *** Q EXPECTED_COUNT 019_02_07_01 QueryOnNameEquality [Tags] e-query 5_7_2 [Tags] e-query core_10_4_3 name=="Eiffel Tower" 1 019_02_07_02 QueryOnNameEqualityAndBoolean [Tags] e-query 5_7_2 [Tags] e-query core_10_4_3 name=="Pisa Tower";almostFull==false 1 019_02_07_03 QueryOnNameEqualityOrBoolean [Tags] e-query 5_7_2 [Tags] e-query core_10_4_3 name=="Pisa Tower"|almostFull==false 3 Loading TP/NGSI-LD/ContextInformation/Consumption/Entity/QueryEntities/019_02_08.robot +6 −6 Changes for TP/NGSI-LD/ContextInformation/Consumption/Entity/QueryEntities/019_02_08.robot: 6 added lines, 6 removed lines. Original line number Diff line number Diff line Loading @@ -13,20 +13,20 @@ Test Template Query entities via POST based on q on a JsonProperty *** Test Cases *** Q JSON_KEYS EXPECTED_COUNT 019_02_08_01 QueryOnNameEquality [Tags] e-query 5_7_2 4_5_24 since_v1.8.1 019_02_08_01 QueryOnJsonPropertyIdEqual [Tags] e-query core_10_4_3 core_5_2_6_4_10 since_v1.8.1 jsonProperty[id]=="some-id" jsonProperty 1 019_02_08_02 QueryOnJsonPropertyTypeEqual [Tags] e-query 5_7_2 4_5_24 since_v1.8.1 [Tags] e-query core_10_4_3 core_5_2_6_4_10 since_v1.8.1 jsonProperty[type]=="some-type" jsonProperty 2 019_02_08_03 QueryOnJsonPropertyIdNoMatch [Tags] e-query 5_7_2 4_5_24 since_v1.8.1 [Tags] e-query core_10_4_3 core_5_2_6_4_10 since_v1.8.1 jsonProperty[id]=="wrong-id" jsonProperty 0 019_02_08_04 QueryOnJsonPropertyIdUnequal [Tags] e-query 5_7_2 4_5_24 since_v1.8.1 [Tags] e-query core_10_4_3 core_5_2_6_4_10 since_v1.8.1 jsonProperty[id]!="wrong-id" jsonProperty 2 019_02_08_05 QueryOnJsonPropertyUnmatchedJsonKeys [Tags] e-query 5_7_2 4_5_24 since_v1.8.1 [Tags] e-query core_10_4_3 core_5_2_6_4_10 since_v1.8.1 jsonProperty[id]=="some-id" anotherProperty 0 Loading TP/NGSI-LD/ContextInformation/Consumption/Entity/QueryEntities/019_02_09.robot +9 −9 Changes for TP/NGSI-LD/ContextInformation/Consumption/Entity/QueryEntities/019_02_09.robot: 9 added lines, 9 removed lines. Original line number Diff line number Diff line Loading @@ -22,28 +22,28 @@ ${SIGHTSEEING_URI} https://uri.etsi.org/ngsi-ld/default-context/sightseeing # first entity: vocabProperty = "monument" (string) # second entity: vocabProperty = ["monument", "sightseeing"] (array) 019_02_09_01 QueryWithExpandValuesEqualMonument [Tags] e-query 5_7_2 4_5_20 since_v1.7.1 [Tags] e-query core_10_4_3 core_5_2_6_4_7 since_v1.7.1 vocabProperty=="monument" vocabProperty 2 019_02_09_02 QueryWithExpandValuesEqualSightseeing [Tags] e-query 5_7_2 4_5_20 since_v1.7.1 [Tags] e-query core_10_4_3 core_5_2_6_4_7 since_v1.7.1 vocabProperty=="sightseeing" vocabProperty 1 019_02_09_03 QueryWithExpandValuesUnequalSightseeing [Tags] e-query 5_7_2 4_5_20 since_v1.7.1 019_02_09_03 QueryWithExpandValuesUnequalMonument [Tags] e-query core_10_4_3 core_5_2_6_4_7 since_v1.7.1 vocabProperty!="monument" vocabProperty 0 019_02_09_04 QueryWithUriMonument [Tags] e-query 5_7_2 4_5_20 since_v1.7.1 [Tags] e-query core_10_4_3 core_5_2_6_4_7 since_v1.7.1 vocabProperty=="${MONUMENT_URI}" ${EMPTY} 2 019_02_09_05 QueryWithUriSightseeing [Tags] e-query 5_7_2 4_5_20 since_v1.7.1 [Tags] e-query core_10_4_3 core_5_2_6_4_7 since_v1.7.1 vocabProperty=="${SIGHTSEEING_URI}" ${EMPTY} 1 019_02_09_06 QueryWithExpandValuesValueListEqual [Tags] e-query 5_7_2 4_5_20 since_v1.7.1 [Tags] e-query core_10_4_3 core_5_2_6_4_7 since_v1.7.1 vocabProperty=="monument","sightseeing" vocabProperty 2 019_02_09_07 QueryWithExpandValuesValueListUnequal [Tags] e-query 5_7_2 4_5_20 since_v1.7.1 [Tags] e-query core_10_4_3 core_5_2_6_4_7 since_v1.7.1 vocabProperty!="monument","sightseeing" vocabProperty 0 019_02_09_08 QueryWithUriValueList [Tags] e-query 5_7_2 4_5_20 since_v1.7.1 [Tags] e-query core_10_4_3 core_5_2_6_4_7 since_v1.7.1 vocabProperty=="${MONUMENT_URI}","${SIGHTSEEING_URI}" ${EMPTY} 2 Loading TP/NGSI-LD/ContextInformation/Consumption/Entity/QueryEntities/019_03_06.robot +7 −7 Changes for TP/NGSI-LD/ContextInformation/Consumption/Entity/QueryEntities/019_03_06.robot: 7 added lines, 7 removed lines. Original line number Diff line number Diff line Loading @@ -11,25 +11,25 @@ Test Template Query entities with invalid q *** Test Cases *** Q 019_03_06_01 QueryWithOperatorMissingValue [Tags] e-query 5_7_2 [Tags] e-query core_10_4_3 attr== 019_03_06_02 QueryWithUnclosedBracket [Tags] e-query 5_7_2 [Tags] e-query core_10_4_3 attr[ 019_03_06_03 QueryWithInvalidSingleExclamationOperator [Tags] e-query 5_7_2 [Tags] e-query core_10_4_3 attr!value 019_03_06_04 QueryStartingWithLogicalOperator [Tags] e-query 5_7_2 [Tags] e-query core_10_4_3 ;attr=="value" 019_03_06_05 QueryWithConsecutiveLogicalOperators [Tags] e-query 5_7_2 [Tags] e-query core_10_4_3 attr=="value";;attr2=="value2" 019_03_06_06 QueryWithInvertedRegexOperator [Tags] e-query 5_7_2 [Tags] e-query core_10_4_3 attr=~"value*" 019_03_06_07 QueryWithUnknownOperator [Tags] e-query 5_7_2 [Tags] e-query core_10_4_3 attr="value" Loading Loading
README.md +10 −6 Changes for README.md: 10 added lines, 6 removed lines. Original line number Diff line number Diff line Loading @@ -156,12 +156,6 @@ test launch command followed by the file name. - Mqtt tests (058) : - Mqtt tests launch a Mosquitto container with docker, thus it requires docker to be installed and running - Temporal pagination (020_13, 020_14, 021_15 & 021_16) : - The pagination tests assume that it will be triggered by 1 or 2 entities with 2 attributes with 20 instances each (40 instances) - Non-pagination tests assume that it will **not** be triggered by 2 entities with 2 attributes with 5 instances each (20 instances) - You should configure your broker accordingly - Interoperability tests (IOP_CNF_*) : - The Interoperability tests assume that it will manage multiple brokers - Multiple NGSI-LD brokers will be running simultaneously on different ports Loading Loading @@ -191,6 +185,16 @@ execute the following command: It will generate a specific `errors.log` file into the `results` folder with the description of the different steps developed and the mismatched observed on them. ### Keep the requests and responses out of the terminal Every request and response is written to the terminal as well as to `log.html`. On a full run that is thousands of them. With `ETSI_QUIET_IO=1` they go to `log.html` and `output.xml` only, and the terminal shows one line per Test Case: ```ETSI_QUIET_IO=1 robot --outputdir ./results ./TP/NGSI-LD``` The `Output` and `Output Notification` keywords also take `console=${False}` for a single call. ### Measure the API coverage of a run [scripts/api_coverage](./scripts/api_coverage) counts, from a run's `output.xml`, which operations of the NGSI-LD Loading
TP/NGSI-LD/ContextInformation/Consumption/Entity/QueryEntities/019_02_07.robot +3 −3 Changes for TP/NGSI-LD/ContextInformation/Consumption/Entity/QueryEntities/019_02_07.robot: 3 added lines, 3 removed lines. Original line number Diff line number Diff line Loading @@ -14,13 +14,13 @@ Test Template Query entities via POST based on q *** Test Cases *** Q EXPECTED_COUNT 019_02_07_01 QueryOnNameEquality [Tags] e-query 5_7_2 [Tags] e-query core_10_4_3 name=="Eiffel Tower" 1 019_02_07_02 QueryOnNameEqualityAndBoolean [Tags] e-query 5_7_2 [Tags] e-query core_10_4_3 name=="Pisa Tower";almostFull==false 1 019_02_07_03 QueryOnNameEqualityOrBoolean [Tags] e-query 5_7_2 [Tags] e-query core_10_4_3 name=="Pisa Tower"|almostFull==false 3 Loading
TP/NGSI-LD/ContextInformation/Consumption/Entity/QueryEntities/019_02_08.robot +6 −6 Changes for TP/NGSI-LD/ContextInformation/Consumption/Entity/QueryEntities/019_02_08.robot: 6 added lines, 6 removed lines. Original line number Diff line number Diff line Loading @@ -13,20 +13,20 @@ Test Template Query entities via POST based on q on a JsonProperty *** Test Cases *** Q JSON_KEYS EXPECTED_COUNT 019_02_08_01 QueryOnNameEquality [Tags] e-query 5_7_2 4_5_24 since_v1.8.1 019_02_08_01 QueryOnJsonPropertyIdEqual [Tags] e-query core_10_4_3 core_5_2_6_4_10 since_v1.8.1 jsonProperty[id]=="some-id" jsonProperty 1 019_02_08_02 QueryOnJsonPropertyTypeEqual [Tags] e-query 5_7_2 4_5_24 since_v1.8.1 [Tags] e-query core_10_4_3 core_5_2_6_4_10 since_v1.8.1 jsonProperty[type]=="some-type" jsonProperty 2 019_02_08_03 QueryOnJsonPropertyIdNoMatch [Tags] e-query 5_7_2 4_5_24 since_v1.8.1 [Tags] e-query core_10_4_3 core_5_2_6_4_10 since_v1.8.1 jsonProperty[id]=="wrong-id" jsonProperty 0 019_02_08_04 QueryOnJsonPropertyIdUnequal [Tags] e-query 5_7_2 4_5_24 since_v1.8.1 [Tags] e-query core_10_4_3 core_5_2_6_4_10 since_v1.8.1 jsonProperty[id]!="wrong-id" jsonProperty 2 019_02_08_05 QueryOnJsonPropertyUnmatchedJsonKeys [Tags] e-query 5_7_2 4_5_24 since_v1.8.1 [Tags] e-query core_10_4_3 core_5_2_6_4_10 since_v1.8.1 jsonProperty[id]=="some-id" anotherProperty 0 Loading
TP/NGSI-LD/ContextInformation/Consumption/Entity/QueryEntities/019_02_09.robot +9 −9 Changes for TP/NGSI-LD/ContextInformation/Consumption/Entity/QueryEntities/019_02_09.robot: 9 added lines, 9 removed lines. Original line number Diff line number Diff line Loading @@ -22,28 +22,28 @@ ${SIGHTSEEING_URI} https://uri.etsi.org/ngsi-ld/default-context/sightseeing # first entity: vocabProperty = "monument" (string) # second entity: vocabProperty = ["monument", "sightseeing"] (array) 019_02_09_01 QueryWithExpandValuesEqualMonument [Tags] e-query 5_7_2 4_5_20 since_v1.7.1 [Tags] e-query core_10_4_3 core_5_2_6_4_7 since_v1.7.1 vocabProperty=="monument" vocabProperty 2 019_02_09_02 QueryWithExpandValuesEqualSightseeing [Tags] e-query 5_7_2 4_5_20 since_v1.7.1 [Tags] e-query core_10_4_3 core_5_2_6_4_7 since_v1.7.1 vocabProperty=="sightseeing" vocabProperty 1 019_02_09_03 QueryWithExpandValuesUnequalSightseeing [Tags] e-query 5_7_2 4_5_20 since_v1.7.1 019_02_09_03 QueryWithExpandValuesUnequalMonument [Tags] e-query core_10_4_3 core_5_2_6_4_7 since_v1.7.1 vocabProperty!="monument" vocabProperty 0 019_02_09_04 QueryWithUriMonument [Tags] e-query 5_7_2 4_5_20 since_v1.7.1 [Tags] e-query core_10_4_3 core_5_2_6_4_7 since_v1.7.1 vocabProperty=="${MONUMENT_URI}" ${EMPTY} 2 019_02_09_05 QueryWithUriSightseeing [Tags] e-query 5_7_2 4_5_20 since_v1.7.1 [Tags] e-query core_10_4_3 core_5_2_6_4_7 since_v1.7.1 vocabProperty=="${SIGHTSEEING_URI}" ${EMPTY} 1 019_02_09_06 QueryWithExpandValuesValueListEqual [Tags] e-query 5_7_2 4_5_20 since_v1.7.1 [Tags] e-query core_10_4_3 core_5_2_6_4_7 since_v1.7.1 vocabProperty=="monument","sightseeing" vocabProperty 2 019_02_09_07 QueryWithExpandValuesValueListUnequal [Tags] e-query 5_7_2 4_5_20 since_v1.7.1 [Tags] e-query core_10_4_3 core_5_2_6_4_7 since_v1.7.1 vocabProperty!="monument","sightseeing" vocabProperty 0 019_02_09_08 QueryWithUriValueList [Tags] e-query 5_7_2 4_5_20 since_v1.7.1 [Tags] e-query core_10_4_3 core_5_2_6_4_7 since_v1.7.1 vocabProperty=="${MONUMENT_URI}","${SIGHTSEEING_URI}" ${EMPTY} 2 Loading
TP/NGSI-LD/ContextInformation/Consumption/Entity/QueryEntities/019_03_06.robot +7 −7 Changes for TP/NGSI-LD/ContextInformation/Consumption/Entity/QueryEntities/019_03_06.robot: 7 added lines, 7 removed lines. Original line number Diff line number Diff line Loading @@ -11,25 +11,25 @@ Test Template Query entities with invalid q *** Test Cases *** Q 019_03_06_01 QueryWithOperatorMissingValue [Tags] e-query 5_7_2 [Tags] e-query core_10_4_3 attr== 019_03_06_02 QueryWithUnclosedBracket [Tags] e-query 5_7_2 [Tags] e-query core_10_4_3 attr[ 019_03_06_03 QueryWithInvalidSingleExclamationOperator [Tags] e-query 5_7_2 [Tags] e-query core_10_4_3 attr!value 019_03_06_04 QueryStartingWithLogicalOperator [Tags] e-query 5_7_2 [Tags] e-query core_10_4_3 ;attr=="value" 019_03_06_05 QueryWithConsecutiveLogicalOperators [Tags] e-query 5_7_2 [Tags] e-query core_10_4_3 attr=="value";;attr2=="value2" 019_03_06_06 QueryWithInvertedRegexOperator [Tags] e-query 5_7_2 [Tags] e-query core_10_4_3 attr=~"value*" 019_03_06_07 QueryWithUnknownOperator [Tags] e-query 5_7_2 [Tags] e-query core_10_4_3 attr="value" Loading