Skip to content
Commits on Source (9)
Hide whitespace changes
Inline Side-by-side
README.md
View file @ 0134d819
......@@ -134,7 +134,7 @@ test launch command followed by the file name.
## Contribute to the Test Suite
In order to contribute to the ETSI NGSI-LD Test Suite, you recommend the installation of an IDE with the corresponding
In order to contribute to the ETSI NGSI-LD Test Suite, it is recommended to install an IDE with the corresponding
Robot Framework. Our recommendations are:
- PyCharm
......@@ -147,8 +147,98 @@ Robot Framework. Our recommendations are:
- Install [Robot Framework Language Server](https://plugins.jetbrains.com/plugin/16086-robot-framework-language-server)
- Define as variable the path of the working directory. In Settings > Languages & Frameworks > Robot Framework
(Project), insert the following: `{"EXECDIR": "{path}/auth-test-suite"}`.
(Project), insert the following: `{"EXECDIR": "{path}/ngsi-ld-test-suite"}`.
### Develop a new Test Case
In order to develop a new Test Case, some rules and conventions have to be followed.
#### Test Case Template
The following template shows the typical structure of a Test Case. It can be used as a template when creating a new
Test Case.
```
*** Settings ***
Documentation {Describe the behavior that is being checked by this Test Case}
Resource ${EXECDIR}/resources/any/necessary/resource/file
# For both setup and teardown steps, try as possible to use Keyword names already declared in doc/analysis/initial_setup.py
Suite Setup {An optional setup step to create data necessary for the Test Case}
Suite Teardown {An optional teardown step to delete data created in the setup step}
Test Template {If the Test Case uses permutations, the name of the Keyword that is called for each permutation}
*** Variables ***
${my_variable}= my_variable
*** Test Cases *** PARAM_1 PARAM_2
XXX_YY_01 Purpose of the first permutation
[Tags] {resource_request_reference} {section_reference_1} {section_reference_2}
param_1_value param_2_value
XXX_YY_02 Purpose of the second permutation
[Tags] {resource_request_reference} {section_reference_1} {section_reference_2}
param_1_value param_2_value
*** Keywords ***
{Keyword name describing what the Test Case is doing}
[Documentation] {Not sure this documentation brings something?}
[Arguments] ${param_1} ${param_2}
# Call operation that is being tested, passing any needed argument
# Perform checks on the response that has been received from the operation
# Optionally call another endpoint to do extra checks (e.g. check an entity has really been created)
# Add there keywords for setup and teardown steps
# Setup step typically creates data necessary for the Test Case and set some variables that will be used by the Test Case
# using Set Suite Variable or Set Test Variable keywords
# Teardown step must delete any data that has been created in setup step
```
Where :
- The possible values for `{resource_request_reference}` are defined in DGR/CIM-0015v211, section 6.1
- The meaning of the `{section_reference_x}` tags is defined in DGR/CIM-0015v211, section 6.2. A Test Case can reference
more than one section in the specification (for instance, a Test Case whose goal is to check that scopes are correctly
created when an entity is created should reference sections 4.18 NGSI-LD Scopes and 5.6.1 Create Entity of the
RGS/CIM-009v131 document)
#### Generate the documentation
The Conformance Tests include an option to automatically generate the documentation of them. If new tests are developed
but include the Check and Request operations already defined, it is not needed to modify the automatic generation system.
Nevertheless, if there are new operations to check and/or new request operations to develop in the Conformance Tests,
it is needed to describe them so that the automatic documentation generated covers these new operations.
Additionally, there is defined a Unit Test mechanism to identify if new Test Suites are included in the system
(See `doc/tests/test_CheckTests.py`) and if there were some changes in any Test Suite (See the `doc/tests/other test_*.py`).
In these cases, it is needed to provide the corresponding information in the Python code:
- When a new Test Case is created, it has to be declared in the corresponding `doc/tests/test_{group}.py` Python file
(where `{group}` represents the group of operations and how they are defined in the ETSI RGS/CIM-0012v211 document).
It includes the creation of the corresponding expected result inside the `doc/files` folders to compare the generation
of the documentation. Complementary to this, updates have to be done:
- In `doc/analysis/initial_setup.py` if it uses a new keyword to set up the initial state of the Test Case
- In `doc/analysis/requests.py` if it updates an endpoint operation, for instance by adding support for a new request
parameter (in `self.op` with the list and position of parameters, in the method that pretty prints the operation in
the automated generation of the documentation associated with the Test Cases. If the positions array is empty, it
means that all the parameters will be taken into consideration)
- When a new global Keyword is created, it has to be declared in the corresponding Python files:
- In `doc/analysis/checks.py` if it is an assertion type keyword (in `self.checks` to reference the method that pretty
prints the operation and `self.args` to identify the position of the arguments) and the corresponding method to
provide the description of this new operation
- In `doc/analysis/requests.py` if it is an endpoint operation (in `self.op` with the list and position of parameters,
in `self.description` to reference the method that pretty print the operation, and add the method that pretty prints
the operation)
- When a new directory containing Test Cases is created, it has to be declared in `doc/generaterobotdata.py` along with
its acronym
Then a new version of the documentation can be generated by running the `doc/generateDocumentationData.py` Python script.
### Run configurations (PyCharm)
......
TP/NGSI-LD/ContextInformation/Subscription/SubscriptionNotificationBehaviour/046_01.robot
View file @ 0134d819
......@@ -23,7 +23,7 @@ ${notification_server_send_url}= http://${notification_server_host}:${not
*** Test Cases ***
046_01_01 Check that a notification is only sent if status is active
[Documentation] Check that a notification is only sent if and only if the status is active
[Tags] sub-notification 5_8_6 046_01
[Tags] sub-notification 5_8_6
Add Initial Entity
Sleep 1s
......
TP/NGSI-LD/ContextInformation/Subscription/SubscriptionNotificationBehaviour/046_02.robot
View file @ 0134d819
......@@ -22,7 +22,7 @@ ${entity_building_filepath}= building-simple-attributes-sample.jsonld
*** Test Cases ***
046_02_01 Check that a notification is sent on the timeInterval
[Documentation] If a Subscription defines a timeInterval member, a Notification shall be sent periodically, when the time interval (in seconds) specified in such value field is reached, regardless of Attribute changes.
[Tags] sub-notification 5_8_6 046_02
[Tags] sub-notification 5_8_6
${response}= Setup Initial Subscriptions
${notification} ${headers}= Wait for notification timeout=${15}
......
TP/NGSI-LD/ContextInformation/Subscription/SubscriptionNotificationBehaviour/046_03.robot
View file @ 0134d819
......@@ -23,7 +23,7 @@ ${fragment_filename}= airQualityLevel-fragment.jsonld
*** Test Cases ***
046_03_01 Check that a notification is sent with all entities
[Documentation] A notification with all subscribed Entities will be included if query or geoquery are not defined.
[Tags] sub-notification 5_8_6 046_03
[Tags] sub-notification 5_8_6
${response}= Setup Initial Subscriptions
Update Entity Attributes ${entity_id} ${fragment_filename} ${CONTENT_TYPE_LD_JSON}
......
TP/NGSI-LD/ContextInformation/Subscription/SubscriptionNotificationBehaviour/046_04.robot
View file @ 0134d819
......@@ -23,7 +23,7 @@ ${fragment_filename}= airQualityLevel-fragment.jsonld
*** Test Cases ***
046_04_01 Check that a notification is sent with all entities
[Documentation] The notification message shall include all the subscribed Entities that changed and that match (as mandated by clauses 4.9 and 4.10) the query and geoquery conditions
[Tags] sub-notification 5_8_6 046_04
[Tags] sub-notification 5_8_6
${response}= Setup Initial Subscriptions
${response1}= Update Entity Attributes ${entity_id} ${fragment_filename} ${CONTENT_TYPE_LD_JSON}
......
TP/NGSI-LD/ContextInformation/Subscription/SubscriptionNotificationBehaviour/046_05.robot
View file @ 0134d819
......@@ -23,7 +23,7 @@ ${fragment_filename}= airQualityLevel-fragment.jsonld
*** Test Cases ***
046_05_01 Check that a notification is sent with all entities
[Documentation] The notification message shall include all the subscribed Entities that changed and that match (as mandated by clauses 4.9 and 4.10) the query and geoquery conditions
[Tags] sub-notification 5_8_6 046_05
[Tags] sub-notification 5_8_6
Setup Initial Subscriptions
${response}= Update Entity Attributes ${entity_id} ${fragment_filename} ${CONTENT_TYPE_LD_JSON}
......
TP/NGSI-LD/ContextInformation/Subscription/SubscriptionNotificationBehaviour/046_06.robot
View file @ 0134d819
......@@ -22,7 +22,7 @@ ${notification_server_send_url}= http://${notification_server_host}:${not
*** Test Cases ***
046_06_01 Check that a notification is sent with all matching entities
[Documentation] only the subscribed Entities matching the query and watched attributes shall be included.
[Tags] sub-notification 5_8_6 046_06
[Tags] sub-notification 5_8_6
${entity_id}= Generate Random Entity Id ${building_id_prefix}
Set Suite Variable ${entity_id}
......
TP/NGSI-LD/ContextInformation/Subscription/SubscriptionNotificationBehaviour/046_07.robot
View file @ 0134d819
......@@ -28,7 +28,7 @@ ${date_format_with_millis} %Y-%m-%dT%H:%M:%S.%f
*** Test Cases ***
046_07_01 Check notification structure
[Documentation] The structure of the notification message shall be as mandated by clause 5.3. Valid notification with attributes as stated above
[Tags] sub-notification 5_8_6 046_07
[Tags] sub-notification 5_8_6
[Setup] Setup Initial Subscriptions ${subscription_payload_file_path}
${response}= Update Entity Attributes ${entity_id} ${fragment_filename} ${CONTENT_TYPE_LD_JSON}
......@@ -41,7 +41,7 @@ ${date_format_with_millis} %Y-%m-%dT%H:%M:%S.%f
046_07_02 Check correct attributes are included
[Documentation] The structure of the notification message shall be as mandated by clause 5.3. The Entity Attributes included (Properties or Relationships) shall be those specified by the notification.attributes member in the Subscription data type (clause 5.2.12).
[Tags] sub-notification 5_8_6 046_07
[Tags] sub-notification 5_8_6
[Setup] Setup Initial Subscriptions ${subscription_payload_file_path_notificationAttributes}
${response}= Update Entity Attributes ${entity_id} ${fragment_filename} ${CONTENT_TYPE_LD_JSON}
......@@ -56,7 +56,7 @@ ${date_format_with_millis} %Y-%m-%dT%H:%M:%S.%f
046_07_03 Check URI expansion is observed
[Documentation] The structure of the notification message shall be as mandated by clause 5.3. URI expansion shall be observed (clause 5.5.7).
[Tags] sub-notification 5_8_6 046_07
[Tags] sub-notification 5_8_6
[Setup] Setup Initial Subscriptions ${subscription_payload_file_path_default_context}
${response}= Update Entity Attributes ${entity_id} ${fragment_filename} ${CONTENT_TYPE_LD_JSON}
......
TP/NGSI-LD/ContextInformation/Subscription/SubscriptionNotificationBehaviour/046_08.robot
View file @ 0134d819
......@@ -25,7 +25,7 @@ ${fragment_filename}= airQualityLevel-fragment.jsonld
*** Test Cases ***
046_08_01 Check that a notification is sent with all attributes
[Documentation] The structure of the notification message shall be as mandated by clause 5.3.1. The absence of the notification.attributes member of a Subscription means that all Entity Attributes shall be included. All attributes are included
[Tags] sub-notification 5_8_6 046_08
[Tags] sub-notification 5_8_6
[Setup] Setup Initial Subscriptions ${False}
${response}= Update Entity Attributes ${entity_id} ${fragment_filename} ${CONTENT_TYPE_LD_JSON}
......@@ -43,7 +43,7 @@ ${fragment_filename}= airQualityLevel-fragment.jsonld
046_08_02 Check that a notification is sent with all attributes in simplified format
[Documentation] The structure of the notification message shall be as mandated by clause 5.3.1. The absence of the notification.attributes member of a Subscription means that all Entity Attributes shall be included If the notification.format member value is "keyValues" then a simplified representation of the entities (as mandated by clause 4.5.3) shall be provided
[Tags] sub-notification 5_8_6 046_08
[Tags] sub-notification 5_8_6
[Setup] Setup Initial Subscriptions ${True}
${response}= Update Entity Attributes ${entity_id} ${fragment_filename} ${CONTENT_TYPE_LD_JSON}
......
TP/NGSI-LD/ContextInformation/Subscription/SubscriptionNotificationBehaviour/046_09.robot
View file @ 0134d819
......@@ -23,7 +23,7 @@ ${notification_server_send_url} http://${notification_server_host}:${not
*** Test Cases ***
046_09_01 Check that a notification is sent to the endpoint
[Documentation] A Notification shall be sent (as mandated by each concrete binding and including any optional endpoint.receiverInfo defined by clause 5.2.22) to the endpoint specified by the endpoint.uri member of the notification structure defined by clause 5.2.1
[Tags] sub-notification 5_8_6 046_09
[Tags] sub-notification 5_8_6
Setup Initial Subscriptions
......
TP/NGSI-LD/ContextInformation/Subscription/SubscriptionNotificationBehaviour/046_10.robot
View file @ 0134d819
......@@ -24,7 +24,7 @@ ${expected_header_links} <${ngsild_test_suite_context}>; rel="htt
*** Test Cases ***
046_10_01 Check that the notification is sent as JSON
[Documentation] The Notification shall be sent as JSON
[Tags] sub-notification 5_8_6 046_10
[Tags] sub-notification 5_8_6
Setup Initial Subscriptions
......
TP/NGSI-LD/ContextInformation/Subscription/SubscriptionNotificationBehaviour/046_11.robot
View file @ 0134d819
......@@ -25,7 +25,7 @@ ${date_format_with_millis}= %Y-%m-%dT%H:%M:%S.%fZ
*** Test Cases ***
046_11_01 Check that timesSent is increased by one
[Documentation] The notification.timesSent member shall be incremented by one.
[Tags] sub-notification 5_8_6 046_11
[Tags] sub-notification 5_8_6
Add Initial Entity
Sleep 1s
......
TP/NGSI-LD/ContextInformation/Subscription/SubscriptionNotificationBehaviour/046_12.robot
View file @ 0134d819
......@@ -23,7 +23,7 @@ ${notification_server_send_url}= http://${notification_server_host}:${not
*** Test Cases ***
046_12_01 Check that lastNotification is updated
[Documentation] The status, lastNotification and lastSuccess members shall be updated with expected value and dates. This test will check these formats.
[Tags] sub-notification 5_8_6 046_12
[Tags] sub-notification 5_8_6
Add Initial Entity
Sleep 1s
......
TP/NGSI-LD/ContextInformation/Subscription/SubscriptionNotificationBehaviour/046_13.robot
View file @ 0134d819
......@@ -23,7 +23,7 @@ ${fragment_filename}= airQualityLevel-fragment.jsonld
*** Test Cases ***
046_13_01 Check that lastFailure and status are updated if a notification could not be sent
[Documentation] If the response to the notification request is different than 200 OK then implementations shall: Update notification.lastFailure with a timestamp representing the current date and time, update notification.status to "failed"
[Tags] sub-notification 5_8_6 046_13
[Tags] sub-notification 5_8_6
@{expected_notification_data_entities}= Create List Building
Update Entity Attributes ${entity_id} ${fragment_filename} ${CONTENT_TYPE_LD_JSON}
......
TP/NGSI-LD/ContextInformation/Subscription/SubscriptionNotificationBehaviour/046_14.robot
View file @ 0134d819
......@@ -23,7 +23,7 @@ ${notification_server_send_url} http://${notification_server_host}:${not
*** Test Cases ***
046_14_01 Check that a notification is sent as JSON-LD
[Documentation] The Notification content shall be JSON-LD when endpoint.accept is set to 'application/ld+json'
[Tags] sub-notification 5_8_6 046_14
[Tags] sub-notification 5_8_6
Add Initial Entity
Sleep 1s
......
data/temporalEntities/expectations/vehicle-temporal-representation-020-04-02-expectation.jsonld
View file @ 0134d819
{
"id":"urn:ngsi-ld:Vehicle:randomUUID",
"type":"Vehicle",
"fuelLevel":[{
"fuelLevel":{
"type":"Property",
"value":67,
"observedAt":"2020-08-01T12:03:00Z"
}],
"speed":[
{
},
"speed":{
"type":"Property",
"value":120,
"observedAt":"2020-08-01T12:03:00Z"
}
]
}
}
\ No newline at end of file
data/temporalEntities/expectations/vehicle-temporal-representation-020-04-03-expectation.jsonld
View file @ 0134d819
{
"id":"urn:ngsi-ld:Vehicle:randomUUID",
"type":"Vehicle",
"fuelLevel":[{
"fuelLevel":{
"type":"Property",
"value":67,
"observedAt":"2020-08-01T12:03:00Z"
}],
},
"speed":[
{
"type":"Property",
......
data/temporalEntities/expectations/vehicle-temporal-representation-delete-fuelLevel-expectation.jsonld
View file @ 0134d819
......@@ -13,14 +13,12 @@
"observedAt":"2020-09-01T12:05:00Z"
}
],
"fuelLevel":[
{
"type":"Property",
"value":40,
"observedAt":"2020-09-01T14:07:00Z",
"datasetId":"urn:ngsi-ld:Vehicle:12345-fuel"
}
],
"fuelLevel":{
"type":"Property",
"value":40,
"observedAt":"2020-09-01T14:07:00Z",
"datasetId":"urn:ngsi-ld:Vehicle:12345-fuel"
},
"@context":[
"https://forge.etsi.org/rep/cim/ngsi-ld-test-suite/-/raw/develop/resources/jsonld-contexts/ngsi-ld-test-suite-compound.jsonld"
]
......
data/temporalEntities/expectations/vehicle-temporal-representation-delete-speed-instanceid-sample.jsonld
View file @ 0134d819
{
"id":"urn:ngsi-ld:Vehicle:randomUUID",
"type":"Vehicle",
"speed":[
{
"type":"Property",
"value":80,
"observedAt":"2020-09-01T12:05:00Z"
}
],
"speed":{
"type":"Property",
"value":80,
"observedAt":"2020-09-01T12:05:00Z"
},
"fuelLevel":[
{
"type":"Property",
......
data/temporalEntities/fragments/vehicle-temporal-modify-attribute-instance-fragment.jsonld
View file @ 0134d819
[
{
"type":"Property",
"value":129,
"observedAt":"2020-09-01T12:03:00Z"
}
]
\ No newline at end of file
{
"type":"Property",
"value":129,
"observedAt":"2020-09-01T12:03:00Z"
}