MEC Robot Test Suite issueshttps://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/-/issues2020-06-09T07:10:30Zhttps://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/-/issues/22Delete trailing comma from json files2020-06-09T07:10:30ZFederico SismondiDelete trailing comma from json filesFound two errors in .json files (tried to push MR but apparently I dont have access to project)
See:
[trailing comma err 1](https://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/blob/2.1.1-fix-plu/SRV/RNIS/jsons/CellChangeSubscriptio...Found two errors in .json files (tried to push MR but apparently I dont have access to project)
See:
[trailing comma err 1](https://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/blob/2.1.1-fix-plu/SRV/RNIS/jsons/CellChangeSubscriptionRequest.json#L9)
[trailing comma err 2](https://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/blob/2.1.1-fix-plu/SRV/RNIS/jsons/UpdateCellChangeSubscriptionRequest.json#L9)https://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/-/issues/21Schema file name is wrong2020-06-08T13:09:56ZjacobvarghesSchema file name is wrong```
TC_MEC_SRV_SRVSUB_002_OK :: Check that the IUT acknowledges the su...
{
...
...
}
| FAIL |
FileNotFoundError: [Errno 2] No such file or directory: 'schemas//SerAvailabilityNotificationSubscription.schema.json'
```
Script is accessi...```
TC_MEC_SRV_SRVSUB_002_OK :: Check that the IUT acknowledges the su...
{
...
...
}
| FAIL |
FileNotFoundError: [Errno 2] No such file or directory: 'schemas//SerAvailabilityNotificationSubscription.schema.json'
```
Script is accessing schema file named `SerAvailabilityNotificationSubscription.schema.json`, but in schema folder, file doesn't have the `schema` key in it. Please fix this by changing the file name to `SerAvailabilityNotificationSubscription.schema.json`.
![image](/uploads/5797f76a0a3605e5d4eb670f23f6c94b/image.png)https://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/-/issues/20TC_MEC_SRV_APPSAQ_002_OK response check is wrong2020-06-08T13:10:43ZjacobvarghesTC_MEC_SRV_APPSAQ_002_OK response check is wrong```
TC_MEC_SRV_APPSAQ_002_OK
[Documentation]
... Check that the IUT notifies the authorised relevant (subscribed) application
... instances when a new service for a given application instance is registered
...
....```
TC_MEC_SRV_APPSAQ_002_OK
[Documentation]
... Check that the IUT notifies the authorised relevant (subscribed) application
... instances when a new service for a given application instance is registered
...
... Reference ETSI GS MEC 011 V2.0.9, clause 8.2.6.3.4
... OpenAPI https://forge.etsi.org/rep/mec/gs011-app-enablement-api/blob/master/MecServiceMgmtApi.yaml#/definitions/ServiceInfo
[Tags] PIC_MEC_PLAT PIC_SERVICES
Create new service ServiceInfo ${APP_INSTANCE_ID}
Check HTTP Response Status Code Is 201
Check HTTP Response Body Json Schema Is ServiceInfo
Check HTTP Response Header Contains Location
>> Check Result Contains ${response['body']} serName ${NEW_SERVICE_NAME}
Check Result Contains
[Arguments] ${source} ${parameter} ${value}
Should Be Present In Json List ${source} ${parameter} ${value}
Should Be Present In Json List
[Arguments] ${expr} ${json_field} ${json_value}
Log Check if ${json_field} is present in ${expr} with the value ${jsonvalue}
:FOR ${item} IN @{expr}
\ Exit For Loop If "${item['${json_field}']}" == "${json_value}"
Log Item found ${item}
[return] ${item}
```
`Check Result Contains` expects the response of the service create as a list of json, which is wrong. This is causing the below failure.
```
| FAIL |
Resolving variable '${item['serName']}' failed: TypeError: string indices must be integers
------------------------------------------------------------------------------
PlatAppServices :: A test suite for validating Application Service... | FAIL |
1 critical test, 0 passed, 1 failed
1 test total, 0 passed, 1 failed
==============================================================================
```
**This same issue is present in most of the test suite, some of them which we used are listed below. Kindly fix all.**
1. TC_MEC_SRV_APPSAQ_002_OK
2. TC_MEC_SRV_APPSAQ_003_OK
3. TC_MEC_SRV_APPSAQ_004_OK
4. TC_MEC_SRV_APPSUB_002_OK
5. TC_MEC_SRV_APPSUB_003_OK
6. TC_MEC_SRV_DNS_002_OK
7. TC_MEC_SRV_SAQ_002_OK
8. TC_MEC_SRV_SRVSUB_002_OK
9. TC_MEC_SRV_SRVSUB_003_OK
10. TC_MEC_SRV_TRAF_002_OK
11. TC_MEC_SRV_TRAF_003_OKhttps://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/-/issues/19SRV/TRAF/PlatTrafficRules.robot TC_MEC_SRV_TRAF_003_OK has multiple issues2020-06-08T13:10:56ZjacobvarghesSRV/TRAF/PlatTrafficRules.robot TC_MEC_SRV_TRAF_003_OK has multiple issues![image](/uploads/df5016b97eab70e0dad402fd88377475/image.png)
![image](/uploads/ebef0de7a26c87f3b7325bcae2ab6d59/image.png)
1. At Line 66 the script calls `Update a traffic rule`, and on line 135 inside `Update a traffic rule` function, ...![image](/uploads/df5016b97eab70e0dad402fd88377475/image.png)
![image](/uploads/ebef0de7a26c87f3b7325bcae2ab6d59/image.png)
1. At Line 66 the script calls `Update a traffic rule`, and on line 135 inside `Update a traffic rule` function, it sends a `POST` message insted of `PUT`, which is wrong.
2. Line number 69 and 70 expects a field names `TrafficRule` to be in the response message, which is not correct. The response of update request will have a single traffic rule entry.
3. Line number 69 and 70 again, `Check Result Contains` function expects the `${source}` input as a list, which is wrong in this case as the update response is not a list.https://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/-/issues/18SRV/TRAF/PlatTrafficRules.robot TC_MEC_SRV_TRAF_002_OK has multiple issues2020-06-08T13:11:02ZjacobvarghesSRV/TRAF/PlatTrafficRules.robot TC_MEC_SRV_TRAF_002_OK has multiple issuesBelow test case has multiple issues to fix.
1. There won't be any `TrafficRule` field in get response body, as expected in the `Check Result Contains` line. We couldn't find it in any of the MEC documents as well about this.
2. `Check R...Below test case has multiple issues to fix.
1. There won't be any `TrafficRule` field in get response body, as expected in the `Check Result Contains` line. We couldn't find it in any of the MEC documents as well about this.
2. `Check Result Contains` line expects the body as a list of traffic rules, which means the request to get the traffic must be `GET ALL`. Current script just query single rule and expects a list of rules, which is wrong.
```
TC_MEC_SRV_TRAF_002_OK
[Documentation]
... Check that the IUT responds with the information on a specific traffic rule
... when queried by a MEC Application
...
... Reference ETSI GS MEC 011 V2.0.9, clause 7.2.8.3.1
... OpenAPI https://forge.etsi.org/rep/mec/gs011-app-enablement-api/blob/v2.0.8/Mp1.yaml#/definitions/TrafficRule
[Tags] PIC_MEC_PLAT PIC_SERVICES
Get individual traffic rule ${APP_INSTANCE_ID} ${TRAFFIC_RULE_ID}
Check HTTP Response Status Code Is 200
Check HTTP Response Body Json Schema Is TrafficRule
Check Result Contains ${response['body']['TrafficRule']} trafficRuleId ${TRAFFIC_RULE_ID}
```
I guess the script can be fixed like below.
```
TC_MEC_SRV_TRAF_002_OK
[Documentation]
... Check that the IUT responds with the information on a specific traffic rule
... when queried by a MEC Application
...
... Reference ETSI GS MEC 011 V2.0.9, clause 7.2.8.3.1
... OpenAPI https://forge.etsi.org/rep/mec/gs011-app-enablement-api/blob/v2.0.8/Mp1.yaml#/definitions/TrafficRule
[Tags] PIC_MEC_PLAT PIC_SERVICES
Get list of traffic rules ${APP_INSTANCE_ID}
Check HTTP Response Status Code Is 200
Check HTTP Response Body Json Schema Is TrafficRuleList
Check Result Contains ${response['body']} trafficRuleId ${TRAFFIC_RULE_ID}
```https://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/-/issues/16SRV/APPSUB/PlatAppSubscriptions.robot TC_MEC_SRV_APPSUB_001_OK SubscriptionL...2020-06-08T13:11:22ZlimingjiSRV/APPSUB/PlatAppSubscriptions.robot TC_MEC_SRV_APPSUB_001_OK SubscriptionLinkList name not rightSubscriptionLinkList -> SubscriptionsLinkListSubscriptionLinkList -> SubscriptionsLinkListhttps://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/-/issues/14SRV/SRVSUB/PlatSrvSubscriptions.robot TC_MEC_SRV_SRVSUB_003_OK Check Result ...2020-06-08T13:11:28ZlimingjiSRV/SRVSUB/PlatSrvSubscriptions.robot TC_MEC_SRV_SRVSUB_003_OK Check Result Contains keyword wrongCheck Result Contains ${response['body']['SerAvailabilityNotificationSubscription']} subscriptionType "SerAvailabilityNotificationSubscription"
should be:
${response['body']}Check Result Contains ${response['body']['SerAvailabilityNotificationSubscription']} subscriptionType "SerAvailabilityNotificationSubscription"
should be:
${response['body']}https://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/-/issues/12SRV/SAQ/PlatServices.robot TC_MEC_SRV_SAQ_002_OK get result keyword wrong2020-06-08T13:11:33ZlimingjiSRV/SAQ/PlatServices.robot TC_MEC_SRV_SAQ_002_OK get result keyword wrongCheck Result Contains ${response['body']['ServiceInfoList']} serInstanceId ${SERVICE_ID}
it should be ${response['body']}, no 'ServiceInfoList' attribute in the resultCheck Result Contains ${response['body']['ServiceInfoList']} serInstanceId ${SERVICE_ID}
it should be ${response['body']}, no 'ServiceInfoList' attribute in the resulthttps://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/-/issues/11SRV/APPSAQ/PlatAppServices.robot should be PUT method @line 2142020-06-08T13:11:39ZlimingjiSRV/APPSAQ/PlatAppServices.robot should be PUT method @line 214Post ${apiRoot}/${apiName}/${apiVersion}/applications/${appInstanceId}/services/${serviceId} ${body}
should be PUT method according to:
mec v2.1.1 api
https://forge.etsi.org/swagger/editor/?url=https://forge.etsi.org/gitlab/mec/gs0...Post ${apiRoot}/${apiName}/${apiVersion}/applications/${appInstanceId}/services/${serviceId} ${body}
should be PUT method according to:
mec v2.1.1 api
https://forge.etsi.org/swagger/editor/?url=https://forge.etsi.org/gitlab/mec/gs011-app-enablement-api/raw/master/MecServiceMgmtApi.yamlhttps://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/-/issues/10SRV/APPSAQ/PlatAppServices.robot check result is not right2020-06-08T13:11:45ZlimingjiSRV/APPSAQ/PlatAppServices.robot check result is not rightCheck Result Contains ${response['body']['ServiceInfo']} serInstanceId ${SERVICE_ID}
'ServiceInfo' is not included in the results according to v2.1.1 mec api
it should be ${response['body']}Check Result Contains ${response['body']['ServiceInfo']} serInstanceId ${SERVICE_ID}
'ServiceInfo' is not included in the results according to v2.1.1 mec api
it should be ${response['body']}https://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/-/issues/9Not sending update request on DNS etag error case2020-06-08T13:12:06ZjacobvarghesNot sending update request on DNS etag error caseLooks like the below testcase supposed to send an update with a wrong header and check the response. But the `Update a DNS Rule with invalid etag` just set the headers, not sending the PUT request.
```
TC_MEC_SRV_DNS_003_PF
[Documen...Looks like the below testcase supposed to send an update with a wrong header and check the response. But the `Update a DNS Rule with invalid etag` just set the headers, not sending the PUT request.
```
TC_MEC_SRV_DNS_003_PF
[Documentation]
... Check that the IUT responds with an error when
... a request sent by a MEC Application doesn't comply with a required condition
...
... Reference ETSI GS MEC 011 V2.0.9, clause 7.2.10.3.2
... OpenAPI https://forge.etsi.org/rep/mec/gs011-app-enablement-api/blob/v2.0.9/MecAppSupportApi.yaml#/definitions/DnsRule
[Tags] PIC_MEC_PLAT PIC_SERVICES
Update a DNS Rule with invalid etag ${APP_INSTANCE_ID} ${DNS_RULE_ID} DnsRuleUpdate
Check HTTP Response Status Code Is 412
Update a DNS Rule with invalid etag
[Arguments] ${appInstanceId} ${dnsRuleId} ${content}
Set Headers {"If-Match": ${INVALID_ETAG}}
```https://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/-/issues/8Message type wrong in DNS update2020-06-08T13:12:28ZjacobvarghesMessage type wrong in DNS updateBelow test case is to update a DNS entry. But looks like the `Update a DNS Rule` statement is sending the POST mesaage instead of PUT.
```
TC_MEC_SRV_DNS_003_OK
[Documentation]
... Check that the IUT updates a specific DNS rul...Below test case is to update a DNS entry. But looks like the `Update a DNS Rule` statement is sending the POST mesaage instead of PUT.
```
TC_MEC_SRV_DNS_003_OK
[Documentation]
... Check that the IUT updates a specific DNS rule
... when commanded by a MEC Application
...
... Reference ETSI GS MEC 011 V2.0.9, clause 7.2.10.3.2
... OpenAPI https://forge.etsi.org/rep/mec/gs011-app-enablement-api/blob/v2.0.9/MecAppSupportApi.yaml#/definitions/DnsRule
[Tags] PIC_MEC_PLAT PIC_SERVICES
Update a DNS Rule ${APP_INSTANCE_ID} ${DNS_RULE_ID} DnsRuleUpdate
Check HTTP Response Status Code Is 200
Check HTTP Response Body Json Schema Is DnsRule
# Check Result Contains ${response['body']['DnsRule']} dnsRuleId ${DNS_RULE_NAME}
# Check Result Contains ${response['body']['DnsRule']} ipAddress ${SOME_IP_ADDRESS}
Update a DNS Rule
[Arguments] ${appInstanceId} ${dnsRuleId} ${content}
Set Headers {"Accept":"application/json"}
Set Headers {"Content-Type":"application/json"}
Set Headers {"Authorization":"${TOKEN}"}
${file}= Catenate SEPARATOR= jsons/ ${content} .json
${body}= Get File ${file}
Post ${apiRoot}/${apiName}/${apiVersion}/applications/${appInstanceId}/dns_rules/${dnsRuleId} ${body}
${output}= Output response
Set Suite Variable ${response} ${output}
```https://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/-/issues/7Response check on DNS testcase for individual query is not proper2020-06-08T13:12:32ZjacobvarghesResponse check on DNS testcase for individual query is not properTest suit: DNS
```
TC_MEC_SRV_DNS_002_OK
[Documentation]
... Check that the IUT responds with the information on a specific DNS rule
... when queried by a MEC Application
...
... Reference ETSI GS MEC 011 ...Test suit: DNS
```
TC_MEC_SRV_DNS_002_OK
[Documentation]
... Check that the IUT responds with the information on a specific DNS rule
... when queried by a MEC Application
...
... Reference ETSI GS MEC 011 V2.0.9, clause 7.2.10.3.1
... OpenAPI https://forge.etsi.org/rep/mec/gs011-app-enablement-api/blob/v2.0.9/MecAppSupportApi.yaml#/definitions/DnsRule
[Tags] PIC_MEC_PLAT PIC_SERVICES
Get individual DNS rule ${APP_INSTANCE_ID} ${DNS_RULE_ID}
Check HTTP Response Status Code Is 200
Check HTTP Response Body Json Schema Is DnsRule
Check Result Contains ${response['body']['DnsRule']} dnsRuleId ${DNS_RULE_ID}
```
The testcase is retrieving an individual dns record and thus the body has to be an object just like below.
```
{
"seconds": 0.002054,
"status": 200,
"body": {
"dnsRuleId": "63684d55-cd66-43a5-975b-ea503c5edb00",
"domainName": "www.example.com",
"ipAddressType": "IP_V4",
"ipAddress": "192.0.2.0",
"ttl": 30,
"state": "ACTIVE"
},
"headers": {
"Content-Type": "application/json; charset=UTF-8",
"Server": "SERVICECENTER/0.0.1",
"Vary": "Accept-Encoding, Origin",
"X-Response-Status": "200",
"Date": "Sat, 23 May 2020 05:47:40 GMT",
"Content-Length": "158"
}
}
```
Here I could see two issues in below line
`Check Result Contains ${response['body']['DnsRule']} dnsRuleId ${DNS_RULE_ID} `
1. Accessing ['DnsRule'] from body, which is different from the API specification.
2. "Check Result Contains" consider the body as a list and check for value inside it, which is wrong. The body is just an object and containes a single dns entry.
Please check and update me if i'm wrong in understanding the script.https://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/-/issues/6Document installation and execution o the tests2020-10-20T10:30:58ZMichele CarignaniDocument installation and execution o the testsIn the README.md file.In the README.md file.Elian KrajaElian Krajahttps://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/-/issues/5remove html files and set git to ignore them2019-09-27T09:55:41ZMichele Carignaniremove html files and set git to ignore themadd to .gitignore
**/*.html
**/*.xmladd to .gitignore
**/*.html
**/*.xmlElian KrajaElian Krajahttps://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/-/issues/4Using txt or py variables?2019-11-05T18:27:05ZMichele CarignaniUsing txt or py variables?Please add examples and list pros cons.Please add examples and list pros cons.https://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/-/issues/3Add CI script2019-09-17T16:21:47ZMichele CarignaniAdd CI scriptAdd a bash script that validates the syntax of all tests with Docker.Add a bash script that validates the syntax of all tests with Docker.Elian KrajaElian Krajahttps://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/-/issues/2Documentation of Test cases2019-10-31T08:32:39ZMichele CarignaniDocumentation of Test casesDebug why the `...` is not working and replace it in place of `#`.Debug why the `...` is not working and replace it in place of `#`.Elian KrajaElian Krajahttps://forge.etsi.org/rep/mec/gs032p3-robot-test-suite/-/issues/1Review how to check one object in a list of many2019-10-31T08:38:29ZMichele CarignaniReview how to check one object in a list of manyProposed approach:
https://forge.etsi.org/gitlab/mec/gs032p3-robot-test-suite/blob/master/resources/GenericKeywords.robot#L35
Are other approaches possible?Proposed approach:
https://forge.etsi.org/gitlab/mec/gs032p3-robot-test-suite/blob/master/resources/GenericKeywords.robot#L35
Are other approaches possible?Elian KrajaElian Kraja