NFV - Network Functions Virtualisation issueshttps://forge.etsi.org/rep/groups/nfv/-/issues2020-06-12T13:12:52Zhttps://forge.etsi.org/rep/nfv/api-tests/-/issues/1022.6.1 Check LINK in Header Bug2020-06-12T13:12:52Zraducccradu@luxoft.com2.6.1 Check LINK in Header Bughttps://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL005/NSDManagement-API/NSDManagementKeywords.robot
Check LINK in Header
${linkURL}= Get Value From Json ${response.headers} $..Link
Should Not Be Empty ${...https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL005/NSDManagement-API/NSDManagementKeywords.robot
Check LINK in Header
${linkURL}= Get Value From Json ${response.headers} $..Link
Should Not Be Empty ${linkURL}
${response['headers']} should be the correct form.https://forge.etsi.org/rep/nfv/api-tests/-/issues/103Typos in NsdInfo schemas SOL005 NSDManagement 2.6.12020-06-12T13:13:02Zraducccradu@luxoft.comTypos in NsdInfo schemas SOL005 NSDManagement 2.6.1"statussss" typo in
https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL005/NSDManagement-API/schemas/NsdInfo.schema.json
and
https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL005/NSDManagement-API/schemas/NsdI..."statussss" typo in
https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL005/NSDManagement-API/schemas/NsdInfo.schema.json
and
https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL005/NSDManagement-API/schemas/NsdInfos.schema.jsonhttps://forge.etsi.org/rep/nfv/api-tests/-/issues/104Missing keywords2020-06-05T10:42:56ZMichele CarignaniMissing keywordsBy running `robot --dryrun` on the files in the projects a set of issues has been identified.
Attached the stderr and stdout of the execution of the script below.
```bash
str=Keywords.robot;
for i in */*/*.robot ; do
[[ "$i" != *"$str...By running `robot --dryrun` on the files in the projects a set of issues has been identified.
Attached the stderr and stdout of the execution of the script below.
```bash
str=Keywords.robot;
for i in */*/*.robot ; do
[[ "$i" != *"$str"* ]] && \
(echo "++++ Dryrun $i" && \
robot --dryrun --output NONE --report NONE --log NONE $i || \
echo "++++ Issues in file $i");
done
```
[VALIDATION_ERRORS.txt](/uploads/907ed31167a98081c254cb8a0929b763/VALIDATION_ERRORS.txt)
[VALIDATION_LOG.txt](/uploads/66a857ba8a81d64955edbab0d43547d1/VALIDATION_LOG.txt)https://forge.etsi.org/rep/nfv/api-tests/-/issues/105Notification API / NotificationEndpoint test purposes to be re-designed?2021-01-14T15:30:16ZMichele CarignaniNotification API / NotificationEndpoint test purposes to be re-designed?## Background
APIs marked `*Notification-API`specify the behaviour of the endpoints that receive notifications for asynchronous events.
When testing such APIs, the role of the FUT is the to implement the server which is able to receive...## Background
APIs marked `*Notification-API`specify the behaviour of the endpoints that receive notifications for asynchronous events.
When testing such APIs, the role of the FUT is the to implement the server which is able to receive the notification. The role of the Test System should be the one of the producer of notifications, i.e. the client in the notification exchanges.
## Problem
The `Notification-API`s Notification Endpoint tests currently implement the server side of the notification exchange (by means of the MockServer).
In more details:
* according to the SOL specs, the API Consumer implements (e.g. exposes )the NotificationEndpoint ("The API producer can use this resource to send notifications related to VNF alarms or about a rebuilt alarm list to a subscribed API consumer, which has provided the URI of this resource during the subscription process. ", SOL 003, 2.6.1, 7.4.6.1)
* in TST 010, we test the server of the resource
* Therefore the function to be tested in the Notification Endpoints is the API Consumer of a certain API (in the case of VNF Fault Management API this is the NFVO)
* In the tests keywords, you will see that the Test systems expects to receive the notifications and it checks if the notification is correct.
-> I am not saying that this is actually wrong, but it is not in line with the general approach to the test.
=> At least the Test objectives should clarify which IUT is expected to be tested.
### Affected test suites (i.e. Robot files)
* [ ] https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-dev/SOL002/VNFPerformanceManagementNotification-API/PerformanceManagementNotification.robot
* [ ] https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-dev/SOL002/VNFIndicatorNotification-API/VnfIndicatorNotification.robot
* [ ] https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-dev/SOL003/VNFFaultManagementNotification-API/NotificationEndpoint.robot
* [ ] https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-dev/SOL003/VirtualisedResourcesQuotaAvailableNotification-API/NotificationEndpoint.robot
## Proposal
### Option 1
To be fixed in 2.7.1 by changing test purposes and test steps to just executed the following:
* Sending the notification
* verifying the response code is correct
### Option 2
* Change the test purpose to express that we are testing the API Producer capability of sending notifications
2.7.1Giacomo BerniniGiacomo Berninihttps://forge.etsi.org/rep/nfv/api-tests/-/issues/106Parametrize the json bodies with configuration variables2021-09-22T08:36:39ZMichele CarignaniParametrize the json bodies with configuration variablesThe JSON payloads in the `json` folders currently need to be manually edited to be configured to suit the FUT.
They could be parametrized or "templated" so that at run time they are filled with the information from the `environment/vari...The JSON payloads in the `json` folders currently need to be manually edited to be configured to suit the FUT.
They could be parametrized or "templated" so that at run time they are filled with the information from the `environment/variables.txt` file.
This is particularly relevant when the JSON payloads should include values that are computed at runtime from the test system, e.g. the callbackURL for notification tests: in automated tests system, the actual endpoint may not be known before test execution time, and the JSON payload need to be edited accordingly by the test system.
## Proposed solutions
* Use a complete templating system
* Use Python string formatting capabilitiesFrank BrydenFrank Brydenhttps://forge.etsi.org/rep/nfv/SOL006/-/issues/20Back out change related to #112020-06-10T05:40:01ZjethanandaniBack out change related to #11Issue #11 committed a change that needs to be backed out of v3.3.1. It will probably go as part of v3.4.1.Issue #11 committed a change that needs to be backed out of v3.3.1. It will probably go as part of v3.4.1.v3.3.1jethanandanijethanandanihttps://forge.etsi.org/rep/nfv/api-tests/-/issues/107IndividualNSDescriptor test suite "Etag" instead of "ETag"2020-06-12T13:13:19Zraducccradu@luxoft.comIndividualNSDescriptor test suite "Etag" instead of "ETag"IndividualNSDescriptor . GET Individual Network Service Descriptor Information
does not contain 'Etag'
https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL005/NSDManagement-API/IndividualNSDescriptor.robotIndividualNSDescriptor . GET Individual Network Service Descriptor Information
does not contain 'Etag'
https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL005/NSDManagement-API/IndividualNSDescriptor.robothttps://forge.etsi.org/rep/nfv/api-tests/-/issues/108Disable Individual Network Service Descriptor issue 2.6.1-fix-plu2020-06-17T21:53:28Zraducccradu@luxoft.comDisable Individual Network Service Descriptor issue 2.6.1-fix-pluhttps://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-dev/SOL005/NSDManagement-API/IndividualNSDescriptor.robot
```
Disable Individual Network Service Descriptor
[Documentation] Test ID: 5.3.1.2.3
... Test title: Disable ...https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-dev/SOL005/NSDManagement-API/IndividualNSDescriptor.robot
```
Disable Individual Network Service Descriptor
[Documentation] Test ID: 5.3.1.2.3
... Test title: Disable Individual Network Service Descriptor
... Test objective: The objective is to test the disabling of an individual Network Service Descriptor and perform a JSON schema and content validation of the collected data structure
... Pre-conditions: One or more Network Service Descriptors are onboarded in the NFVO in enabled operational state.
... Reference: Clause 5.4.3.3.4 - ETSI GS NFV-SOL 005 [3] v2.6.1
... Config ID: Config_prod_NFVO
... Applicability: none
... Post-Conditions: The Network Service Descriptor is in operational state DISABLED and usage state is not modified
Send PATCH to disable Individual Network Service Descriptor
Check HTTP Response Status Code Is 200
Check HTTP Response Body Json Schema Is NsdInfoModification
Check Postcondition Network Service Descriptor is in operational state DISABLED
Check Postcondition Network Service Descriptor usage state is unmodified (Implicit)
```
Between these 2 lines:
Check Postcondition Network Service Descriptor is in operational state DISABLED
Check Postcondition Network Service Descriptor usage state is unmodified (Implicit)
There should be an intermediary GET Request querying the ns descriptor with that id, otherwise in the primitive method
"Check Postcondition Network Service Descriptor usage state " evaluation is applied on the previous response object witch is NsdInfoModifications type and not NsdInfo type.
Suggested fix:
Check Postcondition Network Service Descriptor is in operational state DISABLED
**GET Individual Network Service Descriptor Information**
Check Postcondition Network Service Descriptor usage state is unmodified (Implicit)https://forge.etsi.org/rep/nfv/api-tests/-/issues/109Seaparator missing 2.6.1 NSLCM2020-06-12T13:13:47Zraducccradu@luxoft.comSeaparator missing 2.6.1 NSLCMhttps://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL005/NSLifecycleManagement-API/NSLCMOperationKeywords.robot
```
Check HTTP Response Body Json Schema Is
[Arguments] ${input}
${schema} = Catenate ${input} ...https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL005/NSLifecycleManagement-API/NSLCMOperationKeywords.robot
```
Check HTTP Response Body Json Schema Is
[Arguments] ${input}
${schema} = Catenate ${input} .schema.json
Validate Json ${schema} ${response[0]['body']}
Log Json Schema Validation OK
```
SEPARATOR argument should be used after Catenate keywordhttps://forge.etsi.org/rep/nfv/api-tests/-/issues/110Document ZIP files required to execute the tests and not contained in the rep...2021-02-26T10:43:50ZMichele CarignaniDocument ZIP files required to execute the tests and not contained in the repository## Zip files
It seems that zip files are required to run the following robot files
```
./SOL003/VNFPackageManagement-API/VNFDInIndividualVNFPackage.robot
./SOL003/VNFPackageManagement-API/VNFPackageContent.robot
./SOL005/NSDManagement-...## Zip files
It seems that zip files are required to run the following robot files
```
./SOL003/VNFPackageManagement-API/VNFDInIndividualVNFPackage.robot
./SOL003/VNFPackageManagement-API/VNFPackageContent.robot
./SOL005/NSDManagement-API/NSDContent.robot
./SOL005/NSDManagement-API/NSDManagementKeywords.robot
./SOL005/VNFPackageManagement-API/VNFDInIndividualVNFPackage.robot
./SOL005/VNFPackageManagement-API/VNFPackageContent.robot
```
and they need to be configured in the following files.
```
./SOL002/VNFLifecycleManagement-API/environment/variables.txt
./SOL003/VNFLifecycleManagement-API/environment/configuration.txt
./SOL003/VNFLifecycleOperationGranting-API/environment/variables.txt
./SOL003/VNFPackageManagement-API/environment/vnfdInIndividualVnfPackage.txt
./SOL003/VNFPackageManagement-API/environment/vnfPackageContent.txt
./SOL005/NSDManagement-API/environment/nsDescriptors.txt
./SOL005/NSDManagement-API/environment/pnfDescriptors.txt
./SOL005/NSLifecycleManagement-API/environment/variables.txt
./SOL005/VNFPackageManagement-API/environment/vnfdInIndividualVnfPackage.txt
./SOL005/VNFPackageManagement-API/environment/vnfPackageContent.txt
```
## Tasks
* [ ] Review the variables files and remove ZIP variables when not needed
* [ ] Document which ZIP files need to be created and where2.7.1Giacomo BerniniGiacomo Berninihttps://forge.etsi.org/rep/nfv/api-tests/-/issues/111Unresolvable JSON pointer in NSLifecycleManagement-API .NSInstances.robot2020-06-12T13:13:58Zraducccradu@luxoft.comUnresolvable JSON pointer in NSLifecycleManagement-API .NSInstances.robothttps://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL005/NSLifecycleManagement-API/NSInstances.robot
GET information about multiple NS instances :: Test ID: 5.3.2.1.2 | FAIL |
RefResolutionError: Unresolvable JSON pointer:...https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL005/NSLifecycleManagement-API/NSInstances.robot
GET information about multiple NS instances :: Test ID: 5.3.2.1.2 | FAIL |
RefResolutionError: Unresolvable JSON pointer: 'definitions/VnfInstance'
GET NSInstances with "exclude_fields" attribute selector :: Test I... | FAIL |
RefResolutionError: Unresolvable JSON pointer: 'definitions/VnfInstance
Schema definitions missing from schemas/NSInstance.schema.json .https://forge.etsi.org/rep/nfv/api-tests/-/issues/112NSInstances.robot 2.6.1 Attribute Error2020-06-12T13:14:20Zraducccradu@luxoft.comNSInstances.robot 2.6.1 Attribute Errorhttps://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL005/NSLifecycleManagement-API/NSInstances.robot
GET information about multiple NS instances as Paged Response :: T... | FAIL |
Resolving variable '${response.headers}' faile...https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL005/NSLifecycleManagement-API/NSInstances.robot
GET information about multiple NS instances as Paged Response :: T... | FAIL |
Resolving variable '${response.headers}' failed: AttributeError: 'dict' object has no attribute 'headers'https://forge.etsi.org/rep/nfv/api-tests/-/issues/113instantiationState instead of nsState in InstantiateNSTask.robot 2.6.12020-06-12T13:14:34Zraducccradu@luxoft.cominstantiationState instead of nsState in InstantiateNSTask.robot 2.6.1Expected property 'response body instantiationState' was not found.
https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL005/NSLifecycleManagement-API/NSLCMOperationKeywords.robot
Response body nsState should be used since in...Expected property 'response body instantiationState' was not found.
https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL005/NSLifecycleManagement-API/NSLCMOperationKeywords.robot
Response body nsState should be used since instantiationState is not a property of NsInstance.
https://www.etsi.org/deliver/etsi_gs/NFV-SOL/001_099/005/02.06.01_60/gs_nfv-sol005v020601p.pdfhttps://forge.etsi.org/rep/nfv/api-tests/-/issues/114Etag instead of ETag 2.6.1 IndividualNSInstance.robot2020-06-12T13:14:49Zraducccradu@luxoft.comEtag instead of ETag 2.6.1 IndividualNSInstance.robotIndividualNSInstance.robot
https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL005/NSLifecycleManagement-API/IndividualNSInstance.robot
```
PUT Individual NSInstance - Method not implemented :: Test ID: 5.3...
Property 'Eta...IndividualNSInstance.robot
https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL005/NSLifecycleManagement-API/IndividualNSInstance.robot
```
PUT Individual NSInstance - Method not implemented :: Test ID: 5.3...
Property 'Etag' does not exist in:
```
```
GET Information about an individual NS Instance :: Test ID: 5.3.2.2.2 | FAIL |
Expected property 'response headers Etag' was not found.
```
```
PATCH Individual NSInstance - Method not implemented :: Test ID: 5... | FAIL |
Expected property 'response headers Etag' was not found.
```https://forge.etsi.org/rep/nfv/api-tests/-/issues/115NSLifecycleManagement-API.NSLCMOccurences.robot small bugs 2.6.12020-06-12T13:14:58Zraducccradu@luxoft.comNSLifecycleManagement-API.NSLCMOccurences.robot small bugs 2.6.1https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL005/NSLifecycleManagement-API/NSLCMOccurences.robot
Attribute of NsLcmOpOcc 'operationParams' has cardinality [0-1], so it shouldn't be a required property.
Proof https://ww...https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL005/NSLifecycleManagement-API/NSLCMOccurences.robot
Attribute of NsLcmOpOcc 'operationParams' has cardinality [0-1], so it shouldn't be a required property.
Proof https://www.etsi.org/deliver/etsi_gs/NFV-SOL/001_099/005/02.06.01_60/gs_nfv-sol005v020601p.pdf
```
GET status information about multiple NS LCM occurrences with "exc... | FAIL |
ValidationError: Validation error for schema NsLcmOpOccs.schema.json: 'operationParams' is a required property
GET status information about multiple NS LCM occurrences with "fie... | FAIL |
ValidationError: Validation error for schema NsLcmOpOccs.schema.json: 'operationParams' is a required property
GET status information about multiple NS LCM occurrences with "exc... | FAIL |
ValidationError: Validation error for schema NsLcmOpOccs.schema.json: 'operationParams' is a required property
GET status information about multiple NS LCM occurrences with "all... | FAIL |
ValidationError: Validation error for schema NsLcmOpOccs.schema.json: 'operationParams' is a required property
```
Schemas that should be modified:
```
schemas/NsLcmOpOcc.schema.json
schemas/NsLcmOpOccs.schema.json
```
```
GET information about multiple NS instances as Paged Response :: T... | FAIL |
Resolving variable '${response.headers}' failed: AttributeError: 'dict' object has no attribute 'headers'
```https://forge.etsi.org/rep/nfv/api-tests/-/issues/116VnfLcmMntOperationKeywords.robot SOL003 2.6.1 Missing separator2020-06-12T13:15:06Zraducccradu@luxoft.comVnfLcmMntOperationKeywords.robot SOL003 2.6.1 Missing separatorhttps://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL003/VNFLifecycleManagement-API/VnfLcmMntOperationKeywords.robot
`${schema} = Catenate ${input} .schema.json`
Should be:
`${schema} = Catenate SEPARATOR= $...https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL003/VNFLifecycleManagement-API/VnfLcmMntOperationKeywords.robot
`${schema} = Catenate ${input} .schema.json`
Should be:
`${schema} = Catenate SEPARATOR= ${input} .schema.json`https://forge.etsi.org/rep/nfv/api-tests/-/issues/117VnfLcmMntOperationKeywords.robot SOL003 2.6.1 AttributeError headers2020-06-12T13:15:14Zraducccradu@luxoft.comVnfLcmMntOperationKeywords.robot SOL003 2.6.1 AttributeError headers```
GET information about multiple VNF instances to get Paged Response... | FAIL |
Resolving variable '${response.headers}' failed: AttributeError: 'dict' object has no attribute 'headers'
```
https://forge.etsi.org/rep/nfv/api-tests/bl...```
GET information about multiple VNF instances to get Paged Response... | FAIL |
Resolving variable '${response.headers}' failed: AttributeError: 'dict' object has no attribute 'headers'
```
https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-fix-plu/SOL003/VNFLifecycleManagement-API/VnfLcmMntOperationKeywords.robothttps://forge.etsi.org/rep/nfv/SOL006/-/issues/21Update version in all the models to 2.8.12020-06-19T18:30:59ZjethanandaniUpdate version in all the models to 2.8.1This commit is to update the YANG models to reflect the latest release version.This commit is to update the YANG models to reflect the latest release version.v2.8.1jethanandanijethanandanihttps://forge.etsi.org/rep/nfv/SOL006/-/issues/22Update version in all the models to 3.3.1.2020-06-25T17:14:41ZjethanandaniUpdate version in all the models to 3.3.1.This commit will take care of updating the version in all the models to 3.3.1.This commit will take care of updating the version in all the models to 3.3.1.v3.3.1jethanandanijethanandanihttps://forge.etsi.org/rep/nfv/api-tests/-/issues/118Response variable not updated after the GET call2020-06-17T21:51:45ZElian KrajaResponse variable not updated after the GET callhttps://forge.etsi.org/rep/nfv/api-tests/blob/2.4.1-fix-plu/SOL003/VNFPackageManagement-API/VNFPackageManagementKeywords.robot
Check Postcondition VNFD Exist
Log Checking that vnf pacakge still exists
Set Headers {"Acce...https://forge.etsi.org/rep/nfv/api-tests/blob/2.4.1-fix-plu/SOL003/VNFPackageManagement-API/VNFPackageManagementKeywords.robot
Check Postcondition VNFD Exist
Log Checking that vnf pacakge still exists
Set Headers {"Accept": "${ACCEPT_PLAIN}"}
Set Headers {"Accept": "${ACCEPT_ZIP}"}
Run Keyword If ${AUTH_USAGE} == 1 Set Headers {"Authorization": "${AUTHORIZATION}"}
GET ${apiRoot}/${apiName}/${apiVersion}/vnf_packages/${vnfPkgZipVNFD}/vnfd
Check HTTP Response Status Code Is 200
Suggest to add the following lines after GET if possible:
${output}= Output response
Set Suite Variable ${response} ${output}Elian KrajaElian Kraja