api-tests issueshttps://forge.etsi.org/rep/nfv/api-tests/-/issues2021-01-14T15:30:16Zhttps://forge.etsi.org/rep/nfv/api-tests/-/issues/105Notification API / NotificationEndpoint test purposes to be re-designed?2021-01-14T15:30:16ZMichele CarignaniNotification API / NotificationEndpoint test purposes to be re-designed?## Background
APIs marked `*Notification-API`specify the behaviour of the endpoints that receive notifications for asynchronous events.
When testing such APIs, the role of the FUT is the to implement the server which is able to receive...## Background
APIs marked `*Notification-API`specify the behaviour of the endpoints that receive notifications for asynchronous events.
When testing such APIs, the role of the FUT is the to implement the server which is able to receive the notification. The role of the Test System should be the one of the producer of notifications, i.e. the client in the notification exchanges.
## Problem
The `Notification-API`s Notification Endpoint tests currently implement the server side of the notification exchange (by means of the MockServer).
In more details:
* according to the SOL specs, the API Consumer implements (e.g. exposes )the NotificationEndpoint ("The API producer can use this resource to send notifications related to VNF alarms or about a rebuilt alarm list to a subscribed API consumer, which has provided the URI of this resource during the subscription process. ", SOL 003, 2.6.1, 7.4.6.1)
* in TST 010, we test the server of the resource
* Therefore the function to be tested in the Notification Endpoints is the API Consumer of a certain API (in the case of VNF Fault Management API this is the NFVO)
* In the tests keywords, you will see that the Test systems expects to receive the notifications and it checks if the notification is correct.
-> I am not saying that this is actually wrong, but it is not in line with the general approach to the test.
=> At least the Test objectives should clarify which IUT is expected to be tested.
### Affected test suites (i.e. Robot files)
* [ ] https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-dev/SOL002/VNFPerformanceManagementNotification-API/PerformanceManagementNotification.robot
* [ ] https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-dev/SOL002/VNFIndicatorNotification-API/VnfIndicatorNotification.robot
* [ ] https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-dev/SOL003/VNFFaultManagementNotification-API/NotificationEndpoint.robot
* [ ] https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-dev/SOL003/VirtualisedResourcesQuotaAvailableNotification-API/NotificationEndpoint.robot
## Proposal
### Option 1
To be fixed in 2.7.1 by changing test purposes and test steps to just executed the following:
* Sending the notification
* verifying the response code is correct
### Option 2
* Change the test purpose to express that we are testing the API Producer capability of sending notifications
2.7.1Giacomo BerniniGiacomo Berninihttps://forge.etsi.org/rep/nfv/api-tests/-/issues/110Document ZIP files required to execute the tests and not contained in the rep...2021-02-26T10:43:50ZMichele CarignaniDocument ZIP files required to execute the tests and not contained in the repository## Zip files
It seems that zip files are required to run the following robot files
```
./SOL003/VNFPackageManagement-API/VNFDInIndividualVNFPackage.robot
./SOL003/VNFPackageManagement-API/VNFPackageContent.robot
./SOL005/NSDManagement-...## Zip files
It seems that zip files are required to run the following robot files
```
./SOL003/VNFPackageManagement-API/VNFDInIndividualVNFPackage.robot
./SOL003/VNFPackageManagement-API/VNFPackageContent.robot
./SOL005/NSDManagement-API/NSDContent.robot
./SOL005/NSDManagement-API/NSDManagementKeywords.robot
./SOL005/VNFPackageManagement-API/VNFDInIndividualVNFPackage.robot
./SOL005/VNFPackageManagement-API/VNFPackageContent.robot
```
and they need to be configured in the following files.
```
./SOL002/VNFLifecycleManagement-API/environment/variables.txt
./SOL003/VNFLifecycleManagement-API/environment/configuration.txt
./SOL003/VNFLifecycleOperationGranting-API/environment/variables.txt
./SOL003/VNFPackageManagement-API/environment/vnfdInIndividualVnfPackage.txt
./SOL003/VNFPackageManagement-API/environment/vnfPackageContent.txt
./SOL005/NSDManagement-API/environment/nsDescriptors.txt
./SOL005/NSDManagement-API/environment/pnfDescriptors.txt
./SOL005/NSLifecycleManagement-API/environment/variables.txt
./SOL005/VNFPackageManagement-API/environment/vnfdInIndividualVnfPackage.txt
./SOL005/VNFPackageManagement-API/environment/vnfPackageContent.txt
```
## Tasks
* [ ] Review the variables files and remove ZIP variables when not needed
* [ ] Document which ZIP files need to be created and where2.7.1Giacomo BerniniGiacomo Berninihttps://forge.etsi.org/rep/nfv/api-tests/-/issues/189[VNFPackageManagement-API.IndividualVNFPackage] The 'descriptor_id' is fetche...2021-09-28T10:41:41Zbanerjeesu[VNFPackageManagement-API.IndividualVNFPackage] The 'descriptor_id' is fetched as None, causing validation failureFile: SOL005/VNFPackageManagement-API/VNFPackageManagementKeywords.robot
Line: around 200
---
```
Fetch Information from SOL001 descriptor file
Import Variables ${CURDIR}/descriptors/SOL001/VNFD/vnfd_SOL001.yaml
${descriptor_...File: SOL005/VNFPackageManagement-API/VNFPackageManagementKeywords.robot
Line: around 200
---
```
Fetch Information from SOL001 descriptor file
Import Variables ${CURDIR}/descriptors/SOL001/VNFD/vnfd_SOL001.yaml
${descriptor_id}= Get Variable Value ${topology_template.node_templates.properties.descriptor_id}
${provider}= Get Variable Value ${topology_template.node_templates.properties.provider}
${product_name}= Get Variable Value ${topology_template.node_templates.properties.product_name}
${software_version}= Get Variable Value ${topology_template.node_templates.properties.software_version}
${descriptor_version}= Get Variable Value ${topology_template.node_templates.properties.descriptor_version}
```
---
${topology_template.node_templates.properties.descriptor_id} evaluates to None, as the file SOL005/VNFPackageManagement-API/descriptors/SOL001/VNFD/vnfd_SOL001.yaml does not have 'properties' under 'topology_template.node_templates'. And this is pretty much true for all the variables listed above, provider, product_name, ... so on.
Are the descriptor files (e.g. SOL001/VNFD/vnfd_SOL001.yaml) provided in the plugtest source for reference purposes only?2.7.1