Notification API / NotificationEndpoint test purposes to be re-designed?
Background
APIs marked *Notification-APIspecify the behaviour of the endpoints that receive notifications for asynchronous events.
When testing such APIs, the role of the FUT is the to implement the server which is able to receive the notification. The role of the Test System should be the one of the producer of notifications, i.e. the client in the notification exchanges.
Problem
The Notification-APIs Notification Endpoint tests currently implement the server side of the notification exchange (by means of the MockServer).
In more details:
-
according to the SOL specs, the API Consumer implements (e.g. exposes )the NotificationEndpoint ("The API producer can use this resource to send notifications related to VNF alarms or about a rebuilt alarm list to a subscribed API consumer, which has provided the URI of this resource during the subscription process. ", SOL 003, 2.6.1, 7.4.6.1)
-
in TST 010, we test the server of the resource
-
Therefore the function to be tested in the Notification Endpoints is the API Consumer of a certain API (in the case of VNF Fault Management API this is the NFVO)
-
In the tests keywords, you will see that the Test systems expects to receive the notifications and it checks if the notification is correct.
-> I am not saying that this is actually wrong, but it is not in line with the general approach to the test.
=> At least the Test objectives should clarify which IUT is expected to be tested.
Affected test suites (i.e. Robot files)
- https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-dev/SOL002/VNFPerformanceManagementNotification-API/PerformanceManagementNotification.robot
- https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-dev/SOL002/VNFIndicatorNotification-API/VnfIndicatorNotification.robot
- https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-dev/SOL003/VNFFaultManagementNotification-API/NotificationEndpoint.robot
- https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-dev/SOL003/VirtualisedResourcesQuotaAvailableNotification-API/NotificationEndpoint.robot
Proposal
Option 1
To be fixed in 2.7.1 by changing test purposes and test steps to just executed the following:
- Sending the notification
- verifying the response code is correct
Option 2
- Change the test purpose to express that we are testing the API Producer capability of sending notifications