NFV - Network Functions Virtualisation issueshttps://forge.etsi.org/rep/groups/nfv/-/issues2020-11-18T18:56:34Zhttps://forge.etsi.org/rep/nfv/api-tests/-/issues/48[GENERIC] Asynchronous vs Synchronous should be alternately executed based on...2020-11-18T18:56:34ZElian Kraja[GENERIC] Asynchronous vs Synchronous should be alternately executed based on availabilityhttps://forge.etsi.org/rep/nfv/api-tests/-/issues/82ETag should not be checked on PATCH Operations2020-11-18T18:58:30ZElian KrajaETag should not be checked on PATCH OperationsIn the PATCH operations, it should be avoided the check of the ETag, since the operation will be asynchronous. In fact, a 202 Accepted is returned and as reported in the spec (Table 5.4.3.3.4-2 of SOL003):
`The request was accepted for ...In the PATCH operations, it should be avoided the check of the ETag, since the operation will be asynchronous. In fact, a 202 Accepted is returned and as reported in the spec (Table 5.4.3.3.4-2 of SOL003):
`The request was accepted for processing, but the processing has not been completed.`
FILE: SOL003/VNFLifecycleManagement-API/IndividualVNFInstance.robothttps://forge.etsi.org/rep/nfv/api-tests/-/issues/52Dependency not met: test case 'Send VNF configuration' not found, wanted 'PASS'2020-11-19T13:48:00ZjonnadaDependency not met: test case 'Send VNF configuration' not found, wanted 'PASS'Interface: VNF/configuration
Test Case: Set new VNF Configuration - HTTP Etag precondition unsuccessfulInterface: VNF/configuration
Test Case: Set new VNF Configuration - HTTP Etag precondition unsuccessfulAHMADABBAHMADABBhttps://forge.etsi.org/rep/nfv/api-tests/-/issues/57Dictionary item '${response[0]['headers']['ETag']}' does not contain '=' sepa...2020-11-19T13:49:45ZjonnadaDictionary item '${response[0]['headers']['ETag']}' does not contain '=' separatorInterface: VNF/Configuration
Test Case: (1) Set new VNF Configuration
(2) Set new VNF Configuration - HTTP Etag precondition unsuccessfulInterface: VNF/Configuration
Test Case: (1) Set new VNF Configuration
(2) Set new VNF Configuration - HTTP Etag precondition unsuccessfulAHMADABBAHMADABBhttps://forge.etsi.org/rep/nfv/api-tests/-/issues/61[GENERIC] Schema jsons (IN SOL005) should not contain referenced elements2020-11-19T13:50:36ZElian Kraja[GENERIC] Schema jsons (IN SOL005) should not contain referenced elementsJson schemas in all SOL005 should not contain $ref elements
current:
```json
"_links": {
"description": "Links to resources related to this notification.\n",
"$ref": "#/definitions/LccnLinks"
}
```
Should be
```json
{
"de...Json schemas in all SOL005 should not contain $ref elements
current:
```json
"_links": {
"description": "Links to resources related to this notification.\n",
"$ref": "#/definitions/LccnLinks"
}
```
Should be
```json
{
"description": "Links to resources related to this resource.\n",
"type": "object",
"required": [
"self",
"nsInstance"
],
"properties": {
"self": {
"description": "This type represents a link to a resource.\n",
"type": "object",
"required": [
"href"
],
"properties": {
"href": {
"description": "URI of the referenced resource.\n",
"type": "string",
"format": "url"
}
}
},
"nsInstance": {
"description": "This type represents a link to a resource.\n",
"type": "object",
"required": [
"href"
],
"properties": {
"href": {
"description": "URI of the referenced resource.\n",
"type": "string",
"format": "url"
}
}
},
"cancel": {
"description": "This type represents a link to a resource.\n",
"type": "object",
"required": [
"href"
],
"properties": {
"href": {
"description": "URI of the referenced resource.\n",
"type": "string",
"format": "url"
}
}
},
"retry": {
"description": "This type represents a link to a resource.\n",
"type": "object",
"required": [
"href"
],
"properties": {
"href": {
"description": "URI of the referenced resource.\n",
"type": "string",
"format": "url"
}
}
},
"rollback": {
"description": "This type represents a link to a resource.\n",
"type": "object",
"required": [
"href"
],
"properties": {
"href": {
"description": "URI of the referenced resource.\n",
"type": "string",
"format": "url"
}
}
},
"continue": {
"description": "This type represents a link to a resource.\n",
"type": "object",
"required": [
"href"
],
"properties": {
"href": {
"description": "URI of the referenced resource.\n",
"type": "string",
"format": "url"
}
}
},
"fail": {
"description": "This type represents a link to a resource.\n",
"type": "object",
"required": [
"href"
],
"properties": {
"href": {
"description": "URI of the referenced resource.\n",
"type": "string",
"format": "url"
}
}
}
}
}
```AHMADABBAHMADABBhttps://forge.etsi.org/rep/nfv/api-tests/-/issues/65Error on checking status code in Grants.robot in SOL003/VNFLifeCycleGrantingo...2020-11-19T13:51:45ZElian KrajaError on checking status code in Grants.robot in SOL003/VNFLifeCycleGrantingoperation-APILine 117
```
Should Be Equal ${response.status_code} ${expected_status}
```
* [x] Status_code doesn't exist on the response
* [x] Check should be done using Shoul Be Equal as StringsLine 117
```
Should Be Equal ${response.status_code} ${expected_status}
```
* [x] Status_code doesn't exist on the response
* [x] Check should be done using Shoul Be Equal as StringsAHMADABBAHMADABBhttps://forge.etsi.org/rep/nfv/api-tests/-/issues/67Schema validation failure in VNFConfiguration-API/DELETE VNF Configuration - ...2020-11-19T13:53:34ZpasrichaSchema validation failure in VNFConfiguration-API/DELETE VNF Configuration - Method not implemented Test CaseThe script “Should Be Equal ${response[0]['body']} ${input}” is trying to compare VnfConfigModification object with VnfConfiguration object which seems to have been wrongly defined in SOL002/VNFConfiguration-API/schemas/vnfConfigura...The script “Should Be Equal ${response[0]['body']} ${input}” is trying to compare VnfConfigModification object with VnfConfiguration object which seems to have been wrongly defined in SOL002/VNFConfiguration-API/schemas/vnfConfiguration.schema.json file.https://forge.etsi.org/rep/nfv/api-tests/-/issues/70Wrong SETUP on Instantiate a vnfInstance Conflict2020-11-19T13:55:17ZElian KrajaWrong SETUP on Instantiate a vnfInstance ConflictThe keyword Check resource instantiated should contain the ID of the instance to be checked.
The POST, on Instantiate a vnfInstance Conflict is done on an ID identified by the variable "instantiatedVnfInstanceId" but in Check resource i...The keyword Check resource instantiated should contain the ID of the instance to be checked.
The POST, on Instantiate a vnfInstance Conflict is done on an ID identified by the variable "instantiatedVnfInstanceId" but in Check resource instantiated keyword the check in done on the variable "vnfInstanceId"
FILE: SOL002/VNFLifecycleManagement-API/InstantiateVNFTask.robothttps://forge.etsi.org/rep/nfv/api-tests/-/issues/84Keyword "Check resource instantiated" in SOL003/VNFLifeCycleManagement-API sh...2020-11-19T14:48:27ZElian KrajaKeyword "Check resource instantiated" in SOL003/VNFLifeCycleManagement-API should be parametrizedThe keyword "Check resource instantiated" checks always the same variable ${vnfInstance} even when the task is trying to use another one
I.e.: Instantiate a vnfInstance Conflict in InstantiateVNFTask.robot is trying to instantiate a vnf...The keyword "Check resource instantiated" checks always the same variable ${vnfInstance} even when the task is trying to use another one
I.e.: Instantiate a vnfInstance Conflict in InstantiateVNFTask.robot is trying to instantiate a vnf with ID ${instantiatedVnfInstanceId}, but in the SETUP part, the "Check resource instantiated" performs a check on instance id ${vnfInstance}https://forge.etsi.org/rep/nfv/api-tests/-/issues/91Checking Location header on Operation Tasks should be removed2020-11-19T15:16:14ZElian KrajaChecking Location header on Operation Tasks should be removedLine 22 in VNFLifeCycleManagementAPI/CancelOperationTask.robot
` Should Contain ${headers} Location`
should be removed since it is not required.
The same operation should be done on the other LCM Operations TASKLine 22 in VNFLifeCycleManagementAPI/CancelOperationTask.robot
` Should Contain ${headers} Location`
should be removed since it is not required.
The same operation should be done on the other LCM Operations TASKhttps://forge.etsi.org/rep/nfv/SOL005/-/issues/6Mismatch between AffectedVirtualLink data type in SOL005 and as defined in ya...2020-12-08T17:54:48ZVlademir BrusseMismatch between AffectedVirtualLink data type in SOL005 and as defined in yaml file.It was found a mismatch between the SOL005 v2.7.1 and the SOL005NSLifecycleManagement_def.yaml for an attribute name of the AffectedVirtualLink data type:
AffectedVirtualLink:
description: >
This type provides information abou...It was found a mismatch between the SOL005 v2.7.1 and the SOL005NSLifecycleManagement_def.yaml for an attribute name of the AffectedVirtualLink data type:
AffectedVirtualLink:
description: >
This type provides information about added, deleted, modified and
temporary VLs.
type: object
required:
- id
- virtualLinkDescId
- changeType
- networkResource
```
The solution requires to replace the "id" attribute name with "nsVirtualLinkInstanceId" in all occurrences as defined in SOL005 v2.7.1.Giacomo BerniniGiacomo Berninihttps://forge.etsi.org/rep/nfv/SOL005/-/issues/1Swagger file: SOL005NSLifecycleManagement_def.yaml; ns_lcm_op_occ: "statusEnt...2020-12-15T05:03:17ZbanerjeesuSwagger file: SOL005NSLifecycleManagement_def.yaml; ns_lcm_op_occ: "statusEnteredTime" should be "stateEnteredTime" to match vnf_lcm_op_occThe file https://forge.etsi.org/rep/nfv/SOL005/blob/v2.7.1/src/SOL005/NSLifecycleManagement/definitions/SOL005NSLifecycleManagement_def.yaml NsLcmOpOcc has "statusEnteredTime" misspelt.
It should ideally be "stateEnteredTime" to match ...The file https://forge.etsi.org/rep/nfv/SOL005/blob/v2.7.1/src/SOL005/NSLifecycleManagement/definitions/SOL005NSLifecycleManagement_def.yaml NsLcmOpOcc has "statusEnteredTime" misspelt.
It should ideally be "stateEnteredTime" to match the corresponding field in VnfLcmOpOcc: https://forge.etsi.org/rep/nfv/SOL002-SOL003/blob/v2.7.1/src/definitions/SOL002SOL003VNFLifecycleManagement_def.yaml
Reference: Attached screenshot from 2.7.1 SOL005 Spec.
The description against 'statusEnteredTime' says, "Date-time when the current _state_ has been entered." Please see that it's referred to as "state" in the descriptions and other field names from NsLcmOpOcc as well, e.g. {'operationState'}. Apart from the specification language aspect, we have the below challenge from implementation and client side.
The dissimilarity of the names in SOL005 and corresponding SOL003/SOL002 model attributes may pose limitations when it comes to client-side/server-side code trying to use the JSON-Object-mapping tools (e.g. ObjectMapper) for invoking/serving APIs across NS and VNF LCM operations. The ObjectMappers need to have field transformation logic (to-and-fro) to map "statusEnteredTime" to "stateEnteredTime" and back while moving between the SOL003 and SOL005 APIs for instance.
![Screen_Shot_2020-08-21_at_11.43.35_AM](/uploads/73c67a0ec0f82a5fb40227e6d2526691/Screen_Shot_2020-08-21_at_11.43.35_AM.png)https://forge.etsi.org/rep/nfv/SOL005/-/issues/7Mismatch between AffectedVirtualLink data type in SOL005 and as defined in ya...2020-12-16T14:20:20ZVlademir BrusseMismatch between AffectedVirtualLink data type in SOL005 and as defined in yaml file.In order to have backward compatibility, SOL WG has decided in the #158-e-Meeting to fix this issue in other SOL005 OpenAPI versions.
The issue is same already fixed in other versions:
It was found a mismatch between the SOL005 v2.6.1 ...In order to have backward compatibility, SOL WG has decided in the #158-e-Meeting to fix this issue in other SOL005 OpenAPI versions.
The issue is same already fixed in other versions:
It was found a mismatch between the SOL005 v2.6.1 and the SOL005NSLifecycleManagement_def.yaml in SOL005 OpenAPI v2.6.1 for an attribute name of the AffectedVirtualLink data type:
AffectedVirtualLink: description: > This type provides information about added, deleted, modified and temporary VLs. type: object required: - id - virtualLinkDescId - changeType - networkResource
```
The solution requires to replace the "id" attribute name with "nsVirtualLinkInstanceId" in all occurrences as defined in SOL005 v2.6.1.
```Giacomo BerniniGiacomo Berninihttps://forge.etsi.org/rep/nfv/SOL005/-/issues/8Mismatch between AffectedVirtualLink data type in SOL005 and as defined in ya...2020-12-16T16:17:01ZVlademir BrusseMismatch between AffectedVirtualLink data type in SOL005 and as defined in yaml file.In order to have backward compatibility, SOL WG has decided in the #158-e-Meeting to fix this issue in other SOL005 OpenAPI versions.
The issue is same already fixed in other versions:
It was found a mismatch between the SOL005 v2.5.1 ...In order to have backward compatibility, SOL WG has decided in the #158-e-Meeting to fix this issue in other SOL005 OpenAPI versions.
The issue is same already fixed in other versions:
It was found a mismatch between the SOL005 v2.5.1 and the SOL005NSLifecycleManagement_def.yaml in SOL005 OpenAPI v2.5.1 for an attribute name of the AffectedVirtualLink data type:
AffectedVirtualLink: description: > This type provides information about added, deleted, modified and temporary VLs. type: object required: - id - virtualLinkDescId - changeType - networkResource
```
The solution requires to replace the "id" attribute name with "nsVirtualLinkInstanceId" in all occurrences as defined in SOL005 v2.5.1.
```Giacomo BerniniGiacomo Berninihttps://forge.etsi.org/rep/nfv/SOL005/-/issues/9Mismatch between AffectedVirtualLink data type in SOL005 and as defined in ya...2020-12-16T17:36:44ZVlademir BrusseMismatch between AffectedVirtualLink data type in SOL005 and as defined in yaml file.In order to have backward compatibility, SOL WG has decided in the #158-e-Meeting to fix this issue in other SOL005 OpenAPI versions.
The issue is same already fixed in other versions:
It was found a mismatch between the SOL005 v2.4.1 ...In order to have backward compatibility, SOL WG has decided in the #158-e-Meeting to fix this issue in other SOL005 OpenAPI versions.
The issue is same already fixed in other versions:
It was found a mismatch between the SOL005 v2.4.1 and the SOL005NSLifecycleManagement_def.yaml in SOL005 OpenAPI v2.4.1 for an attribute name of the AffectedVirtualLink data type:
AffectedVirtualLink: description: > This type provides information about added, deleted, modified and temporary VLs. type: object required: - id - virtualLinkDescId - changeType - networkResource
```
The solution requires to replace the "id" attribute name with "nsVirtualLinkInstanceId" in all occurrences as defined in SOL005 v2.4.1.
```Giacomo BerniniGiacomo Berninihttps://forge.etsi.org/rep/nfv/SOL005/-/issues/16NsLocationConstraint data type.2021-01-13T18:08:17ZVlademir BrusseNsLocationConstraint data type.The InstantiateNsRequest data type has the nestedNslocationConstraints attribute defined as items: $ref: "#/definitions/NsLocationConstraint".
NsLocationConstraint does not exist in SOL005 v3.3.1 and should be replaced with NestedNsloc...The InstantiateNsRequest data type has the nestedNslocationConstraints attribute defined as items: $ref: "#/definitions/NsLocationConstraint".
NsLocationConstraint does not exist in SOL005 v3.3.1 and should be replaced with NestedNslocationConstrainsts and the definition of the NsLocationConstraint data type (line 4921) removed.Giacomo BerniniGiacomo Berninihttps://forge.etsi.org/rep/nfv/SOL005/-/issues/13nsCpHandle attribute.2021-01-13T18:09:09ZVlademir BrussensCpHandle attribute.The nsCpHandle attribute with cardinality 0..1 in NsLinkPortInfo and VnffgInfo data types should have the yaml code generated as type “object” and not type “array” (as it is generated in the SOL005NSLifecycleManagement_def.yaml file of t...The nsCpHandle attribute with cardinality 0..1 in NsLinkPortInfo and VnffgInfo data types should have the yaml code generated as type “object” and not type “array” (as it is generated in the SOL005NSLifecycleManagement_def.yaml file of the SOL005 v2.6.1), because the cardinality 0..1 means that attribute is Optional and not an Array.
The NfpInfo data type also has the cpGroup attribute defined as items: $ref: "#/definitions/NsCpHandle" that should be items: $ref: "#/definitions/CpGroupInfo" as defined for NfpData.Giacomo BerniniGiacomo Berninihttps://forge.etsi.org/rep/nfv/api-tests/-/issues/105Notification API / NotificationEndpoint test purposes to be re-designed?2021-01-14T15:30:16ZMichele CarignaniNotification API / NotificationEndpoint test purposes to be re-designed?## Background
APIs marked `*Notification-API`specify the behaviour of the endpoints that receive notifications for asynchronous events.
When testing such APIs, the role of the FUT is the to implement the server which is able to receive...## Background
APIs marked `*Notification-API`specify the behaviour of the endpoints that receive notifications for asynchronous events.
When testing such APIs, the role of the FUT is the to implement the server which is able to receive the notification. The role of the Test System should be the one of the producer of notifications, i.e. the client in the notification exchanges.
## Problem
The `Notification-API`s Notification Endpoint tests currently implement the server side of the notification exchange (by means of the MockServer).
In more details:
* according to the SOL specs, the API Consumer implements (e.g. exposes )the NotificationEndpoint ("The API producer can use this resource to send notifications related to VNF alarms or about a rebuilt alarm list to a subscribed API consumer, which has provided the URI of this resource during the subscription process. ", SOL 003, 2.6.1, 7.4.6.1)
* in TST 010, we test the server of the resource
* Therefore the function to be tested in the Notification Endpoints is the API Consumer of a certain API (in the case of VNF Fault Management API this is the NFVO)
* In the tests keywords, you will see that the Test systems expects to receive the notifications and it checks if the notification is correct.
-> I am not saying that this is actually wrong, but it is not in line with the general approach to the test.
=> At least the Test objectives should clarify which IUT is expected to be tested.
### Affected test suites (i.e. Robot files)
* [ ] https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-dev/SOL002/VNFPerformanceManagementNotification-API/PerformanceManagementNotification.robot
* [ ] https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-dev/SOL002/VNFIndicatorNotification-API/VnfIndicatorNotification.robot
* [ ] https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-dev/SOL003/VNFFaultManagementNotification-API/NotificationEndpoint.robot
* [ ] https://forge.etsi.org/rep/nfv/api-tests/blob/2.6.1-dev/SOL003/VirtualisedResourcesQuotaAvailableNotification-API/NotificationEndpoint.robot
## Proposal
### Option 1
To be fixed in 2.7.1 by changing test purposes and test steps to just executed the following:
* Sending the notification
* verifying the response code is correct
### Option 2
* Change the test purpose to express that we are testing the API Producer capability of sending notifications
2.7.1Giacomo BerniniGiacomo Berninihttps://forge.etsi.org/rep/nfv/SOL005/-/issues/10Swagger File: VNFPackageManagement.yaml; Get '/vnf_packages' should not have ...2021-01-19T12:05:55ZjaissSwagger File: VNFPackageManagement.yaml; Get '/vnf_packages' should not have property 'VnfPkgInfo'The file:
https://forge.etsi.org/rep/nfv/SOL005/blob/v2.7.1/src/SOL005/VNFPackageManagement/VNFPackageManagement.yaml
Get '/vnf_packages' has following response:
```yaml
schema:
type: array
items:
...The file:
https://forge.etsi.org/rep/nfv/SOL005/blob/v2.7.1/src/SOL005/VNFPackageManagement/VNFPackageManagement.yaml
Get '/vnf_packages' has following response:
```yaml
schema:
type: array
items:
properties:
VnfPkgInfo:
$ref: "definitions/SOL005VNFPackageManagement_def.yaml#/definitions/VnfPkgInfo"
```
whereas its should be
```yaml
schema:
type: array
items:
$ref: "definitions/SOL005VNFPackageManagement_def.yaml#/definitions/VnfPkgInfo"
```
property 'VnfPkgInfo' is not required. Its a Data Type and not property.
Reference: Attached screenshot from 2.7.1 SOL005 Spec.
![Screen_Shot_2020-12-16_at_12.44.19_PM](/uploads/bdfb142a57c5a0c24a500452345cd293/Screen_Shot_2020-12-16_at_12.44.19_PM.png)https://forge.etsi.org/rep/nfv/SOL005/-/issues/14nsCpHandle attribute.2021-01-19T14:20:59ZVlademir BrussensCpHandle attribute.The nsCpHandle attribute with cardinality 0..1 in NsLinkPortInfo and VnffgInfo data types should have the yaml code generated as type “object” and not type “array” (as it is generated in the SOL005NSLifecycleManagement_def.yaml file of t...The nsCpHandle attribute with cardinality 0..1 in NsLinkPortInfo and VnffgInfo data types should have the yaml code generated as type “object” and not type “array” (as it is generated in the SOL005NSLifecycleManagement_def.yaml file of the SOL005 v2.5.1), because the cardinality 0..1 means that attribute is Optional and not an Array.
The NfpInfo data type also has the cpGroup attribute defined as items: $ref: "#/definitions/NsCpHandle" that should be items: $ref: "#/definitions/CpGroupInfo" as defined for NfpData.Giacomo BerniniGiacomo Bernini