api-tests issueshttps://forge.etsi.org/rep/nfv/api-tests/-/issues2021-10-08T01:33:04Zhttps://forge.etsi.org/rep/nfv/api-tests/-/issues/194Keywords not running when testing Individual VNFInstances SOL003 VNFLifecycle...2021-10-08T01:33:04ZyaoyuyKeywords not running when testing Individual VNFInstances SOL003 VNFLifecycleMangementAPI V2.6.1when testing task PATCH Individual VNFInstance Precondition failed and PATCH Individual VNFInstance Conflict
the keywords Launch another LCM operation and SET etag should run, but in the log file, It seems not runnningwhen testing task PATCH Individual VNFInstance Precondition failed and PATCH Individual VNFInstance Conflict
the keywords Launch another LCM operation and SET etag should run, but in the log file, It seems not runnninghttps://forge.etsi.org/rep/nfv/api-tests/-/issues/193unexpected GET method runs after http code 303 returned in Subscriptions.robo...2021-10-05T10:15:18Zyaoyuyunexpected GET method runs after http code 303 returned in Subscriptions.robot SOL003 VNFLifecycleManagementAPIWhen testing Create a new Subscription - NO-DUPLICATION task,
robot runs GET API after getting 303 in header, and the response value now is from the GET API which cause the task failed.When testing Create a new Subscription - NO-DUPLICATION task,
robot runs GET API after getting 303 in header, and the response value now is from the GET API which cause the task failed.https://forge.etsi.org/rep/nfv/api-tests/-/issues/192Wrong Parameter name in IndividualVNFInstance.robot SOL003 VNFLifecysleManage...2021-10-05T10:15:18ZyaoyuyWrong Parameter name in IndividualVNFInstance.robot SOL003 VNFLifecysleManagementAPIIn `DELETE Individual VNFInstance Conflict` test task
when check resource existence, the API is
`Get ${apiRoot}/${apiName}/${apiVersion}/vnf_instances/${vnfInstanceId}`
it should be
`Get ${apiRoot}/${apiName}/${apiVersion}/vnf_instance...In `DELETE Individual VNFInstance Conflict` test task
when check resource existence, the API is
`Get ${apiRoot}/${apiName}/${apiVersion}/vnf_instances/${vnfInstanceId}`
it should be
`Get ${apiRoot}/${apiName}/${apiVersion}/vnf_instances/${instantiatedVnfInstanceId}`https://forge.etsi.org/rep/nfv/api-tests/-/issues/190Wrong Problem Details JSON Object set in Keywords Check HTTP Response Body Js...2021-10-05T10:15:18ZyaoyuyWrong Problem Details JSON Object set in Keywords Check HTTP Response Body Json Schema Is ProblemDetails SOL005 VNFPackageManagementAPI v2.6.1When check HTTP Response Body Json Schema Is ProblemDetails
the problem details json object should be application/problem+json according to the RFC7807
but now the value is application/json
https://datatracker.ietf.org/doc/html/rfc7807When check HTTP Response Body Json Schema Is ProblemDetails
the problem details json object should be application/problem+json according to the RFC7807
but now the value is application/json
https://datatracker.ietf.org/doc/html/rfc7807https://forge.etsi.org/rep/nfv/api-tests/-/issues/187Missing set suite variable in keyword in vnfpackagemanagement keyword at mult...2021-09-21T16:43:57ZsinghadiMissing set suite variable in keyword in vnfpackagemanagement keyword at multiple check position```
Run Keyword If ${AUTH_USAGE} == 1 Set Headers {"${AUTHORIZATION_HEADER}":"${AUTHORIZATION_TOKEN}"}
GET ${apiRoot}/${apiName}/${apiVersion}/vnf_packages/${vnfPkgZipVNFD}/vnfd
Check HTTP Response Status Code Is ...```
Run Keyword If ${AUTH_USAGE} == 1 Set Headers {"${AUTHORIZATION_HEADER}":"${AUTHORIZATION_TOKEN}"}
GET ${apiRoot}/${apiName}/${apiVersion}/vnf_packages/${vnfPkgZipVNFD}/vnfd
Check HTTP Response Status Code Is 200
```https://forge.etsi.org/rep/nfv/api-tests/-/issues/186Unsupported function tested in test case VNFLifecycleManagementAPI Subscripti...2021-06-28T13:36:44ZyaoyuyUnsupported function tested in test case VNFLifecycleManagementAPI Subscriptions SOL003 v2.6.1According to the specicfication file, the parameters supported by the GET method on subscriptions are filter and nextpage_opaque_marker.<br>
However, the attribute selector like exclude_fileds,exlude_default is also tested in Subscriptio...According to the specicfication file, the parameters supported by the GET method on subscriptions are filter and nextpage_opaque_marker.<br>
However, the attribute selector like exclude_fileds,exlude_default is also tested in Subscriptions tests.<br>
Test cases are GET subscriptions with all_fields attribute selector, GET subscriptions with exclude_default attribute selector,GET subscriptions with fields attribute selector,GET subscriptions with exclude_fields attribute selector.
Is this part missing in the specification file or the test cases should be removed?
https://www.etsi.org/deliver/etsi_gs/NFV-SOL/001_099/003/02.06.01_60/gs_NFV-SOL003v020601p.pdf (Table 5.4.18.3.2-1: URI query parameters supported by the GET method on this resource)https://forge.etsi.org/rep/nfv/api-tests/-/issues/185Parameter value settings in VNFPackageManagementAPI SOL003/SOL0052021-09-22T08:37:04ZyaoyuyParameter value settings in VNFPackageManagementAPI SOL003/SOL005According to the specification file,
when getting the VNF Packages with attribute-based filter, the format should be like (eq,vnfdId,41fdd38a-3d4c-465c-83e0-f80e014425f8)
But now in the parameter text file, the sample settings are offere...According to the specification file,
when getting the VNF Packages with attribute-based filter, the format should be like (eq,vnfdId,41fdd38a-3d4c-465c-83e0-f80e014425f8)
But now in the parameter text file, the sample settings are offered like "vnfdId=41fdd38a-3d4c-465c-83e0-f80e014425f8 ,vnfProvider=NXW"
The reference files are as followed:
https://www.etsi.org/deliver/etsi_gs/NFV-SOL/001_099/003/02.06.01_60/gs_NFV-SOL003v020601p.pdf (Table 10.4.2.3.2-1: URI query parameters supported by the GET method on this resource)
https://www.etsi.org/deliver/etsi_gs/NFV-SOL/001_099/013/02.06.01_60/gs_NFV-SOL013v020601p.pdf (5.2 Attribute-based filtering)
(Maybe it won't be an issue, since the values can be changed when testing.(`・ω・´))https://forge.etsi.org/rep/nfv/api-tests/-/issues/174SOL002/vnflcm/{apiMajorVersion}/vnf_lcm_op_occs/${vnfLcmOpOccId}/fail API POS...2021-06-03T08:49:17ZguptanavSOL002/vnflcm/{apiMajorVersion}/vnf_lcm_op_occs/${vnfLcmOpOccId}/fail API POST V2.6.1When testing case 'POST Fail operation task' some Internal Server Error and Can not find requested vnf instance
*Attached are complete logs of the same* :
[a05010be-3af1-4aa1-b331-03d6659b9b1d.VNFLifecycleManagement-API.FailOperationT...When testing case 'POST Fail operation task' some Internal Server Error and Can not find requested vnf instance
*Attached are complete logs of the same* :
[a05010be-3af1-4aa1-b331-03d6659b9b1d.VNFLifecycleManagement-API.FailOperationTask.log.html](/uploads/85820332f35826f2c631de05e92cff32/a05010be-3af1-4aa1-b331-03d6659b9b1d.VNFLifecycleManagement-API.FailOperationTask.log.html)https://forge.etsi.org/rep/nfv/api-tests/-/issues/173SOL002 and SOL003 /vnflcm/{apiMajorVersion}/subscriptions/${subscriptionId} D...2021-05-28T07:11:32ZguptanavSOL002 and SOL003 /vnflcm/{apiMajorVersion}/subscriptions/${subscriptionId} DELETE V2.6.1When testing cases' '**DELETE an individual subscription**' HIVE TAP return 204 but expected is 405
*Attached are complete logs of the same* :
[8fd09181-deab-44dd-84f4-b5d96f391396.VNFLifecycleManagement-API.IndividualSubscription.log...When testing cases' '**DELETE an individual subscription**' HIVE TAP return 204 but expected is 405
*Attached are complete logs of the same* :
[8fd09181-deab-44dd-84f4-b5d96f391396.VNFLifecycleManagement-API.IndividualSubscription.log.html](/uploads/8bc7401b73e41327fb16f4f1bbe40ba7/8fd09181-deab-44dd-84f4-b5d96f391396.VNFLifecycleManagement-API.IndividualSubscription.log.html)
[fd116d50-052f-475c-9fb0-5fc382afdd73.VNFLifecycleManagement-API.IndividualSubscription.log.html](/uploads/275995211db731eeaf15f81f8536e16d/fd116d50-052f-475c-9fb0-5fc382afdd73.VNFLifecycleManagement-API.IndividualSubscription.log.html)https://forge.etsi.org/rep/nfv/api-tests/-/issues/172SOL002 /vnflcm/{apiMajorVersion}/subscriptions POST V2.6.12021-03-18T06:28:46ZguptanavSOL002 /vnflcm/{apiMajorVersion}/subscriptions POST V2.6.1When testing cases' **POST Create a new subscription**' it got failed with error message 'FileNotFoundError: [Errno 2] No such file or directory: 'schemas//VnfLcmOpOcc.schema.json'.' comes.
*Attached are complete logs of the same* :
[...When testing cases' **POST Create a new subscription**' it got failed with error message 'FileNotFoundError: [Errno 2] No such file or directory: 'schemas//VnfLcmOpOcc.schema.json'.' comes.
*Attached are complete logs of the same* :
[a1d86506-aee1-4893-ba5b-368a4e0a2f8c.VNFLifecycleManagement-API.Subscriptions.log.html](/uploads/2ac94ad8afddc6eb3f47fb87a88c9567/a1d86506-aee1-4893-ba5b-368a4e0a2f8c.VNFLifecycleManagement-API.Subscriptions.log.html)https://forge.etsi.org/rep/nfv/api-tests/-/issues/171SOL002 /vnflcm/{apiMajorVersion}/vnf_lcm_op_occs, /vnflcm/{apiMajorVersion}/v...2021-05-28T07:11:07ZguptanavSOL002 /vnflcm/{apiMajorVersion}/vnf_lcm_op_occs, /vnflcm/{apiMajorVersion}/vnf_lcm_op_occs/${vnfLcmOpOccId}, /vnflcm/{apiMajorVersion}/subscriptions and /vnflcm/{apiMajorVersion}/subscriptions/${subscriptionId} GET V2.6.1When testing cases
* '**GET status information about multiple VNF LCM operation occurrences**'
* '**Get information about multiple VNF instances**
* '**GET Subscriptions**'
* '**GET Individual Subscription**'
they all got fai...When testing cases
* '**GET status information about multiple VNF LCM operation occurrences**'
* '**Get information about multiple VNF instances**
* '**GET Subscriptions**'
* '**GET Individual Subscription**'
they all got failed with error message 'FileNotFoundError: [Errno 2] No such file or directory: 'schemas//VnfLcmOpOcc.schema.json'.' comes.
*Attached are complete logs of the same* :
[1011cde5-cbc8-4e78-abae-7f1e04825cde.VNFLifecycleManagement-API.VnfLcmOperationOccurences.log.html](/uploads/2ca2ec54d04c7e8bd9ad493b66a4a1a2/1011cde5-cbc8-4e78-abae-7f1e04825cde.VNFLifecycleManagement-API.VnfLcmOperationOccurences.log.html)[e040fbc1-df19-4e79-a62a-1920dc7d9649.VNFLifecycleManagement-API.IndividualVnfLcmOperationOccurence.log.html](/uploads/ac24de9919bac234e6149aa8ad3294c6/e040fbc1-df19-4e79-a62a-1920dc7d9649.VNFLifecycleManagement-API.IndividualVnfLcmOperationOccurence.log.html)[a1d86506-aee1-4893-ba5b-368a4e0a2f8c.VNFLifecycleManagement-API.Subscriptions.log.html](/uploads/59eb245f335e2fa95b7c48e37c9c0b83/a1d86506-aee1-4893-ba5b-368a4e0a2f8c.VNFLifecycleManagement-API.Subscriptions.log.html)[8fd09181-deab-44dd-84f4-b5d96f391396.VNFLifecycleManagement-API.IndividualSubscription.log.html](/uploads/708c1228e41dbbcf897ede19b5953c51/8fd09181-deab-44dd-84f4-b5d96f391396.VNFLifecycleManagement-API.IndividualSubscription.log.html)https://forge.etsi.org/rep/nfv/api-tests/-/issues/170SOL002 /vnflcm/{apiMajorVersion}/vnf_instances POST, /vnflcm/{apiMajorVersion...2021-05-28T07:11:16ZguptanavSOL002 /vnflcm/{apiMajorVersion}/vnf_instances POST, /vnflcm/{apiMajorVersion}/vnf_instances GET and /vnflcm/{apiMajorVersion}/vnf_instances/${vnfInstanceId} GET V2.6.1When testing cases
* '**Create a new vnfInstance**' some validation error for schema vnfInstance.schema.json: '*vnfPkgInfoId*' is a required property comes but HTTP return status is 201
* '**GET information about multiple VNF insta...When testing cases
* '**Create a new vnfInstance**' some validation error for schema vnfInstance.schema.json: '*vnfPkgInfoId*' is a required property comes but HTTP return status is 201
* '**GET information about multiple VNF instances**' some validation error for schema vnfInstance.schema.json: '*vnfPkgInfoId*' is a required property comes but HTTP return status is 201
* '**Get Information about an individual VNF Instance**' some validation error for schema vnfInstance.schema.json: '*vnfPkgInfoId*' is a required property comes but HTTP return status is 200
*Attached are complete logs of the same* :
[82461eeb-3b91-49b0-b713-9f177d06e830.VNFLifecycleManagement-API.VNFInstances.log.html](/uploads/fe2c3cf96711fd950ab429de66f8d8e4/82461eeb-3b91-49b0-b713-9f177d06e830.VNFLifecycleManagement-API.VNFInstances.log.html)[d7ccf875-e3c1-4823-a5fa-6f6e21790574.VNFLifecycleManagement-API.IndividualVNFInstance.log.html](/uploads/bdde85d95aa68e13898197f9a2b25354/d7ccf875-e3c1-4823-a5fa-6f6e21790574.VNFLifecycleManagement-API.IndividualVNFInstance.log.html)https://forge.etsi.org/rep/nfv/api-tests/-/issues/168TST issue in '/vnflcm/{apiMajorVersion}/vnf_lcm_op_occs SOL003 v2.6.1 api2021-03-09T14:17:25ZguptanavTST issue in '/vnflcm/{apiMajorVersion}/vnf_lcm_op_occs SOL003 v2.6.1 apiwhen testing **/vnflcm/{apiMajorVersion}/vnf_lcm_op_occs SOL003** api
HTTP return code is 200 but test suite is fail as it is expecting 202 return code
[ocss_TST_ISSUE.txt](/uploads/e8d79d9fe5cd48e124226acae9be210b/ocss_TST_ISSUE.txt)when testing **/vnflcm/{apiMajorVersion}/vnf_lcm_op_occs SOL003** api
HTTP return code is 200 but test suite is fail as it is expecting 202 return code
[ocss_TST_ISSUE.txt](/uploads/e8d79d9fe5cd48e124226acae9be210b/ocss_TST_ISSUE.txt)https://forge.etsi.org/rep/nfv/api-tests/-/issues/167POST Terminate a vnfInstance Conflict returned http code 202 SOL003 VNFLifecy...2021-06-03T08:49:17ZyaoyuyPOST Terminate a vnfInstance Conflict returned http code 202 SOL003 VNFLifecycleManagementAPI v2.6.1According to the specifitiaction, when termanting a not-instantiate state vnf instance, it should return http code 409, while the code check if it is 202.
specificaition SOL003 V2.6.1 Chapter 5.4.8.3.1
Typically, this is due to the fact...According to the specifitiaction, when termanting a not-instantiate state vnf instance, it should return http code 409, while the code check if it is 202.
specificaition SOL003 V2.6.1 Chapter 5.4.8.3.1
Typically, this is due to the fact that the "Individual VNF
instance" resource is in NOT_INSTANTIATED state,
that another lifecycle management operation is
ongoing, or that a required (see note) child attribute of
the "extensions" attribute has not been set.https://forge.etsi.org/rep/nfv/api-tests/-/issues/166Wrong data type for links SOL005 VNFPackageManagementAPI v2.6.12021-06-03T08:49:17ZyaoyuyWrong data type for links SOL005 VNFPackageManagementAPI v2.6.1Test failed beacause meeting the error as followed:
ValidationError: Validation error for schema links.schema.json: {'self': {'href': '/vnfpkgm/v1/vnf_packages/04d7a289-50d7-42c6-bcab-795712f0c3a5'}, 'packageContent': {'href': '/vnfpkgm/...Test failed beacause meeting the error as followed:
ValidationError: Validation error for schema links.schema.json: {'self': {'href': '/vnfpkgm/v1/vnf_packages/04d7a289-50d7-42c6-bcab-795712f0c3a5'}, 'packageContent': {'href': '/vnfpkgm/v1/vnf_packages/04d7a289-50d7-42c6-bcab-795712f0c3a5/package_content'}} is not of type 'array'
refered to the links.schema.json file, the code is
`"type": "array",`
but according to the sol specification (SOL005 v2.6.1 9.5.2.5 Type: VnfPkgInfo), the links's data type should be Structure (inlined)
[249cd2b9-168c-4272-80ff-0d348154432c.VNFPackageManagement-API.VNFPackages.log.html](/uploads/0e4260747fe1eabc4af0a4475b20f9de/249cd2b9-168c-4272-80ff-0d348154432c.VNFPackageManagement-API.VNFPackages.log.html)https://forge.etsi.org/rep/nfv/api-tests/-/issues/165Using the same parameter when checking full size and Content-length SOL003 v2...2021-06-03T08:49:17ZyaoyuyUsing the same parameter when checking full size and Content-length SOL003 v2.6.1 VNFPackageManagementAPISame parameter is used to check for the file size and range size.and they can only be eqaul if the range size is same with the file size.
eg. the file size is 51 and the request range is range :bytes=0-50
so the response in Content-R...Same parameter is used to check for the file size and range size.and they can only be eqaul if the range size is same with the file size.
eg. the file size is 51 and the request range is range :bytes=0-50
so the response in Content-Range will be bytes 0-50/51
and the Content-Length will be 51 too
Another situation is the file size is 1000 and the request range is range :bytes=0-50
so the response in Content-Range will be bytes 0-50/1000
and the Content-Length will be 51
The parameter used to be ${full_size} and ${length} but it is changed now.
```
Check HTTP Response Header Content-Range Is Present and Matches the requested range
Log Check Content-Range HTTP Header
Should Contain ${response['headers']} Content-Range
Should Be Equal As Strings ${response['headers']['Content-Range']} bytes ${range}/${full_length}
Log Header Content-Range is present
Check HTTP Response Header Content-Length Is Present and Matches the requested range length
Log Check Content-Length HTTP Header
Should Contain ${response['headers']} Content-Length
Should Be Equal As Integers ${response['headers']['Content-Length']} ${full_length}
Log Header Content-Length is present
```https://forge.etsi.org/rep/nfv/api-tests/-/issues/164SOL005 NSDManagement-API Subscriptions NFVO_DUPLICATION==0 is used in both te...2021-02-23T19:05:20ZvignaudSOL005 NSDManagement-API Subscriptions NFVO_DUPLICATION==0 is used in both test cases.In both use case :
`Create duplicated NSD Management subscription with NFVO not creating duplicated subscriptions`
and
`Create duplicated NSD Management subscription with NFVO creating duplicated subscriptions`
Seems to use NFVO_DU...In both use case :
`Create duplicated NSD Management subscription with NFVO not creating duplicated subscriptions`
and
`Create duplicated NSD Management subscription with NFVO creating duplicated subscriptions`
Seems to use NFVO_DUPLICATION==0 for skipping the test.https://forge.etsi.org/rep/nfv/api-tests/-/issues/163SOL005 NSDManagement IndividualSubscription PUT & PATCH methods fail to evalu...2021-03-04T08:26:19ZvignaudSOL005 NSDManagement IndividualSubscription PUT & PATCH methods fail to evaluate JsonThose methods fail with error message :
`Evaluating expression 'json.loads('''{'id': '527bb09d-29e2-4025-962e-1e21134a6836', <...SNIP...>''')' failed: JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 ...Those methods fail with error message :
`Evaluating expression 'json.loads('''{'id': '527bb09d-29e2-4025-962e-1e21134a6836', <...SNIP...>''')' failed: JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)`
[log.html](/uploads/38b97df5a0ca96e185e8c0d123fc86bf/log.html)https://forge.etsi.org/rep/nfv/api-tests/-/issues/162SOL005 IndividualNSDescriptor nsdUsageState is retreived on a 4042021-06-03T08:49:17ZvignaudSOL005 IndividualNSDescriptor nsdUsageState is retreived on a 404In `GET Individual Network Service Descriptor Information with invalid resource identifier` test case fail because
`Resolving variable '${response['body']['nsdUsageState']}' failed: KeyError: 'nsdUsageState'`
Of course it does not exi...In `GET Individual Network Service Descriptor Information with invalid resource identifier` test case fail because
`Resolving variable '${response['body']['nsdUsageState']}' failed: KeyError: 'nsdUsageState'`
Of course it does not exist on a ProblemDetails response.
[log.html](/uploads/4085f817643c1c793a5cc186e5a6389f/log.html)https://forge.etsi.org/rep/nfv/api-tests/-/issues/161SOL002 / IndividualVnfLcmOperationOccurence / 2.6.1 Multiple problems with PA...2021-06-03T08:49:17ZvignaudSOL002 / IndividualVnfLcmOperationOccurence / 2.6.1 Multiple problems with PATCH methodsHi,
According SOL002 v2.6.1 §5.4.3.3.4 The method PATCH exists.
Note that the patch method is using a vnfLcmOpOccId
> Patch ${apiRoot}/${apiName}/${apiVersion}/vnf_instances/${vnfLcmOpOccId}
According SOL002 v2.6.1 §5.4.3.3.5 The metho...Hi,
According SOL002 v2.6.1 §5.4.3.3.4 The method PATCH exists.
Note that the patch method is using a vnfLcmOpOccId
> Patch ${apiRoot}/${apiName}/${apiVersion}/vnf_instances/${vnfLcmOpOccId}
According SOL002 v2.6.1 §5.4.3.3.5 The method DELETE exists.
And same strange Id
> Delete ${apiRoot}/${apiName}/${apiVersion}/vnf_instances/${vnfLcmOpOccId}