Added tests for thresholds & updated versions

    ...    Test title: Get Individual Performance Report

    ...    Test title: Get Individual Performance Report

    ...    Test objective: The objective is to test the retrieval of an individual NS performance report associated to a monitoring job and perform a JSON schema validation of the collected report data structure

    ...    Test objective: The objective is to test the retrieval of an individual NS performance report associated to a monitoring job and perform a JSON schema validation of the collected report data structure

    ...    Pre-conditions: A NS instance is instantiated. One or more NS performance reports are set for a monitoring job in the NFVO.

    ...    Pre-conditions: A NS instance is instantiated. One or more NS performance reports are set for a monitoring job in the NFVO.

    ...    Reference: Clause 7.4.4.3.2 - ETSI GS NFV-SOL 005 [3] v2.7.1

    ...    Reference: Clause 7.4.4.3.2 - ETSI GS NFV-SOL 005 [3] v3.3.1

    ...    Config ID: Config_prod_NFVO

    ...    Config ID: Config_prod_NFVO

    ...    Applicability: none

    ...    Applicability: none

    ...    Post-Conditions: none

    ...    Post-Conditions: none

    ...    Test title: Get Individual Performance Report with invalid resource endpoint

    ...    Test title: Get Individual Performance Report with invalid resource endpoint

    ...    Test objective:  The objective is to test that the retrieval of an individual NS performance report associated to a monitoring job fails when using an invalid resource endpoint 

    ...    Test objective:  The objective is to test that the retrieval of an individual NS performance report associated to a monitoring job fails when using an invalid resource endpoint 

    ...    Pre-conditions: A NS instance is instantiated. One or more NS performance reports are set for a monitoring job in the NFVO.

    ...    Pre-conditions: A NS instance is instantiated. One or more NS performance reports are set for a monitoring job in the NFVO.

    ...    Reference: Clause 7.4.4.3.2 - ETSI GS NFV-SOL 005 [3] v2.7.1

    ...    Reference: Clause 7.4.4.3.2 - ETSI GS NFV-SOL 005 [3] v3.3.1

    ...    Config ID: Config_prod_NFVO

    ...    Config ID: Config_prod_NFVO

    ...    Applicability: none

    ...    Applicability: none

    ...    Post-Conditions: none

    ...    Post-Conditions: none

    ...    Test title: POST Individual Performance Report - Method not implemented

    ...    Test title: POST Individual Performance Report - Method not implemented

    ...    Test objective: The objective is to test that POST method is not allowed to create a new NS performance report within a monitoring job

    ...    Test objective: The objective is to test that POST method is not allowed to create a new NS performance report within a monitoring job

    ...    Pre-conditions: none

    ...    Pre-conditions: none

    ...    Reference: Clause 7.4.4.3.1 - ETSI GS NFV-SOL 005 [3] v2.7.1

    ...    Reference: Clause 7.4.4.3.1 - ETSI GS NFV-SOL 005 [3] v3.3.1

    ...    Config ID: Config_prod_NFVO

    ...    Config ID: Config_prod_NFVO

    ...    Applicability: none

    ...    Applicability: none

    ...    Post-Conditions: none

    ...    Post-Conditions: none

    ...    Test title: PUT Individual Performance Report - Method not implemented

    ...    Test title: PUT Individual Performance Report - Method not implemented

    ...    Test objective: The objective is to test that PUT method is not allowed to update an existing NS performance report within a monitoring job

    ...    Test objective: The objective is to test that PUT method is not allowed to update an existing NS performance report within a monitoring job

    ...    Pre-conditions: none

    ...    Pre-conditions: none

    ...    Reference: Clause 7.4.4.3.3 - ETSI GS NFV-SOL 005 [3] v2.7.1

    ...    Reference: Clause 7.4.4.3.3 - ETSI GS NFV-SOL 005 [3] v3.3.1

    ...    Config ID: Config_prod_NFVO

    ...    Config ID: Config_prod_NFVO

    ...    Applicability: none

    ...    Applicability: none

    ...    Post-Conditions: none

    ...    Post-Conditions: none

    ...    Test title: PATCH Individual Performance Report - Method not implemented

    ...    Test title: PATCH Individual Performance Report - Method not implemented

    ...    Test objective: The objective is to test that PATCH method is not allowed to modify an existing NS performance report within a monitoring job

    ...    Test objective: The objective is to test that PATCH method is not allowed to modify an existing NS performance report within a monitoring job

    ...    Pre-conditions: none

    ...    Pre-conditions: none

    ...    Reference: Clause 7.4.4.3.4 - ETSI GS NFV-SOL 005 [3] v2.7.1

    ...    Reference: Clause 7.4.4.3.4 - ETSI GS NFV-SOL 005 [3] v3.3.1

    ...    Config ID: Config_prod_NFVO

    ...    Config ID: Config_prod_NFVO

    ...    Applicability: none

    ...    Applicability: none

    ...    Post-Conditions: none

    ...    Post-Conditions: none

    ...    Test title: DELETE Individual Performance Report - Method not implemented

    ...    Test title: DELETE Individual Performance Report - Method not implemented

    ...    Test objective: The objective is to test that DELETE method is not allowed to delete an existing NS performance report within a monitoring job

    ...    Test objective: The objective is to test that DELETE method is not allowed to delete an existing NS performance report within a monitoring job

    ...    Pre-conditions: none

    ...    Pre-conditions: none

    ...    Reference: Clause 7.4.4.3.5 - ETSI GS NFV-SOL 005 [3] v2.7.1

    ...    Reference: Clause 7.4.4.3.5 - ETSI GS NFV-SOL 005 [3] v3.3.1

    ...    Config ID: Config_prod_NFVO

    ...    Config ID: Config_prod_NFVO

    ...    Applicability: none

    ...    Applicability: none

    ...    Post-Conditions: none

    ...    Post-Conditions: none

    ...    Test title: GET Individual Threshold

    ...    Test title: GET Individual Threshold

    ...    Test objective: The objective is to test the retrieval of an individual NS performance threshold and perform a JSON schema and content validation of the collected threshold data structure

    ...    Test objective: The objective is to test the retrieval of an individual NS performance threshold and perform a JSON schema and content validation of the collected threshold data structure

    ...    Pre-conditions: A NS instance is instantiated. One or more NS performance thresholds are set in the NFVO.

    ...    Pre-conditions: A NS instance is instantiated. One or more NS performance thresholds are set in the NFVO.

    ...    Reference: Clause 7.4.6.3.2 - ETSI GS NFV-SOL 005 [3] v2.7.1

    ...    Reference: Clause 7.4.6.3.2 - ETSI GS NFV-SOL 005 [3] v3.3.1

    ...    Config ID: Config_prod_NFVO

    ...    Config ID: Config_prod_NFVO

    ...    Applicability: none

    ...    Applicability: none

    ...    Post-Conditions: none

    ...    Post-Conditions: none

    ...    Test title: GET Individual Threshold with invalid resource identifier

    ...    Test title: GET Individual Threshold with invalid resource identifier

    ...    Test objective: The objective is to test that the retrieval of an individual NS performance threshold fails when using an invalid resource identifier

    ...    Test objective: The objective is to test that the retrieval of an individual NS performance threshold fails when using an invalid resource identifier

    ...    Pre-conditions: A NS instance is instantiated. One or more NS performance jobs are set in the NFVO.

    ...    Pre-conditions: A NS instance is instantiated. One or more NS performance jobs are set in the NFVO.

    ...    Reference: Clause 7.4.6.3.2 - ETSI GS NFV-SOL 005 [3] v2.7.1

    ...    Reference: Clause 7.4.6.3.2 - ETSI GS NFV-SOL 005 [3] v3.3.1

    ...    Config ID: Config_prod_NFVO

    ...    Config ID: Config_prod_NFVO

    ...    Applicability: none

    ...    Applicability: none

    ...    Post-Conditions: none

    ...    Post-Conditions: none

    ...    Test title: DELETE Individual Threshold with invalid resource identifier

    ...    Test title: DELETE Individual Threshold with invalid resource identifier

    ...    Test objective: The objective is to test the deletion of an individual NS performance threshold

    ...    Test objective: The objective is to test the deletion of an individual NS performance threshold

    ...    Pre-conditions: A NS instance is instantiated. One or more NS performance thresholds are set in the NFVO.

    ...    Pre-conditions: A NS instance is instantiated. One or more NS performance thresholds are set in the NFVO.

    ...    Reference: Clause 7.4.6.3.5 - ETSI GS NFV-SOL 005 [3] v2.7.1

    ...    Reference: Clause 7.4.6.3.5 - ETSI GS NFV-SOL 005 [3] v3.3.1

    ...    Config ID: Config_prod_NFVO

    ...    Config ID: Config_prod_NFVO

    ...    Applicability: none

    ...    Applicability: none

    ...    Post-Conditions: none   

    ...    Post-Conditions: none   

    ...    Test title: POST Individual Threshold - Method not implemented

    ...    Test title: POST Individual Threshold - Method not implemented

    ...    Test objective: The objective is to test that POST method is not allowed to create a new NS performance Threshold

    ...    Test objective: The objective is to test that POST method is not allowed to create a new NS performance Threshold

    ...    Pre-conditions: A NS instance is instantiated

    ...    Pre-conditions: A NS instance is instantiated

    ...    Reference: Clause 7.4.6.3.1 - ETSI GS NFV-SOL 005 [3] v2.7.1

    ...    Reference: Clause 7.4.6.3.1 - ETSI GS NFV-SOL 005 [3] v3.3.1

    ...    Config ID: Config_prod_NFVO

    ...    Config ID: Config_prod_NFVO

    ...    Applicability: none

    ...    Applicability: none

    ...    Post-Conditions: none

    ...    Post-Conditions: none

    ...    Test title: PUT Individual Threshold - Method not implemented

    ...    Test title: PUT Individual Threshold - Method not implemented

    ...    Test objective: The objective is to test that PUT method is not allowed to update an existing NS performance threshold

    ...    Test objective: The objective is to test that PUT method is not allowed to update an existing NS performance threshold

    ...    Pre-conditions: A NS instance is instantiated. One or more NS performance thresholds are set in the NFVO.

    ...    Pre-conditions: A NS instance is instantiated. One or more NS performance thresholds are set in the NFVO.

    ...    Reference: Clause 7.4.6.3.3 - ETSI GS NFV-SOL 005 [3] v2.7.1

    ...    Reference: Clause 7.4.6.3.3 - ETSI GS NFV-SOL 005 [3] v3.3.1

    ...    Config ID: Config_prod_NFVO

    ...    Config ID: Config_prod_NFVO

    ...    Applicability: none

    ...    Applicability: none

    ...    Post-Conditions: none

    ...    Post-Conditions: none

    ...    Test title: PATCH Individual Threshold 

    ...    Test title: PATCH Individual Threshold 

    ...    Test objective: The objective is to test that PATCH method modify an existing NS performance threshold

    ...    Test objective: The objective is to test that PATCH method modify an existing NS performance threshold

    ...    Pre-conditions: A NS instance is instantiated. One or more NS performance thresholds are set in the NFVO.

    ...    Pre-conditions: A NS instance is instantiated. One or more NS performance thresholds are set in the NFVO.

    ...    Reference: Clause 7.4.6.3.4 - ETSI GS NFV-SOL 005 [3] v2.7.1

    ...    Reference: Clause 7.4.6.3.4 - ETSI GS NFV-SOL 005 [3] v3.3.1

    ...    Config ID: Config_prod_NFVO

    ...    Config ID: Config_prod_NFVO

    ...    Applicability: none

    ...    Applicability: none

    ...    Post-Conditions: The NS performance Threshold is modified by the operation

    ...    Post-Conditions: The NS performance Threshold is modified by the operation

    ...    Test title: DELETE Individual Threshold

    ...    Test title: DELETE Individual Threshold

    ...    Test objective: The objective is to test the deletion of an individual NS performance threshold

    ...    Test objective: The objective is to test the deletion of an individual NS performance threshold

    ...    Pre-conditions: A NS instance is instantiated. One or more NS performance thresholds are set in the NFVO.

    ...    Pre-conditions: A NS instance is instantiated. One or more NS performance thresholds are set in the NFVO.

    ...    Reference: Clause 7.4.6.3.5 - ETSI GS NFV-SOL 005 [3] v2.7.1

    ...    Reference: Clause 7.4.6.3.5 - ETSI GS NFV-SOL 005 [3] v3.3.1

    ...    Config ID: Config_prod_NFVO

    ...    Config ID: Config_prod_NFVO

    ...    Applicability: none

    ...    Applicability: none

    ...    Post-Conditions: The NS performance Threshold is not available anymore in the NFVO    

    ...    Post-Conditions: The NS performance Threshold is not available anymore in the NFVO    

    ...    Test title: PATCH Individual Threshold - Precondition failed 

    ...    Test title: PATCH Individual Threshold - Precondition failed 

    ...    Test objective: The objective is to attempt to Modify an individual NS Performance threshold fails, where the precondition was not met

    ...    Test objective: The objective is to attempt to Modify an individual NS Performance threshold fails, where the precondition was not met

    ...    Pre-conditions: A NS instance is instantiated. One or more NS performance thresholds are set in the NFVO.

    ...    Pre-conditions: A NS instance is instantiated. One or more NS performance thresholds are set in the NFVO.

    ...    Reference: Clause 7.4.6.3.4 - ETSI GS NFV-SOL 005 [3] v2.7.1

    ...    Reference: Clause 7.4.6.3.4 - ETSI GS NFV-SOL 005 [3] v3.3.1

    ...    Config ID: Config_prod_NFVO

    ...    Config ID: Config_prod_NFVO

    ...    Applicability: none

    ...    Applicability: none

    ...    Post-Conditions: The NS performance Threshold is not modified by the operation

    ...    Post-Conditions: The NS performance Threshold is not modified by the operation

    Check HTTP Response Status Code Is    412

    Check HTTP Response Status Code Is    412

    Check HTTP Response Body Json Schema Is    ProblemDetails

    Check HTTP Response Body Json Schema Is    ProblemDetails

    Check Postcondition NS performance Threshold is Unmodified (Implicit)

    Check Postcondition NS performance Threshold is Unmodified (Implicit)

PATCH Individual Threshold - Unprocessable Entity

    [Documentation]    Test ID: 5.3.4.2.9

    ...    Test title: PATCH Individual NS Performance Job - Unprocessable Entity

    ...    Test objective: The objective is to test that PATCH method cannot modify an exsisting individual PM job resource when a request contains syntactically correct data but the data cannot be processed.

    ...    Pre-conditions: A NS instance is instantiated. One or more NS performance jobs are set in the NFVO.

    ...    Reference: Clause 7.4.6.3.4 - ETSI GS NFV-SOL 005 [3] v3.3.1

    ...    Config ID: Config_prod_NFVO

    ...    Applicability: none

    ...    Post-Conditions: The NS performance Threshold is not modified by the operation

    Send Patch request for Individual Threshold with unprocessable entity

    Check HTTP Response Status Code Is    422

    Check HTTP Response Body Json Schema Is    ProblemDetails

    Check Postcondition NS performance Threshold is Unmodified (Implicit)

 No newline at end of file

    Set Suite Variable    ${response}    ${output}

    Set Suite Variable    ${response}    ${output}

Send Post Request Create new Performance Threshold

Send Post Request Create new Performance Threshold

    Log    Creating a new THreshold

    Log    Creating a new Threshold

    Run Keyword If    ${AUTH_USAGE} == 1    Set Headers    {"${AUTHORIZATION_HEADER}":"${AUTHORIZATION_TOKEN}"}

    Run Keyword If    ${AUTH_USAGE} == 1    Set Headers    {"${AUTHORIZATION_HEADER}":"${AUTHORIZATION_TOKEN}"}

    Set Headers    {"Accept": "${ACCEPT_JSON}"}

    Set Headers    {"Accept": "${ACCEPT_JSON}"}

    Set Headers    {"Content-Type": "${CONTENT_TYPE_JSON}"}

    Set Headers    {"Content-Type": "${CONTENT_TYPE_JSON}"}

    ${output}=    Output    response

    ${output}=    Output    response

    Set Suite Variable    ${response}    ${output}

    Set Suite Variable    ${response}    ${output}

Send POST request for Performance threshold with unprocessable entity

    Log    Creating a new Threshold

    Run Keyword If    ${AUTH_USAGE} == 1    Set Headers    {"${AUTHORIZATION_HEADER}":"${AUTHORIZATION_TOKEN}"}

    Set Headers    {"Accept": "${ACCEPT_JSON}"}

    Set Headers    {"Content-Type": "${CONTENT_TYPE_JSON}"}

    ${template}=    Get File    jsons/CreateThresholdRequest.json

    ${body}=        Format String   ${template}    callback_uri=${unreachable_callback_uri}:${callback_port}

    POST    ${apiRoot}/${apiName}/${apiMajorVersion}/thresholds    ${body}

    ${output}=    Output    response

    Set Suite Variable    ${response}    ${output}

Send PUT Request for all Performance Thresholds

Send PUT Request for all Performance Thresholds

    Log    PUT THresholds

    Log    PUT THresholds

    Run Keyword If    ${AUTH_USAGE} == 1    Set Headers    {"${AUTHORIZATION_HEADER}":"${AUTHORIZATION_TOKEN}"}

    Run Keyword If    ${AUTH_USAGE} == 1    Set Headers    {"${AUTHORIZATION_HEADER}":"${AUTHORIZATION_TOKEN}"}

    ${output}=    Output    response

    ${output}=    Output    response

    Set Suite Variable    ${response}    ${output}

    Set Suite Variable    ${response}    ${output}

Send Patch request for Individual Threshold with unprocessable entity

    Set Headers  {"Accept":"${ACCEPT_JSON}"} 

    Set Headers  {"Content-Type": "${CONTENT_TYPE_PATCH}"} 

    Run Keyword If    ${AUTH_USAGE} == 1    Set Headers    {"${AUTHORIZATION_HEADER}":"${AUTHORIZATION_TOKEN}"}

    ${template}=    Get File    jsons/ThresholdModifications.json

    ${body}=        Format String   ${template}    callback_uri=${unreachable_callback_uri}:${callback_port}

    PATCH    ${apiRoot}/${apiName}/${apiMajorVersion}/thresholds/${thresholdId}     ${body}

    ${output}=    Output    response

    Set Suite Variable    ${response}    ${output}

Check Postcondition NS performance Threshold is Unmodified (Implicit)

Check Postcondition NS performance Threshold is Unmodified (Implicit)

    Log    Check postconidtion threshold not modified

    Log    Check postconidtion threshold not modified

    GET individual NS performance Threshold

    GET individual NS performance Threshold

    ...    Test title: NS Performance Information Availability Notification

    ...    Test title: NS Performance Information Availability Notification

    ...    Test objective: The objective is to test that NS Performance Information Availability Notification is delivered with success to the notification consumer

    ...    Test objective: The objective is to test that NS Performance Information Availability Notification is delivered with success to the notification consumer

    ...    Pre-conditions: A NS performance job is created, and information availability notifications is available in the NFVO.

    ...    Pre-conditions: A NS performance job is created, and information availability notifications is available in the NFVO.

    ...    Reference: Clause 7.4.9.3.1 - ETSI GS NFV-SOL 005 [3] v2.7.1

    ...    Reference: Clause 7.4.9.3.1 - ETSI GS NFV-SOL 005 [3] v3.3.1

    ...    Config ID: Config_prod_Notif_Endpoint

    ...    Config ID: Config_prod_Notif_Endpoint

    ...    Applicability: none

    ...    Applicability: none

    ...    Post-Conditions: none 

    ...    Post-Conditions: none 

    ...    Test title: NS Threshold Crossed Notification

    ...    Test title: NS Threshold Crossed Notification

    ...    Test objective: The objective is to test that NS Threshold Crossed Notification is delivered with success to the notification consumer

    ...    Test objective: The objective is to test that NS Threshold Crossed Notification is delivered with success to the notification consumer

    ...    Pre-conditions: A NS performance job is created, and threshold crossed notifications is available in the NFVO.

    ...    Pre-conditions: A NS performance job is created, and threshold crossed notifications is available in the NFVO.

    ...    Reference: Clause 7.4.9.3.1 - ETSI GS NFV-SOL 005 [3] v2.7.1

    ...    Reference: Clause 7.4.9.3.1 - ETSI GS NFV-SOL 005 [3] v3.3.1

    ...    Config ID: Config_prod_Notif_Endpoint

    ...    Config ID: Config_prod_Notif_Endpoint

    ...    Applicability: none

    ...    Applicability: none

    ...    Post-Conditions: none 

    ...    Post-Conditions: none 

    ...    Test title: NS Performance Information Availability Notification

    ...    Test title: NS Performance Information Availability Notification

    ...    Test objective: The objective is to test the dispatch of NS Performance Information Availability Notification when new NS performance information is available in the NFVO, and perform a JSON schema and content validation of the delivered notification. The action that triggers the notification under test is an explicit test step, but it is not performed by the test system.

    ...    Test objective: The objective is to test the dispatch of NS Performance Information Availability Notification when new NS performance information is available in the NFVO, and perform a JSON schema and content validation of the delivered notification. The action that triggers the notification under test is an explicit test step, but it is not performed by the test system.

    ...    Pre-conditions: A NS performance job is created, and a subscription for information availability notifications is available in the NFVO.

    ...    Pre-conditions: A NS performance job is created, and a subscription for information availability notifications is available in the NFVO.

    ...    Reference: Clause 7.4.9.3.1 - ETSI GS NFV-SOL 005 [3] v2.7.1

    ...    Reference: Clause 7.4.9.3.1 - ETSI GS NFV-SOL 005 [3] v3.3.1

    ...    Config ID: Config_prod_NFVO

    ...    Config ID: Config_prod_NFVO

    ...    Applicability: none

    ...    Applicability: none

    ...    Post-Conditions: none 

    ...    Post-Conditions: none 

    ...    Test title: NS Threshold Crossed Notification

    ...    Test title: NS Threshold Crossed Notification

    ...    Test objective: The objective is to test the dispatch of NS Threshold Crossed Notification when a previously set NS performance metric threshold is crossed, and perform a JSON schema and content validation of the delivered notification. The action that triggers the notification under test is an explicit test step, but it is not performed by the test system.

    ...    Test objective: The objective is to test the dispatch of NS Threshold Crossed Notification when a previously set NS performance metric threshold is crossed, and perform a JSON schema and content validation of the delivered notification. The action that triggers the notification under test is an explicit test step, but it is not performed by the test system.

    ...    Pre-conditions: A NS performance job is created, and a threshold subscription is available in the NFVO.

    ...    Pre-conditions: A NS performance job is created, and a threshold subscription is available in the NFVO.

    ...    Reference: Clause 7.4.9.3.1 - ETSI GS NFV-SOL 005 [3] v2.7.1

    ...    Reference: Clause 7.4.9.3.1 - ETSI GS NFV-SOL 005 [3] v3.3.1

    ...    Config ID: Config_prod_NFVO

    ...    Config ID: Config_prod_NFVO

    ...    Applicability: none

    ...    Applicability: none

    ...    Post-Conditions: none 

    ...    Post-Conditions: none