Templates and folder structures for SOL005 APIs.

(Patch 2 fixes errors in build scripts and openapis templates)

Change-Id: I60a1253130d9e84461819aa8f3f7509d41b22212
Signed-off-by: Michele Carignani <michele.carignani@etsi.org>

.gitignore

+build/
+*.pyc

.jenkins.sh

+#!/bin/bash
+# Copyright ETSI 2017
+# See: https://forge.etsi.org/etsi-forge-copyright-statement.txt
+
+cd "$(dirname "$0")"
+
+run_dir="$(pwd)"
+
+rm build/*-API.yaml
+rm build/*-API.json
+
+cd docker
+
+./build-container.sh
+./run-container.sh "${run_dir}"
+
+cd ..
+
+python ./scripts/add_change_comment.py
+
+exit $?

COPYRIGHT

+ETSI FORGE COPYRIGHT STATEMENT
+
+Copyright (c) ETSI 2018.
+
+This software is subject to copyrights owned by ETSI. Non-exclusive permission 
+is hereby granted, free of charge, to copy, reproduce and amend this file 
+under the following conditions: It is provided "as is", without warranty of any 
+kind, expressed or implied. 
+
+ETSI shall never be liable for any claim, damages, or other liability arising 
+from its use or inability of use.This permission does not apply to any documentation 
+associated with this file for which ETSI keeps all rights reserved. The present 
+copyright notice shall be included in all copies of whole or part of this 
+software and shall not imply any sub-license right.
+

Readme.md

+# NFV SOL005 APIs
+
+This repository hosts the [OpenAPI](https://www.openapis.org/) specificatons and other documentation
+for the APIs defined in ETSI NFV GSs [SOL005](http://www.etsi.org/deliver/etsi_gs/NFV-SOL/001_099/005/02.04.01_60/gs_NFV-SOL005v020401p.pdf).
+
+The APIs described in this repository are defined for the `Os-Ma-nfvo` reference point.
+
+**IMPORTANT: Please note that this file might be not aligned to the current version of the ETSI Group Specification it refers to and has not been approved by the ETSI NFV ISG. In case of discrepancies the published ETSI Group Specification takes precedence.**
+
+More information at [NFV Solutions wiki](https://nfvwiki.etsi.org/index.php?title=NFV_Solutions).
+
+## Content structure
+
+All the main OpenAPI files should be put in the `src` directory and organized with the following schema:
+
+    src/deliverable-number/api-name/api-name.yaml
+   
+e.g.
+
+    src/SOL005/NSDManagement/NSDManagement.yaml
+   
+   
+## Building
+
+The build file `.jenkins.sh` will validate the files, evaluate JSON references and store the generated files in the build directory. In case the build is executed by Jenkins, the merged file will be store in a specific directory to be web-served.
+
+To build locally, once Docker is installed, simple run
+
+    $ sudo bash .jenkins.sh
+   
+and the results will be stored in a `build/` directory in the project folder.
+
+## How to raise issues
+
+Change requests can be filed at [ETSI Forge Bugzilla](forge.etsi.org/bugzilla/buglist.cgi?component=Nfv-Openapis&list_id=62&product=NFV). Please report errors, bugs or other issues [here](https://forge.etsi.org/bugzilla/enter_bug.cgi?product=NFV).
+
+## How to contribute
+
+ETSI Forge uses Gerrit to manage submissions to the repository, any submission is tracked with a Change. Currently open and closed Changes can be reviewed [here](https://forge.etsi.org/gerrit/#/q/project:NFV-SOL005).
+
+### Latest builds
+
+See the recent builds from:
+
+* [Master](https://forge.etsi.org/jenkins/job/NFV%20-%20Network%20Functions%20Virtualisation/job/sol005-master/)
+* [Gerrit changes](https://forge.etsi.org/jenkins/job/NFV%20-%20Network%20Functions%20Virtualisation/job/sol005/).
+
+## License
+
+Copyright (c) ETSI 2018.
+ 
+This software is subject to copyrights owned by ETSI. Non-exclusive permission 
+is hereby granted, free of charge, to copy, reproduce and amend this file 
+under the following conditions: It is provided "as is", without warranty of any 
+kind, expressed or implied. 
+
+ETSI shall never be liable for any claim, damages, or other liability arising 
+from its use or inability of use.This permission does not apply to any documentation 
+associated with this file for which ETSI keeps all rights reserved. The present 
+copyright notice shall be included in all copies of whole or part of this 
+file and shall not imply any sub-license right.
+

docker/Dockerfile

+# Copyright ETSI 2017
+# See: https://forge.etsi.org/etsi-forge-copyright-statement.txt
+
+FROM alpine:3.6
+
+RUN env
+RUN apk update
+RUN apk add bash
+RUN apk add nodejs
+RUN apk add nodejs-npm
+RUN apk add asciidoctor
+RUN apk add openjdk8
+RUN apk add ca-certificates wget && update-ca-certificates 
+RUN apk add openssl
+RUN gem install rdoc --pre || gem install rdoc --pre
+RUN gem install asciidoctor-pdf-cjk
+RUN wget https://forge.etsi.org/swagger2markup-cli-1.3.2.jar
+RUN npm config set proxy $http_proxy
+RUN npm install -g swagger-tools@0.10.3
+RUN npm install -g json-refs@3.0.2
+RUN npm install -g yamljs@0.3.0
+
+ADD validate-in-docker.sh /validate-in-docker.sh
+RUN chmod +x /validate-in-docker.sh
+
+ADD swg2mrkup /bin/swg2mrkup
+RUN chmod +x /bin/swg2mrkup
+
+ADD oas2pdf /bin/oas2pdf
+RUN chmod +x /bin/oas2pdf
+
+ENTRYPOINT ["/validate-in-docker.sh"]
+

docker/build-container.sh

+#!/bin/bash
+# Copyright ETSI 2017
+# See: https://forge.etsi.org/etsi-forge-copyright-statement.txt
+
+#set -x
+
+proxy = ""
+if [[ ! -v http_proxy ]]; then
+    echo "http_proxy is not set"
+elif [[ -z "$http_proxy" ]]; then
+    echo "http_proxy is empty"
+else
+    echo "http_proxy is set to $http_proxy"
+    if [[ $http_proxy =~ ^http:\/\/[0-9] ]]; then
+        echo "starts with http"
+        proxy=$http_proxy
+    elif [[ $http_proxy =~ ^[0-9] ]]; then
+        echo "starts with number"
+        proxy=http://$http_proxy
+    fi
+fi
+
+echo "Proxy set to $proxy"
+
+docker build --build-arg http_proxy=$proxy --build-arg https_proxy=$proxy -t openapivalidator .
\ No newline at end of file

docker/oas2pdf

+#!/bin/bash
+#Copyright (c) ETSI 2017.
+
+# This software is subject to copyrights owned by ETSI. Non-exclusive permission 
+# is hereby granted, free of charge, to copy, reproduce and amend this file 
+# under the following conditions: It is provided "as is", without warranty of any 
+# kind, expressed or implied. 
+
+# ETSI shall never be liable for any claim, damages, or other liability arising 
+# from its use or inability of use.This permission does not apply to any documentation 
+# associated with this file for which ETSI keeps all rights reserved. The present 
+# copyright notice shall be included in all copies of whole or part of this 
+# software and shall not imply any sub-license right.
+#
+# Author: michele.carignani@etsi.org
+#
+# This script takes an OpenAPI file in input and creates
+# a PDF file with the content.
+#
+# Prerequisites: 
+#  - https://github.com/Swagger2Markup/swagger2markup-cli
+#  - Asciidoctor (sudo apt-get install asciidoctor)
+#  - Asciidoctor-pdf (sudo gem install asciidoctor-pdf)
+#
+# Usage:
+#    oas2pdf <oasfile> (e.g. myapi.json)
+#
+# Result:
+#    A new pdf file is created in the working directory (e.g. myapi.pdf)
+#
+#
+#
+#
+
+# Configuration
+
+# Change this if needed
+SWG2MRKUP=swg2mrkup
+ASCIIDOCPDF=asciidoctor-pdf
+
+# Setup
+WD=$(pwd)
+TWD=$(mktemp -d)
+
+INFILE=$(basename ${1?"Error: missing input file name"} )
+OUTFILE="${INFILE%.*}"
+
+echo "$INFILE, $OUTFILE"
+
+echo "Converting to asciidoc.."
+${SWG2MRKUP} convert -i "$1" -f "$TWD/$OUTFILE"
+echo
+
+echo "Converting to PDF.."
+"${ASCIIDOCPDF}" "$TWD/${OUTFILE}.adoc" -o "$WD/${OUTFILE}.pdf"
+echo
+
+echo "Done."
+# Clean up
+rm -r $TWD
+

docker/run-container.sh

+#!/bin/bash
+# Copyright ETSI 2017
+# See: https://forge.etsi.org/etsi-forge-copyright-statement.txt
+
+# Change this variable to true if you want
+# to copy the created yaml files to a web served directory
+#
+#
+WEB_PUBLISH=true
+
+f="${1:-`pwd`}"
+filter="$2"
+
+echo "Mounting dir $f as /work"
+
+if [ -n "$JOB_BASE_NAME" -a "$WEB_PUBLISH" = true ] ; then 
+  s="/var/www/html/api/nfv/$JOB_BASE_NAME/$BUILD_NUMBER"
+  mkdir -v -p "$s"
+  echo "Mounting dir $s as /storage"  
+  docker run -v "$f":/work -v "$s":/storage openapivalidator "/work" "/storage" "$filter"
+else
+  docker run -v "$f":/work openapivalidator "/work" "/storage" "$filter"
+fi 
+
+
+
+

docker/swg2mrkup

+#!/bin/bash
+#Copyright (c) ETSI 2017.
+
+# This software is subject to copyrights owned by ETSI. Non-exclusive permission 
+# is hereby granted, free of charge, to copy, reproduce and amend this file 
+# under the following conditions: It is provided "as is", without warranty of any 
+# kind, expressed or implied. 
+
+# ETSI shall never be liable for any claim, damages, or other liability arising 
+# from its use or inability of use.This permission does not apply to any documentation 
+# associated with this file for which ETSI keeps all rights reserved. The present 
+# copyright notice shall be included in all copies of whole or part of this 
+# software and shall not imply any sub-license right.
+#
+# Author: michele.carignani@etsi.org
+
+java -jar /swagger2markup-cli-1.3.2.jar $@

docker/validate-in-docker.sh

+#!/bin/bash
+# Copyright ETSI 2017
+# See: https://forge.etsi.org/etsi-forge-copyright-statement.txt
+
+# Merges OpenAPIs interfaces in the working directory
+# and validates them individually.
+# Exit status is 0 if all validation passed, 1 otherwise
+
+# Usage:
+# $0 <working-directory> <storage-dir>
+
+#set -x
+
+function store_api () {
+	f="$1"
+	cp -v "$1" "${storage_dir}/"
+}
+
+function validate_api () {
+  file="$1"
+  api="$2"
+  deliverable="$3"
+  echo "--- Merging file."
+  merged_file="../build/${deliverable}-${api}-API.yaml"
+  json_file="../build/${deliverable}-${api}-API.json"
+  json-refs resolve "${file}" > "${merged_file}"
+  yaml2json "${merged_file}" > "${json_file}"
+  
+  # Create the PDF version
+  echo "--- Create PDF..."
+  oas2pdf "${json_file}" 2>/dev/null 1>/dev/null 
+  mv "${deliverable}-${api}-API.pdf" "../build/"
+
+  echo "--- Validating ${merged_file}"
+  swagger-tools validate "${merged_file}"
+  vres=$?
+  echo "--- Validation done ($vres)."
+
+  # If validation succedes, store the generated file
+  [ $vres -a -d "/storage" ] && store_api "${merged_file}"
+  [ $vres -a -d "/storage" ] && store_api "${json_file}"
+  [ $vres -a -d "/storage" ] && store_api "../build/${deliverable}-${api}-API.pdf"
+  
+  
+  return $vres
+}
+
+# usage get_api_from_fn <file_name>
+#  e.g. get_api_from_fn /path/to/SOL003/Api1/Api1.yaml returns Api1
+function get_api_from_fn () {
+  echo "$(basename $(dirname $1))"
+}
+
+# usage get_api_from_fn <file_name>
+#  e.g. get_api_from_fn /path/to/SOL003/Api1/Api1.yaml returns Api1
+function get_deliverable_from_fn () {
+  echo "$(basename $(dirname $(dirname $1 )))"
+}
+
+## Main ##
+
+wd="${1?"Usage: $0 <working-directory> <storage-directory> [<filter_regex>]"}"
+storage_dir="${2?"Usage: $0 <working-directory> <storage-directory> [<filter_regex>]"}"
+
+filter="$3"
+echo "Using filter '$filter'"
+echo
+
+mkdir -p "$wd/build"
+
+echo "Entering dir $wd/src"
+cd "$wd/src"
+
+# Stores the overall validation result
+# (single results in OR)
+fres=0
+
+
+for f in $(find -name "*.yaml") ; do
+      # echo "Found yaml file: $f"
+      file=$(basename "$f")
+      api=$(get_api_from_fn $f)
+      deliverable=$(get_deliverable_from_fn $f)
+      if [ "$file" = "$api.yaml" ]; then
+          if [[ -n "$filter" &&  ! "$f" =~ ^[a-zA-Z0-9\.\/\-]*$filter[a-zA-Z\.\/0-9\-]*$ ]] ; then 
+            echo "Filtered out: $f (api: $api) (deliverable:$deliverable)."
+          else 
+            echo "-- Will validate: $f (api: $api) (deliverable:$deliverable)"         
+            validate_api "$f" "$api" "$deliverable"
+            res=$?
+            fres=$(($fres||$res))
+          fi
+      fi
+done
+
+chmod -R o+w "$wd/build"
+
+# Exit code needed for jenkins to know the verdict of the build
+echo "-- Final validator returns $fres."
+exit $fres

scripts/add_change_comment.py

+"""
+Add comments on gerrit
+"""
+
+import os
+import sys
+import jnk_grt_comment as jgc
+import glob
+
+CHANGE = os.environ.get('GERRIT_CHANGE_NUMBER') or None
+PATCH = os.environ.get('GERRIT_PATCHSET_NUMBER') or None
+URL = os.environ.get('BUILD_URL') or None
+
+# print glob.glob("**/*-API.yaml")
+
+if not (CHANGE and PATCH and URL):
+    print "No Gerrit change information found. Exiting"
+    sys.exit()
+
+print "START Comment on Gerrit change ---"
+
+jgc.comment_openapis_artifacts(CHANGE, PATCH, URL)
+
+print "DONE Comment on Gerrit change ---\n"

scripts/jnk_grt_comment.py

+#!/env/python2.7
+
+"""
+Comments on gerrit
+"""
+
+import os.path
+import subprocess
+import glob
+
+def mk_swg_ui_link(artifact, build_uri):
+    """
+      creates the link to the swagger ui
+    """
+    swg_ui = "https://forge.etsi.org/swagger/ui"
+    return swg_ui + "/?url=" + build_uri + "/artifact/"+ artifact +"/*view*/"
+
+
+def mk_swg_ui_comment(files, bu):
+    """
+      creates the link to the swagger ui
+    """
+    msg = "Review links for the built OpenAPIS:"
+    for artifact in glob.glob(files):
+        msg += "\n * " + os.path.basename(artifact) + ": " + mk_swg_ui_link(artifact, bu)
+    return msg
+
+def mk_grt_comment_cmd(change, payload):
+    """
+      creates the link to the swagger ui
+    """
+    command = "ssh -p 29418 forge.etsi.org \"gerrit review --message '"+payload+"'  "+change + "\""
+    print command
+    return command
+
+
+def send_gerrit_comment(change, patchset):
+    """
+      creates the link to the swagger ui
+    """
+    cmd = mk_grt_comment_cmd(change, patchset)
+    print "Executing: " + cmd
+    subprocess.call(cmd, shell=True)
+
+def jenkins_gerrit_comment(change_num, patch_num, comment):
+    send_gerrit_comment(change_num+","+patch_num, comment)
+
+def comment_openapis_artifacts(cn, pn, bu):
+    comment = mk_swg_ui_comment("**/*-API.yaml", bu)
+    # print "comment: "+comment
+    jenkins_gerrit_comment(str(cn), str(pn), comment)
\ No newline at end of file

src/SOL005/NSDManagement/NSDManagement.yaml

+swagger: "2.0"
+
+info:
+  version: "2.4.1"
+  title: DRAFT - SOL005 - NSD Management Interface
+  description: >
+    DRAFT - SOL005 - NSD Management Interface
+    IMPORTANT: Please note that this file might be not aligned to the current
+    version of the ETSI Group Specification it refers to and has not been
+    approved by the ETSI NFV ISG. In case of discrepancies the published ETSI
+    Group Specification takes precedence.
+    Please report bugs to https://forge.etsi.org/bugzilla/buglist.cgi?component=Nfv-Openapis
+  license:
+    name: "ETSI Forge copyright notice"
+    url: https://forge.etsi.org/etsi-forge-copyright-notice.txt
+  contact: 
+    name: "NFV-SOL WG"
+externalDocs:
+  description: ETSI GS NFV-SOL 005 V2.4.1
+  url: http://www.etsi.org/deliver/etsi_gs/NFV-SOL/001_099/005/02.04.01_60/gs_NFV-SOL005v020401p.pdf
+basePath: "/nsd/v1"
+
+schemes:
+  - https
+
+consumes:
+  - "application/json"
+produces:
+  - "application/json"
+
+paths:
+  /resource:
+    get:
+      responses:
+        200:
+          description: Success
\ No newline at end of file

src/SOL005/NSFaultManagement/NSFaultManagement.yaml

+swagger: "2.0"
+
+info:
+  version: "2.4.1"
+  title: DRAFT - SOL005 - NS Fault Management Interface
+  description: >
+    DRAFT - SOL005 - NS Fault Management Interface
+    IMPORTANT: Please note that this file might be not aligned to the current
+    version of the ETSI Group Specification it refers to and has not been
+    approved by the ETSI NFV ISG. In case of discrepancies the published ETSI
+    Group Specification takes precedence.
+    Please report bugs to https://forge.etsi.org/bugzilla/buglist.cgi?component=Nfv-Openapis
+  license:
+    name: "ETSI Forge copyright notice"
+    url: https://forge.etsi.org/etsi-forge-copyright-notice.txt
+  contact: 
+    name: "NFV-SOL WG"
+externalDocs:
+  description: ETSI GS NFV-SOL 005 V2.4.1
+  url: http://www.etsi.org/deliver/etsi_gs/NFV-SOL/001_099/005/02.04.01_60/gs_NFV-SOL005v020401p.pdf
+basePath: "/nsfm/v1"
+
+schemes:
+  - https
+
+consumes:
+  - "application/json"
+produces:
+  - "application/json"
+
+paths:
+  /resource:
+    get:
+      responses:
+        200:
+          description: Success
\ No newline at end of file

src/SOL005/NSLifecycleManagement/NSLifecycleManagement.yaml

+swagger: "2.0"
+
+info:
+  version: "2.4.1"
+  title: DRAFT - SOL005 - NS Lifecycle Management Interface
+  description: >
+    DRAFT - SOL005 - NS Lifecycle Management Interface
+    IMPORTANT: Please note that this file might be not aligned to the current
+    version of the ETSI Group Specification it refers to and has not been
+    approved by the ETSI NFV ISG. In case of discrepancies the published ETSI
+    Group Specification takes precedence.
+    Please report bugs to https://forge.etsi.org/bugzilla/buglist.cgi?component=Nfv-Openapis
+  license:
+    name: "ETSI Forge copyright notice"
+    url: https://forge.etsi.org/etsi-forge-copyright-notice.txt
+  contact: 
+    name: "NFV-SOL WG"
+externalDocs:
+  description: ETSI GS NFV-SOL 005 V2.4.1
+  url: http://www.etsi.org/deliver/etsi_gs/NFV-SOL/001_099/005/02.04.01_60/gs_NFV-SOL005v020401p.pdf
+basePath: "/nslcm/v1"
+
+schemes:
+  - https
+
+consumes:
+  - "application/json"
+produces:
+  - "application/json"
+
+paths:
+  /resource:
+    get:
+      responses:
+        200:
+          description: Success
\ No newline at end of file

src/SOL005/NSPerformanceManagement/NSperformanceManagement.yaml

+swagger: "2.0"
+
+info:
+  version: "2.4.1"
+  title: DRAFT - SOL005 - NS Performance Management Interface
+  description: >
+    DRAFT - SOL005 - NS Performance Management Interface
+    IMPORTANT: Please note that this file might be not aligned to the current
+    version of the ETSI Group Specification it refers to and has not been
+    approved by the ETSI NFV ISG. In case of discrepancies the published ETSI
+    Group Specification takes precedence.
+    Please report bugs to https://forge.etsi.org/bugzilla/buglist.cgi?component=Nfv-Openapis
+  license:
+    name: "ETSI Forge copyright notice"
+    url: https://forge.etsi.org/etsi-forge-copyright-notice.txt
+  contact: 
+    name: "NFV-SOL WG"
+externalDocs:
+  description: ETSI GS NFV-SOL 005 V2.4.1
+  url: http://www.etsi.org/deliver/etsi_gs/NFV-SOL/001_099/005/02.04.01_60/gs_NFV-SOL005v020401p.pdf
+basePath: "/nspm/v1"
+
+schemes:
+  - https
+
+consumes:
+  - "application/json"
+produces:
+  - "application/json"
+
+paths:
+  /resource:
+    get:
+      responses:
+        200:
+          description: Success
\ No newline at end of file

src/SOL005/VNFPackageManagement/VNFPackageManagement.yaml

+swagger: "2.0"
+
+info:
+  version: "2.4.1"
+  title: DRAFT - SOL005 - VNF Package Management Interface
+  description: >
+    DRAFT - SOL005 - VNF Package Management Interface
+    IMPORTANT: Please note that this file might be not aligned to the current
+    version of the ETSI Group Specification it refers to and has not been
+    approved by the ETSI NFV ISG. In case of discrepancies the published ETSI
+    Group Specification takes precedence.
+    Please report bugs to https://forge.etsi.org/bugzilla/buglist.cgi?component=Nfv-Openapis
+  license:
+    name: "ETSI Forge copyright notice"
+    url: https://forge.etsi.org/etsi-forge-copyright-notice.txt