Commit 0d252ad1 authored by Mark Canterbury's avatar Mark Canterbury
Browse files

Attempting to include generation in CI/CD

parent baa3aa92

.gitlab-ci.yml

+1 −0
Original line number Original line Diff line number Diff line
@@ -137,6 +137,7 @@ generate_artefacts: 
    - echo $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME

    - echo $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME

    - echo $ARTEFACT_NAME

    - echo $ARTEFACT_NAME

    - forgelib-changedocs -x asn -x asn1 -x xsd -x xml -x json -v -d -c -l --startdelimiter coversheets/delimiter_start.docx --enddelimiter coversheets/delimiter_end.docx https://$CI_SERVER_HOST/rep $CI_PROJECT_ID $CI_MERGE_REQUEST_IID

    - forgelib-changedocs -x asn -x asn1 -x xsd -x xml -x json -v -d -c -l --startdelimiter coversheets/delimiter_start.docx --enddelimiter coversheets/delimiter_end.docx https://$CI_SERVER_HOST/rep $CI_PROJECT_ID $CI_MERGE_REQUEST_IID

    - 104196/publish.sh v0.0.1 104196/ts_spec_template.docx 104196 ts_cover_skeleton.docx ts

  artifacts:

  artifacts:

    untracked: true

    untracked: true

    paths:

    paths:

104196/.gitignore

0 → 100644
+1 −0
Original line number Original line Diff line number Diff line 
baseline
 
 No newline at end of file

104196/104196.md

+263 −263
Original line number Original line Diff line number Diff line
@@ -196,15 +196,15 @@ It is recommended that the following parts of an ETSI Technical Specification ar 




1. Fundamental model. The ETSI model has a clear boundary between what is AO-managed and what is provider-managed. There are strong reasons to have clear boundaries of responsibility: it is fundamental to many legal and policy requirements in various jurisdictions. 

1. Fundamental model. The ETSI model has a clear boundary between what is AO-managed and what is provider-managed. There are strong reasons to have clear boundaries of responsibility: it is fundamental to many legal and policy requirements in various jurisdictions. 





1. Definitions: explaining that terms from the relevant specifications (e.g. ETSI TS 103 120 [i.2], ETSI TS 103 280 [i.4] and ETSI TS 103 976 [i.1]) should be used where suitable.

2. Definitions: explaining that terms from the relevant specifications (e.g. ETSI TS 103 120 [i.2], ETSI TS 103 280 [i.4] and ETSI TS 103 976 [i.1]) should be used where suitable.





1. Identifiers and house-keeping. This would include using provider and AO identifiers. 

3. Identifiers and house-keeping. This would include using provider and AO identifiers. 





1. Request types: For certain situations (e.g. vehicles), there is a clear set of request types (see ETSI TS 103 976 [i.1]). It is recommended that these are used where they are suitable. 

4. Request types: For certain situations (e.g. vehicles), there is a clear set of request types (see ETSI TS 103 976 [i.1]). It is recommended that these are used where they are suitable. 





1. Results: Where a standardised request type has been used (see item D), it is recommended to use the corresponding response structures (e.g. see ETSI TS 103 976 [i.1]). F)	Workflow: it is recommended to follow the Workflow steps from an ETSI TS (mostly this is defined in ETSI TS 103 120 [i.2] e.g. see the Simple Workflow in Annex H.2). This gives a structure for when results are delivered and when error responses are sent etc.

5. Results: Where a standardised request type has been used (see item D), it is recommended to use the corresponding response structures (e.g. see ETSI TS 103 976 [i.1]). F)	Workflow: it is recommended to follow the Workflow steps from an ETSI TS (mostly this is defined in ETSI TS 103 120 [i.2] e.g. see the Simple Workflow in Annex H.2). This gives a structure for when results are delivered and when error responses are sent etc.





1. Look at ways that ETSI TC LI data structures (ETSI TS 103 705 [i.5]) can be used to help portal design.  

6. Look at ways that ETSI TC LI data structures (ETSI TS 103 705 [i.5]) can be used to help portal design.  









User interface design can help support the above points. For example, it would be helpful for a user interface to guide an AO user through the ETSI-defined fields, enforcing the strongly-typed definitions from the very start of the process and using ETSI-defined terminology. It would be helpful for the user interface to guide users through the ETSI-defined workflow.

User interface design can help support the above points. For example, it would be helpful for a user interface to guide an AO user through the ETSI-defined fields, enforcing the strongly-typed definitions from the very start of the process and using ETSI-defined terminology. It would be helpful for the user interface to guide users through the ETSI-defined workflow.

104196/build.bat

+10 −0
Original line number Original line Diff line number Diff line 
docker run --rm -v ./:/tmp/ -w "/tmp" "forge.etsi.org:5050/cti/md-specs-dev/tools/markdowntools:master" processMDSpec -fmo "%1" -fmf frontmatter.md

docker run --rm -v ./:/tmp/ -w "/tmp" "forge.etsi.org:5050/cti/md-specs-dev/tools/markdowntools:master" processMDSpec "%1" > combined.md





docker run --rm -v ./:/tmp/^

docker run --rm -v ./:/tmp/^

 "forge.etsi.org:5050/cti/md-specs-dev/tools/generatebaseline:master"^

 "forge.etsi.org:5050/cti/md-specs-dev/tools/generatebaseline:master"^

 pandocFilter -ts -fp -o "./" "./tmp/%1"

 pandocFilter -ts -fp -o "./" "./tmp/%1"
@@ -13,3 +17,9 @@ docker run --rm -v ./:/data^ 




docker run --rm -v ./:/tmp/ -w "/tmp" "forge.etsi.org:5050/cti/md-specs-dev/tools/generatebaseline:master" update_toc "baseline/ts_104196.docx" "baseline/ts_104196.docx"

docker run --rm -v ./:/tmp/ -w "/tmp" "forge.etsi.org:5050/cti/md-specs-dev/tools/generatebaseline:master" update_toc "baseline/ts_104196.docx" "baseline/ts_104196.docx"

docker run --rm -v ./:/tmp/ -w "/tmp" "forge.etsi.org:5050/cti/md-specs-dev/tools/generatebaseline:master" update_format_styles "baseline/ts_104196.docx" "baseline/ts_104196.docx"

docker run --rm -v ./:/tmp/ -w "/tmp" "forge.etsi.org:5050/cti/md-specs-dev/tools/generatebaseline:master" update_format_styles "baseline/ts_104196.docx" "baseline/ts_104196.docx"



docker container run --rm -v ./:/tmp/ -w "/tmp" "forge.3gpp.org:5050/tools/3gpp-scripts/forgelib:v2.28.0" forgelib-baseline "baseline/ts_cover_skeleton.docx" frontmatter.md "baseline/ts_104196.docx"



del combined.md

del frontmatter.md

del mermaid-filter.err
 
 No newline at end of file

104196/publish.sh

0 → 100644
+98 −0
Original line number Original line Diff line number Diff line 
#

#	publish_specs.sh

#

#	Script to generate documents from markdown files with Pandoc. Outputs:

#		- .docx

#		- .pdf

#		- .epub 

#

#	(c) 2024 by Miguel Angel Reina Ortega

#	License: BSD 3-Clause License. See the LICENSE file for further details.

#

#!/bin/bash



PANDOC_DOCKER_IMAGE=forge.etsi.org:5050/cti/md-specs-dev/tools/generatebaseline/pandoc:master

GENERATE_BASELINE_DOCKER_IMAGE=forge.etsi.org:5050/cti/md-specs-dev/tools/generatebaseline:master

MARKDOWN_TOOLS_DOCKER_IMAGE=forge.etsi.org:5050/cti/md-specs-dev/tools/markdowntools:master

FORGELIB_DOCKER_IMAGE=forge.3gpp.org:5050/tools/3gpp-scripts/forgelib:v2.28.0



echo "\n------ Checking for docker image --------"

docker logout $(echo "$PANDOC_DOCKER_IMAGE" | cut -d "/" -f 1) 

docker pull "$PANDOC_DOCKER_IMAGE"

docker logout $(echo "$GENERATE_BASELINE_DOCKER_IMAGE" | cut -d "/" -f 1)

docker pull "$GENERATE_BASELINE_DOCKER_IMAGE"

docker logout $(echo "$MARKDOWN_TOOLS_DOCKER_IMAGE" | cut -d "/" -f 1)

docker pull "$MARKDOWN_TOOLS_DOCKER_IMAGE"

docker logout $(echo "$FORGELIB_DOCKER_IMAGE" | cut -d "/" -f 1)

docker pull "$FORGELIB_DOCKER_IMAGE"



echo "------ Removing previous outputs --------"

rm -f baseline/*.docx 2>/dev/null || true



echo "------ Parsing repo URL --------"



TAG_NAME=$1

echo "TAG NAME:" $TAG_NAME

SPEC_TEMPLATE=$2

echo "SPEC TEMPLATE:" $SPEC_TEMPLATE 

SPEC_NAME=$3

echo "SPEC NAME:" $SPEC_NAME

ETSI_COVER_TEMPLATE=$4

echo "SPEC COVER TEMPLATE:" $ETSI_COVER_TEMPLATE

CONTRIBUTION_TYPE=$5

echo "CONTRIBUTION TYPE:" $CONTRIBUTION_TYPE



echo "------ Getting .md file(s) ------"

# If there are no .md files, then simply exit

ls | grep -q 'md'

specs=$?

if [ ! $specs ] ; then 

    echo "-- No Markdown files."

    exit 0

fi



for i in $(find -name "*.svg") ; do

  echo "\n------ Converting SVG to PNG for pandoc --------"

	png="${i%.svg}.png"

    if [ -f "$png" ]; then

      echo "Skipping $i — PNG already exists at $png"

      continue

    fi

done



for i in *.md ;  do

    if [ $i != 'README.md' ]; then

    	if [[ $i =~ (GS|GR|TS|TR|EN|EG).*\.md ]] ; then

        	echo "\n------ Processing MD file to combine all clauses (::include) -------"

            docker run --rm -v $(pwd):/tmp/ -u $(id -u):$(id -g) -w "/tmp" "$MARKDOWN_TOOLS_DOCKER_IMAGE" processMDSpec -fmo "$i" -fmf frontmatter.md

            docker run --rm -v $(pwd):/tmp/ -u $(id -u):$(id -g) -w "/tmp" "$MARKDOWN_TOOLS_DOCKER_IMAGE" processMDSpec "$i" > combined.md

            cat combined.md > $i

			echo "\n------ Adding TOC to spec --------"

	      	#docker run --rm -v $(pwd):/tmp/ -u $(id -u):$(id -g) "$GENERATE_BASELINE_DOCKER_IMAGE" generateTOC --add-content "/tmp/$i"

	  		echo "\n------ Preparing spec --------"

	    	docker run --rm -v $(pwd):/tmp/ -u $(id -u):$(id -g) "$GENERATE_BASELINE_DOCKER_IMAGE" pandocFilter -ts -fp -o "/tmp/" "/tmp/$i"

			echo "\n------ Publishing spec --------"

			mkdir baseline

	    	# Word output

	    	docker run --rm -v $(pwd):/data -u $(id -u):$(id -g) -w "/data" "$PANDOC_DOCKER_IMAGE" "/data/$i" --toc --metadata toc-title="Contents" -F mermaid-filter -f markdown+escaped_line_breaks -t docx --reference-doc "/data/${SPEC_TEMPLATE}" -o "baseline/${SPEC_NAME}_${TAG_NAME}.docx"

	    	#echo 'docker run --rm -v $(pwd):/data -u $(id -u):$(id -g) "$DOCKER_IMAGE" "/data/$i" -F mermaid-filter -f markdown+escaped_line_breaks -t pdf -o "${4}_${2}.pdf"'

	    	if [ -f "frontmatter.md" ] && [ "$(cat frontmatter.md | tr -d ' \n\t')" != "{}" ]; then

	        	docker container run --rm -v $(pwd):/tmp/ -u $(id -u):$(id -g) -w "/tmp" "$FORGELIB_DOCKER_IMAGE" forgelib-baseline ${ETSI_COVER_TEMPLATE} frontmatter.md "baseline/${SPEC_NAME}_${TAG_NAME}.docx"

	    	fi

			#if [ "$6" != "onlyDocx" ]; then

				#Do not convert to PDF or EPUB until a solution is found to include the spec cover pages

				# PDF Output

				#docker run --rm -v $(pwd):/data -u $(id -u):$(id -g) -w "/data" "$PANDOC_DOCKER_IMAGE" pandoc "/data/$i" -F mermaid-filter -f markdown+escaped_line_breaks -t pdf -o "baseline/${4}_${2}.pdf"

				# EPUB Output

				#docker run --rm -v $(pwd):/data -u $(id -u):$(id -g) -w "/data" "$PANDOC_DOCKER_IMAGE" pandoc "/data/$i" -f markdown+escaped_line_breaks -t epub3 -o "baseline/${4}_${2}.epub" --metadata title="${PROJECT_NAME}_${2}" --metadata creator="oneM2M Partnership Project" --metadata rights="Copyright 2024 oneM2M Partners

	#hip Project"

			#fi

			echo "\n------ Postprocessing spec --------"

			docker run --rm -v $(pwd):/tmp/ -u $(id -u):$(id -g) -w "/tmp" "$GENERATE_BASELINE_DOCKER_IMAGE" update_toc "baseline/${SPEC_NAME}_${TAG_NAME}.docx" "baseline/${SPEC_NAME}_${TAG_NAME}.docx"

			docker run --rm -v $(pwd):/tmp/ -u $(id -u):$(id -g) -w "/tmp" "$GENERATE_BASELINE_DOCKER_IMAGE" update_format_styles "baseline/${SPEC_NAME}_${TAG_NAME}.docx" "baseline/${SPEC_NAME}_${TAG_NAME}.docx"

		fi

	fi

done





exit 0