proto3-gen.md

# Protobuf Schema Generation

[OpenAPI Generator](https://openapi-generator.tech) is used to generate protobuf schema (`.proto3`) files from OpenAPI specifications of MEC013 Locatio API.

>**NOTE:** At the time of writing, the tool does not support OAS 3.1 version and we have to first convert the [Locatio API](./LocationAPI.yaml) to OAS 3.0 for generating protobuf schema.

1. Convert OAS for [Locatio API](./LocationAPI.yaml) from 3.1 to 3.0​

   - Change the value of `openapi` field from 3.1.0 to 3.0.0​

   - Use this [VS code extension](https://marketplace.visualstudio.com/items?itemName=42Crunch.vscode-openapi) to see the errors in the downgraded YAML (v3.0)​

   - Manually fix the errors​
     - mostly related to `examples` <--> `example` interchange​
     - or some 3.1 fields that are not supported in 3.0​ (comment them out)

2. Generate proto files
   - Install the `openapi-generator-cli.jar` using the installation procedure mentioned [here](https://openapi-generator.tech/docs/installation#jar).
   - Generate the proto files using the following command 
        ```sh
        $ java -jar openapi-generator-cli.jar generate -i LocationAPI.yaml -g protobuf-schema -o proto3/ --package-name mec013
        ```

3. Carefully inspect the generated `.proto` files for any inconsistencies. Some of the things to look out for:
    - Proto3 generated files for enumerations, structures containing allOf, oneOf, anyOf etc. may need to be touched manually
    - Check that all the nested models are being _imported_ correctly in their parent models
    - Remove redundant proto files


4. Validate protobuf schema by generating code from proto3 descriptions in different languages. See [this section](#code-generation-from-proto3) for more details.

# Code Generation from proto3

Below are some code generation examples for Python, Go and Ruby. For other languages, please refer to https://grpc.io/docs/quickstart/.

### Python

1. Install the grpcio-tools package
    ```sh
    $ pip install grpcio-tools
    ```

2. Create a directory for generated Python stubs
    ```sh
    $ mkdir python-stubs 
    ```

3. Run the following commands from the root of the directory containing this README that you are reading.

   - Models:
  
        ```sh
        $ python -m grpc_tools.protoc -I./proto3 --python_out=./python-stubs ./proto3/models/*
        ```
    
        The above command will generate .py files for all the data models in the ./models directory

   - Services:

        ```sh
        $ python -m grpc_tools.protoc -I./proto3 --python_out=./python-stubs --grpc_python_out=./python-stubs ./proto3/services/location_service.proto
        ```

        The above command will generate two files for the Location service:
        - _location_service_pb2.py_: containing the python data models used in the Location service file
        - _location_service_pb2_grpc.py_: containing all the classes and functions needed for the supported HTTP methods in the Location API

### Go

1. Install protocol buffer compiler
    ```sh
    $ apt install -y protobuf-compiler
    ```
2. Install Go plugins for `protoc`
    ```sh
    $ go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.28.1
    ```
    ```sh
    $ go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.2.0
    ```
3. Update `PATH` so `protoc` can find the plugins
    ```sh
    $ export PATH="$PATH:$(go env GOPATH)/bin"
    ```
4. Define a go package by appending `option go_package = "./mec013.services.locationservice";` in all .proto files like this:

    ```Go
    ...

    syntax = "proto3";

    package mec013.services.locationservice;

    option go_package = "./mec013.services.locationservice";

    import public "models/<xyz>.proto";

    ...
    ```
5. Generate Go code for models and services
    ```sh 
    $ mkdir go-stubs
    $ protoc --go_out=./go-stubs ./proto3/models/* -I./proto3
    $ protoc --go_out=./go-stubs ./proto3/services/* --go-grpc_out=go-stubs -I./proto3
    ```
    > The generated `<data_model>.pb.go` files will contain all the protocol buffer code to populate, serialize, and retrieve request and response message types defined in the `models` folder.
    > And the `location_service_grpc.pb.go` will contain the stubs for the methods defined in the `location_service.proto` file.

### Ruby

1. Install gRPC Ruby Plugin and required tools
   ```sh
   $ gem install grpc
   $ sudo apt install ruby-grpc-tools
   ```

2. Generate code
    ```sh
    $ mkdir ruby-stubs
    ```
    
    Run the following command to create Ruby modules for all the data models defined in the proto files.

    ```sh
    $ grpc_tools_ruby_protoc -I./proto3 --ruby_out=ruby-stubs ./proto3/models/*
    ```
    Run the following command to generate `location_service_pb.rb` and `location_service_services_pb.rb` files, containing stub and service classes for the endpoints and methods defined in MEC013 Location service.

    ```sh
    $ grpc_tools_ruby_protoc -I./proto3 --ruby_out=ruby-stubs --grpc_out=ruby-stubs ./proto3/services/*
    ```