Commit 59a251ae authored by Yann Garcia's avatar Yann Garcia
Browse files

Enhance learning phase

parent 3c179ff3
Loading
Loading
Loading
Loading
+270 −40
Original line number Diff line number Diff line
@@ -128,6 +128,7 @@ For the purposes of the present document, the [following] abbreviations [given i
`RSSI    Received Signal Strength Indicator`
`RTT     Round Trip Time`
`TC      Traffic Control`
`TOS     Traffic Order Service`
`TUN/TAP Network Tunneling Interfaces`
`UE      User Equipment`
`UML     Unified Modeling Language`
@@ -526,7 +527,7 @@ monaco-harbor/
├── outputs/                           iperf3 output files
├── results/                           Simulation results
├── sumo/                              SUMO support
│   ├── README.md                      Full SUMO usage guide
│   ├── README.md                      Full SUMO usage guide
│   ├── monaco-harbor.sumocfg          SUMO configuration
│   ├── monaco-harbor.net.xml          Real OSM network (1 796 edges)
│   ├── osm/
@@ -585,6 +586,10 @@ The NED file models the network topology:
- One NAT router;
- One external router/server side for emulation.

The figure below shows the network topology described by the NED file.

![Figure 5.3.2.4.2.3.2: Network topology described by the NED file](media/monaco-harbor-4.png)

The packets go through ExtLowerEthernetInterface, veth pairs, and a NAT router that rewrites addresses so external applications can talk through the simulated network. Here are the different parts of the complete MonacoHarbor.ned file that models the network topology:

1. The Imports sections from INET and Simu5G frameworks and the network declaration;
@@ -603,16 +608,30 @@ The figure below shows the network topology described by the NED file. The diffe

The IP plane is the following:

| Subnet | Mask | Role | Nodes |
|--------|------|------|-------|
| 10.0.0.0/24 | 255.255.255.0 | 5G Core backhaul + UE cellular | gNB .3, UPF .4, ue_car .1, ue_ped .2 |
| 10.0.1.0/24 | 255.255.255.0 | UPF ↔ Router (N9) | UPF .1, Router .2 |
| 10.2.0.0/16 | 255.255.0.0 | WiFi internal | RSU .1, wifi_srv .2 |
| 192.168.10.0/24 | 255.255.255.0 | veth car | sim-veth-car .1 ↔ veth-car .2 |
| 192.168.11.0/24 | 255.255.255.0 | veth pedestrian | sim-veth-ped .1 ↔ veth-ped .2 |
| 192.168.20.0/24 | 255.255.255.0 | veth 5G server | router ↔ ns_srv |
| 192.168.21.0/24 | 255.255.255.0 | veth WiFi server | wifi_srv ↔ ns_wifi |
| 192.168.30.0/24 | 255.255.255.0 | veth RSU | rsu ↔ ns_rsu |

| Node / Namespace | Interface | Address / 24 | Role |
|-----------------|-----------|--------------|------|
| `gnb` | cellular | 10.0.0.100 | NR radio (EN-DC secondary) |
| `gnb` | x2ppp0 | 10.0.5.1 | X2 → eNB |
| `enb` | cellular | 10.0.0.200 | LTE radio (EN-DC master) |
| `enb` | pppIf | 10.0.1.2 | N3 backhaul → UPF |
| `enb` | x2ppp0 | 10.0.5.2 | X2 ← gNB |
| `upf` | ppp0 | 10.0.1.1 | eNB-side N3 |
| `upf` | filterGate (pppIf) | 10.0.2.1 | Router-side N9 |
| `router` | ppp0 | 10.0.2.2 | UPF N9 |
| `router` | **ppp1** | **10.0.3.1** | **WiFi backhaul** |
| `router` | eth0 | 192.168.20.1 | veth → ns_srv |
| `ue_car` | cellular / eth0 | 10.0.0.1 / 192.168.10.1 | Car (EN-DC UE) |
| `ue_ped` | cellular / eth0 | 10.0.0.2 / 192.168.11.1 | Pedestrian (EN-DC UE) |
| `rsu` | wlan0 / eth0 | 10.2.0.1 / 192.168.30.1 | RSU |
| `wifi_srv` | wlan0 | 10.2.0.2 | WiFi internal |
| `wifi_srv` | **ppp0** | **10.0.3.2** | **Router backhaul** |
| `wifi_srv` | eth0 | 192.168.21.1 | veth → ns_wifi_srv |
| `ns_car` | veth-car | 192.168.10.2 | iperf3 client |
| `ns_ped` | veth-ped | 192.168.11.2 | iperf3 client |
| `ns_rsu` | veth-rsu | 192.168.30.2 | iperf3 client |
| `ns_srv` | veth-srv | 192.168.20.2 | iperf3 server (5G) || `ns_wifi_srv` | veth-wifi | 192.168.21.2 | iperf3 server (WiFi) |


The network path is the following:
- 5G/LTE path: ue_car + ue_ped  ──  NR  ──  gnb ── ppp ── upf ── ppp ── router
@@ -679,6 +698,29 @@ Because our goal is to demonstrate Simu5G/SUMO mobility around Monaco harbor, we
netconvert --osm-files osm/monaco-harbor.osm --output-file monaco-harbor.net.xml --geometry.remove --roundabouts.guess --tls.guess-signals --tls.discard-simple --tls.join --no-turnarounds --remove-edges.isolated --osm.sidewalks --ignore-errors
```

The GPS coordinates of the different nodes are described below:
Origin = **(maxlat, minlon)** = top-left corner of the OSM bounding box.

| Node | GPS (lat, lon) | XY (m) | Z (m) |
|------|---------------|--------|-------|
| `gnb` (NR secondary) | 43.7310°N 7.4228°E | (499.7, 338.6) | 30 |
| `enb` (LTE master) | 43.7315°N 7.4190°E | (193.0, 282.9) | 30 |
| `ue_car` start | 43.7320°N 7.4175°E | (73.4, 227.3) | 1.5 |
| `ue_ped` start | 43.7292°N 7.4183°E | (137.8, 539.0) | 1.5 |
| `rsu` / `ap` | 43.7304°N 7.4219°E | (427.4, 405.4) | 1.5 / 4 |
| `wifi_srv` | 43.7305°N 7.4221°E | (443.4, 394.3) | 4 |

**Note1:** The origin for the geometry conversion formula is the following: 
```
Origin = **(maxlat, minlon)** = top-left corner of the OSM bounding box.
```

**Note2:** The geometry conversion formula is the following:
```
X = (lon − 7.416587°) × cos(43.734042°) × 111320 m/deg   [east]
Y = (43.734042° − lat) × 111320 m/deg                     [south]
```

The result is shown below (netedit monaco-harbor.net.xml):

![Figure 5.3.2.4.2.1.2: OSM Web Wizard sript output](media/monaco-harbor-3.jpg)
@@ -741,12 +783,19 @@ The concern is that the WiFI attachment could not be done !?

###### 5.3.2.4.2.6 Round5: Adding LTE POA into the project

The purpose is to add a LTE POA into the project to support fallback from 5G to LTE. To verify that the 5G to LTE fallback mechanism is working as expected, the following procedures can be used:
1. Start the iperf3 server on the server side;
2. Start the iperf3 client on the UE-side;
3. Verify that the traffic is continuously flowing through the simulated network ofr a period of time long enough for the car to cover a complete circuit of the harbor;
4. Terminate the simulation;
5. Delete the veth.
The purpose is to add a LTE POA into the project to support fallback from 5G to LTE. To verify that the 5G to LTE fallback mechanism is working as expected, the previous procedure is enhance using TOS field. Three traffic classes (via TOS) are used:
- TOS < 10 → LTE master (eNB)
```bash
sudo ip netns exec ns_car iperf3 -c 192.168.20.2 -t 20 -i 1 -S 0x00
```
- 10 <= TOS < 20 → NR secondary (gNB-preferred)
```bash
sudo ip netns exec ns_car iperf3 -c 192.168.20.2 -t 20 -i 1 -S 0x0a
```
- TOS >= 20 → split bearer (NR+LTE; should gracefully continue on LTE if NR degrades)
```bash
sudo ip netns exec ns_car iperf3 -c 192.168.20.2 -t 20 -i 1 -S 0x28
```

###### 5.3.2.4.2.7 Round6: Conclusion

@@ -778,19 +827,9 @@ $ sumo-gui

**Note 2:** Do not forget to add SUMO environment variables to the shell profile.

**Note 3:** <mark>The geospatial coordinates conversions are the following:

- From the SUMO road network scenario to the Simu5G/SimuLTE simulation;
- From the Simu5G/SimuLTE simulation to the SUMO road network scenario;
- From the SUMO road network scenario to geo-coordinates;
- From the geo-coordinates to the SUMO road network scenario.
</mark>

**TODO**: To be refined

#### 5.3.2.5 Veins learning

#### 5.3.2.5.1 Veins architecture overview
##### 5.3.2.5.1 Veins architecture overview

The Veins framework includes a comprehensive suite of models to make vehicular network simulations as realistic as possible, without sacrificing speed. The GUI and IDE of OMNeT++ and SUMO can be used for quickly setting up and interactively running simulations.

@@ -798,13 +837,121 @@ The figure below shows the Veins architecture overview.

![Figure 5.3.2.5.1: Veins architecture overview](media/5.3.2.5.1.ViensArchitectureOverview.png)

#### 5.3.2.5.2 Enhance monaco-harbor project with Veins
##### 5.3.2.5.2 Enhance monaco-harbor project with Veins

The purpose is to enhance the monaco-harbor project with Veins.
The purpose is to enhance the monaco-harbor project with Veins support. Basically, the Veins framework is used to simulate the vehicular mobility and the handovers between the different network technologies. It provides a TraCI interface to communicate with the SUMO simulation and the OMNeT++ simulation.   

**TODO**
The steps to follow are:

1. Add the Veins framework to the project;
2. Configure the Veins framework;
3. Run the simulation;
4. Verify that the traffic is flowing through the simulated network;
5. Terminate the simulation;
6. Delete the veth.

###### 5.3.2.5.2.1 Round0: Describe the project

This project is the continuation of the monaco-harbor project ([clause 5.3.2.4.2](#53242-focused-on-vehicular-mobility-at-monaco)). The purpose is to enhance the project with Veins support: cars and pedestrians are moving on the road network and the handovers between the different network technologies are simulated.

###### 5.3.2.5.2.2 Round1: Add the Veins framework to the project

The main update is located in the following files:
- setenv-simu5g.sh: Adding veins environment variables;
- Makefile: Adding veins dependencies;
- MonacoVeinsInetMobility.ned file: declares the mobility module type attached to TraCI-spawned UEs;
- MonacoVeinsInetManager.ned file: declares the scenario manager module used in the network (wrappers for  C++ TraCI bridge);
- MonacoVeinsInetManager.hh/cc file: Adding veins header/code;
- MonacoVeinsInetMobility.hh/cc file: Adding veins code;
- omnetpp.ini file: Adding veins configuration;
- patch_veins_traci_api22.sh: Patching Veins to use TraCI API version 22.

####### 5.3.2.5.2.2.1 TraCI - INET bridge implementation: NED files

Two .ned files are required as wrappers for custom C++ TraCI bridge:

- MonacoVeinsInetManager.ned: declares the scenario manager module used in the network (veinsManager). The role of the MonacoVeinsInetManager.ned file is to define a module that extends Veins’ TraCIScenarioManagerLaunchd and points @class(...) to the C++ class monaco_harbor::MonacoVeinsInetManager;
- MonacoVeinsInetMobility.ned: declares the mobility module type attached to TraCI-spawned UEs. The role of the MonacoVeinsInetMobility.ned file is to define a mobility type extending INET MobilityBase, bound to the C++ class monaco_harbor::MonacoVeinsInetMobility.

####### 5.3.2.5.2.2.2 TraCI - INET bridge implementation: C++ code

The class MonacoVeinsInetManager.cc extends Veins’ launchd scenario manager behavior and is responsible for wiring TraCI vehicle lifecycle events into INET-compatible module initialization and motion updates.

The class MonacoVeinsInetMobility.cc is the mobility implementation used by ue_car[*] / ue_ped[*] to expose SUMO-driven motion as INET mobility state. It stores tores last known motion state:
- lastPosition;
- lastVelocity;
- lastOrientation.

###### 5.3.2.5.2.3 Round2: Execution and debug the simulation

The command to start the simulation using the Makefile is the following:
```bash
$ source ./scripts/setenv-simu5g.sh
$ clear && make clean && make && make run
```

The script 'scripts/launch_all.sh' can also be used to start the simulation:
```bash
$ ./scripts/launch_all.sh
```

The simulation is executed and the traffic is flowing through the simulated network. The following steps are used to verify the simulation:

1. Verify that the traffic is flowing through the simulated network;
2. Verify that the RTT is between 10 to 20 milliseconds;
3. Verify that the throughput is between 10 to 20 Mbps;
4. Verify that the packet loss is 0%;
5. Verify that the jitter is 0%.
**TODO: To be refined**

#### 5.3.2.5.3 Installation
The following commands can be used to debug the simulation:
- Start the simulation
```bash
$ sudo SIM_TIME_LIMIT=30s ./scripts/launch_all.sh
```
- Check the network interface configuration for all network interfaces (see [network topology](#532423-network-topology)). Here is an example with the car network interface:
```bash
$ sudo ip netns exec ns_car ip address
2: veth-car@if13: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 32:b6:18:1b:94:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.10.2/24 scope global veth-car
       valid_lft forever preferred_lft forever
    inet6 fe80::30b6:18ff:fe1b:94b7/64 scope link
       valid_lft forever preferred_lft forever
```

- Check router access
```bash
$ sudo ip netns exec ns_car ping 192.168.20.1
PING 192.168.20.1 (192.168.20.1) 56(84) bytes of data.
64 bytes from 192.168.20.1: icmp_seq=3 ttl=31 time=1124 ms
64 bytes from 192.168.20.1: icmp_seq=6 ttl=31 time=1124 ms
64 bytes from 192.168.20.1: icmp_seq=9 ttl=31 time=1135 ms
64 bytes from 192.168.20.1: icmp_seq=10 ttl=31 time=111 ms
64 bytes from 192.168.20.1: icmp_seq=14 ttl=31 time=1119 ms
64 bytes from 192.168.20.1: icmp_seq=17 ttl=31 time=1124 ms
64 bytes from 192.168.20.1: icmp_seq=19 ttl=31 time=1120 ms
```

- Check the route to the UE
```bash
$ sudo ip netns exec ns_car ip route get 192.168.10.2 from 192.168.20.2 iif veth-car
local 192.168.10.2 from 192.168.20.2 dev lo
    cache <local> iif veth-car
```

###### 5.3.2.5.2.4 Round3: Conclusion

Adding support of the Veins framework was the last step of the learning process. The project build reflect now the EdgeNative  Connector 4g-5g-macro-v2x network topology:
- 4G/LTE POA;
- 5G NR POA;
- WiFi POA;
- V2X POA;
- Cars and pedestrians mobile node;
- Road-side unit as a stationary node;
- Traffic flow through the network is simulated (e.g. using iperf3 as in AdvantEDGE demo1 and demo2).

##### 5.3.2.5.3 Installation

The procedure to install Veins is the following:
```bash
@@ -815,7 +962,34 @@ $ make
$ make install
```

#### 5.3.2.6 Gathering network information
Due to use of SUMO framework 1.25.0, TraCI API version 22 is added to Veins 5.3.1. This is done by the scripts/patch_veins_traci_api22.sh sript.

##### 5.3.2.5.4 Notes

**Note 1:** Use Veins online documentation to learn the basics of Veins.

**Note 2:** Do not forget to add Veins environment variables to the shell profile.

#### 5.3.2.6 Gathering positions of the UEs

The purpose is to implement the code to extract the positions of the UEs from the simulation. They is one alternative:

1. Develop an external application to extract the positions of the UEs from the simulation.
2. Extend the simulation code to extract the positions of the UEs, or

The first option is the most flexible and prevent the simulation to crash of the telemetry is crashing but it duplicates the subscritpion and is more complex to implement beacause of the libsumo library.

The second option is the simplest and most direct but it requires efficiency to prevent overloading ofthe simulation (Omnet++ is single thread and mono process). To prevent overloading, the following constraints apply:

- Use UDP instead of TCP even if there is a risk of message loss;
- Use non blocking sockets to avoid blocking the simulation;
- Conversion from X/Y/Z coordinates to geo-coordinates will be done on the server side to reduce the time to generate each UDP message.

<mark>
There is one issue to consider: the zone concept is not supported by the Simu5G/SimuLTE emulation. The best way to handle this is to let the EdgeNative Connector to handle the zone concept when receiving the UEs location from the simulation.
</mark>

#### 5.3.2.7 Gathering network information

The purpose is to gather the network information from the simulation. The network information element are the one described by the ETSI MEC 012 [\[i.5\]](#_ref_i.5) and ETSI MEC 013 [\[i.6\]](#_ref_i.6):

@@ -823,20 +997,65 @@ The purpose is to gather the network information from the simulation. The networ
- RAB information;
- L2 measurements.

#### 5.3.2.6.1 Gathering S1 bearer information
The retained method here is to provide RNI datas on request from the EdgeNative Connector. The following constraints still apply:

- Use UDP instead of TCP even if there is a risk of message loss;
- Use non blocking sockets to avoid blocking the simulation;
- The response message is a binary packed message containing the RNI datas.

#### 5.3.2.8 Gathering network characteristics

**TODO**: To be refined

#### 5.3.2.9 Conclusions

#### 5.3.2.7 Gathering positions of the UE
The completion of the learning process comes with a first demo project to verify the network emulator is working as expected.
The demo project contains:

**TODO**: To be refined
In addition, the project provides UEs location sent periodically to a peer application (e.g. the EdgeNative Connector) and network information on request from a peer application (e.g. the EdgeNative Connector).

The following network information is provided:

#### 5.3.2.8 Gathering network characteristics
- S1 bearer information;
- RAB information;
- L2 measurements.

**TODO**: To be refined
The following network characteristics are measured:

- Throughput;
- Latency;
- Jitter;
- Packet loss.

During this learning process, the following topics shall be concidered:

- Convert an EdgeNative Connector network scenarion into NED files and INI files;
- Convert an EdgeNative Connector network scenarion UEs paths into SUMO files;
- Generate the code project structure and the Makefile;

To execute the demo project, the following command can be used:
```bash	
$ clear && make teardown ; make bin/libmonaco-harbor.so && SIM_TIME_LIMIT=60s make run | tee run.log
```

The script 'scripts/launch_all.sh' can also be used to start the simulation plus the real application at the UE side (iperf3) and the server side (iperf3):
```bash
$ ./scripts/launch_all.sh
```

To verify that the traffic is flowing through the simulated network:
1. Verify that the RTT is between 10 to 20 milliseconds;
```bash
$ sudo ip netns exec ns_car ping 192.168.20.1
```
2. Verify that the throughput is between 10 to 20 Mbps;
**TODO: To be refined**
3. Verify that the packet loss is 0%;
```bash
$ sudo ip netns exec ns_car ping 192.168.20.1
```
4. Verify that the jitter is 0%.
**TODO: To be refined**

## 5.4 Round 1: Architecture and Spikes

@@ -881,11 +1100,22 @@ Based on the previous round, the following tasks are proposed for this round:

**TODO**

## 5.9 Round 6: Use demo1/demo2 based of iperf3 to verify the network characteristcs measurements and the values exposed by MEC RNIS and Location services
## 5.9 Round 6: EdgeNative Connector network scenarion conversion

The purpose of this round is to convert the EdgeNative Connector network scenarion into a complete project with NED files, INI files and SUMO files.
The following tasks are proposed:

- Task 1: Convert the EdgeNative Connector network scenarion into NED files and INI files;
- Task 2: Convert the EdgeNative Connector network scenarion UEs paths into SUMO files;
- Task 3: Generate the code project structure and the Makefile;

**TODO**

## 5.10 Round 7: Use demo1/demo2 based of iperf3 to verify the network characteristcs measurements and the values exposed by MEC RNIS and Location services

**TODO**

## 5.10 Round 7: Verify that all demo applications and tutorials are working with the new network simulator
## 5.11 Round 8: Verify that all demo applications and tutorials are working with the new network simulator

**TODO**

+247 KiB (412 KiB)
Loading image diff...
+462 KiB
Loading image diff...