(WIP) Docker compose quality of life improvements and/or cleanup before 0.1.0 release

.gitmodules

@@ -1,3 +1,3 @@
 [submodule "cdsp/automotive-viss2"]
-	path = cdsp/automotive-viss2
-	url = https://github.com/w3c/automotive-viss2.git
+	path = cdsp/vissr
+	url = https://github.com/COVESA/vissr.git

docker/README.md

@@ -1,20 +1,20 @@
 This directory contains files related to the Docker deployment of the Central Data Service Playground.
 
 # Central Data Service Playground
-NOTE: The documentation below is a work in progress as we work on the Phase 1 PoC. The intention is to help but it may be incomplete.
+The compose file `docker-compose-cdsp.yml` provides a containerized deployment of the playground using docker.
 
-FIXME: Document deployment and the basic result for each backend. Suggest following simple template set by https://github.com/docker/awesome-compose as a starting point.
+## VISSR docker image build setup
+The [VISSR VISS Data Server](https://github.com/COVESA/vissr) has no pre-built image in a docker image repository and must therefore be built. Whilst the upstream documentation for VISSR is considered the reference documentation for build environment setup this section collects information that we have needed for a successful build of master branch commit 27d6de0.
 
-## WAII docker build setup
-The [WAII VISS Data Server](https://github.com/w3c/automotive-viss2) has no pre-built image in a docker image repository and must therefore be built. Whilst the upstream documentation for WAII is considered the reference documentation for build environment setup this section collects information that we have needed for a successful build of master branch commit bd92da77.
+NOTE: This project intends to work towards the availability of a pre-built VISSR docker image so that the playground docker deployment can be a simple download of images.
 
 ### Install golang
-The WAII [build tutorial](https://w3c.github.io/automotive-viss2/build-system/) says to install golang version 1.13 or later.
+The VISSR [build tutorial](https://covesa.github.io/vissr/build-system/) says to install golang version 1.13 or later.
 Go install instructions can be found here: https://go.dev/learn/
 
 Depending on your distro you may need to setup `GOROOT` and `GOPATH`. This was not required on Ubuntu 20 LTS, but was on Mac.
 
-### Create persistent local volume /tmp/docker
+### Create persistent local volume `/tmp/docker`
 The upstream Docker compose assumes the existence of the local directory `/tmp/docker` but does not create it. Fix the issue by creating it yourself:
 
 ```
@@ -23,46 +23,102 @@ $ mkdir /tmp/docker
 See upstream issue report https://github.com/w3c/automotive-viss2/issues/99 for details.
 
 ### Generate credentials (testCredGen build error)
-The upstream Dockerfile assumes that credentials have already been created and will fail to build if they are not found. Generate them by running `./testCredGen.sh ca` from the directory `/cdsp/automotive-viss2/testCredGenRun`
+The upstream Dockerfile assumes that credentials have already been created and will fail to build if they are not found. Generate them by running `./testCredGen.sh ca` from the directory `/cdsp/vissr/testCredGenRun`
 
 See upstream issue report https://github.com/w3c/automotive-viss2/issues/86 for details.
 
 ### Disable Access Grant support (agt_public_key.rsa build error)
-There is a current issue with the upstream WAII VISS Server Dockerfile in which the server fails to build due to a missing public key for the Access Control server. See upstream issue report https://github.com/w3c/automotive-viss2/issues/88 for details. After discussion with the upstream maintainers the current workaround is to comment out the relevant following line from the end of the Dockerfile. The change should be made to `cdsp/cdsp/automotive-viss2/Dockerfile`
+There is a current issue with the upstream VISSR VISS Server Dockerfile in which the server fails to build due to a missing public key for the Access Control server. See upstream issue report https://github.com/w3c/automotive-viss2/issues/88 for details. After discussion with the upstream maintainers the current workaround is to comment out the relevant following line from the end of the `vissv2server` section of the Dockerfile. The change should be made to `cdsp/cdsp/vissr/Dockerfile.rlserver`
 ```
 #COPY --from=builder /build/server/agt_server/agt_public_key.rsa .
 ```
 
-If your project requires Access Grant support please discuss enabling it with the WAII community.
+If your project requires Access Grant support please discuss enabling it with the VISSR community.
 
-### Mac build error "ERROR [internal] load metadata for docker.io"
+### Generate `vss_vissv2.binary`
+The VISSR server component requires a file called `vss_vissv2.binary` to understand the VSS tree it must work with. Unfortunately, VISSR provides no default file and you must therefore generate it yourself.
+
+Instructions for doing that can be found in the VISSR documentation site [here](https://covesa.github.io/vissr/server/#vss-tree-configuration)
+
+Tip: The playground maintainers have found that the method involving running `make binary` in a git clone of the VSS source tree to generate the file is straight forward. Note: check the VSS readme for the python requirements for the tooling.
+
+### Tip: Corporate CA security (download error "tls: failed to verify certificate:")
+If you are working behind a corporate security system that places a _man-in-the-middle_ between your host and the internet you may see security errors when artifacts are downloaded as part of the build process.
+
+For example:
+
+```
+3.254 client/client-1.0/simple_client.go:27:2: github.com/akamensky/[email protected]: Get "https://proxy.golang.org/github.com/akamensky/argparse/@v/v1.4.0.zip": tls: failed to verify certificate: x509: certificate signed by unknown authority
+```
+This is an attribute of docker, rather than the playground and results when the CA root certificate for the security system is not known.
+
+A solution is to add the CA root certificate to the VISSR Dockerfile which will update the CA store as part of the build and allow the downloads to proceed successfully.
+
+The VISSR Dockerfile already includes an example of this for a Cisco system which uses the CA root certificate `cisco.crt`.
+
+From `cdsp/vissr/Dockerfile.rlserver`:
+```
+#corporate proxy settings can sometimes cause tls verification error. Add root crt to docker container.
+COPY testCredGen/cicso-umbrella/cisco.crt /usr/local/share/ca-certificates/cisco.crt
+RUN update-ca-certificates
+```
+To add your own:
+1. Ask your IT for the CA root certificate for the security system they use.
+2. Copy the `.crt` file into `cdsp/vissr/testCredGen`
+3. Add your own line to the Dockerfile to copy the certificate so that it is included when `update-ca-certificates` is run. As shown below.
+
+```
+COPY testCredGen/cicso-umbrella/cisco.crt /usr/local/share/ca-certificates/cisco.crt
+COPY testCredGen/<my company .crt>  /usr/local/share/
+RUN update-ca-certificates
+```
+
+### Tip: Mac build error "ERROR [internal] load metadata for docker.io"
 On a Mac build errors related to docker metadata such as `ERROR [internal] load metadata for docker.io/library/golang` have been observed. This [serverfault article](https://serverfault.com/a/1131599) suggests commenting the line `"credsStore": "desktop"` from the Docker `config.json` for your user. This was found to work.
 
-## With Apache IoTDB data store backend
-### Deploy with Docker Compose
+### Tip: I am blocked by VISSR build issues or do not want to build
+You can start working with the playground by starting the deployment with just the Apache IoTDB server which has a pre-built docker image. Its extensive feature set will let you begin experimenting with timeseries data.
+
+In parallel you can work through any VISSR build issues to add VISS support northbound and the other features it supports.
+
+Deploy (start) just IoTDB:
+```
+$ sudo docker compose -f docker-compose-cdsp.yml up -d iotdb-service
+[+] Running 1/0
+ ✔ Container iotdb-service  Running                 0.0s
+```
+
+## Deploy with Docker Compose
+### Start/stop containers
 Start the containers:
 ```
-$ sudo docker compose -f docker-compose-waii-iotdb.yml up -d
-[+] Running 5/5
- ✔ Network cdsp-waii-iotdb_default   Created                                                                                                             0.1s
- ✔ Container iotdb-service           Started                                                                                                             0.1s
- ✔ Container waii_container_volumes  Started                                                                                                             0.1s
- ✔ Container vissv2server            Started                                                                                                             0.1s
- ✔ Container app_redis               Started
+$ sudo docker compose -f docker-compose-cdsp.yml up -d
+[+] Running 4/5
+ ⠴ Network cdsp_default               Created                 1.5s
+ ✔ Container vissr_container_volumes  Started                 0.5s
+ ✔ Container iotdb-service            Started                 0.7s
+ ✔ Container app_redis                Started                 1.0s
+ ✔ Container vissv2server             Started                 1.3s
 ```
 
 Stop and remove the containers:
 ```
-$ sudo docker compose -f docker-compose-waii-iotdb.yml down
+$ sudo docker compose -f docker-compose-cdsp.yml down
+[+] Running 5/5
+ ✔ Container vissv2server             Removed                 0.0s
+ ✔ Container app_redis                Removed                 0.3s
+ ✔ Container iotdb-service            Removed                 2.2s
+ ✔ Container vissr_container_volumes  Removed                 0.0s
+ ✔ Network cdsp_default               Removed                 0.1s
 ```
 ### Expected Result
 Listing should show three running containers as shown below:
 ```
 $ sudo docker ps
-CONTAINER ID   IMAGE                           COMMAND                  CREATED          STATUS          PORTS                                                          NAMES
-72d793c9619c   cdsp-waii-iotdb-vissv2server    "/app/vissv2server -…"   21 minutes ago   Up 21 minutes   127.0.0.1:8081->8081/tcp, 127.0.0.1:8887-8888->8887-8888/tcp   vissv2server
-b9418de22ea5   redis                           "docker-entrypoint.s…"   21 minutes ago   Up 21 minutes   6379/tcp                                                       app_redis
-24858f79203b   apache/iotdb:1.2.2-standalone   "/usr/bin/dumb-init …"   21 minutes ago   Up 21 minutes   0.0.0.0:6667->6667/tcp, :::6667->6667/tcp                      iotdb-service
+NAME            IMAGE                           COMMAND                  SERVICE         CREATED          STATUS          PORTS
+app_redis       redis                           "docker-entrypoint.s…"   redis           10 minutes ago   Up 10 minutes   6379/tcp
+iotdb-service   apache/iotdb:1.2.2-standalone   "/usr/bin/dumb-init …"   iotdb-service   10 minutes ago   Up 10 minutes   0.0.0.0:6667->6667/tcp, :::6667->6667/tcp
+vissv2server    cdsp-vissv2server               "/app/vissv2server -…"   vissv2server    10 minutes ago   Up 4 seconds    0.0.0.0:8081->8081/tcp, 0.0.0.0:8600->8600/tcp, 0.0.0.0:8887->8887/tcp, 127.0.0.1:8888->8888/tcp
 ```
 #### Apache IoTDB
 You can confirm the Apache IoTDB server is running by connecting to it with the IoTDB CLI client (_quit_ to exit the client):
@@ -82,10 +138,10 @@ Starting IoTDB Cli
 Successfully login at iotdb-service:6667
 IoTDB>
 ```
-#### WAII VISS server
-You can confirm the WAII VISS server is running by using one of its included clients. 
+#### VISSR VISS server
+You can confirm the VISSR VISS server is running by using one of its included clients. 
 
-The following example uses the javascript HTML client from `automotive-viss2/client/client-1.0/Javascript/httpclient.html`:
+The following example uses the javascript HTML client from `vissr/client/client-1.0/Javascript/httpclient.html`:
 
 1. Get the IP address of the VISS server using docker inspect (in this example 192.168.128.5 is returned.):
 ```
@@ -108,9 +164,4 @@ $ sudo docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}
   Server: {"data":{"dp":{"ts":"2024-01-10T14:56:48Z","value":"Data-not-found"},"path":"Vehicle.Speed"},"ts":"2024-01-10T14:56:48Z"}
 ```
 
-You can query what VSS nodes the server understands by asking for the VSS path list using the URL `http://localhost:8081/vsspathlist`. Entering that URL in your web browser will typically give you a graphical rendering of the JSON data returned.
-
-## With Realm data store backend
-TBA
-## With SQLite or Redis data store backend
-TBA
+You can query what VSS nodes the server understands by asking for the VSS path list using the URL `http://localhost:8081/vsspathlist`. Entering that URL in your web browser will typically give you a graphical rendering of the JSON data returned.

docker/docker-compose-waii-iotdb.yml → docker/docker-compose-cdsp.yml

@@ -1,5 +1,4 @@
-version: "3"
-name: cdsp-waii-iotdb
+name: cdsp
 
 services:
 
@@ -8,6 +7,7 @@ services:
     image: apache/iotdb:1.2.2-standalone
     hostname: iotdb-service
     container_name: iotdb-service
+    restart: always
     ports:
       - "6667:6667"
     environment:
@@ -29,12 +29,11 @@ services:
         - ./data:/iotdb/data
         - ./logs:/iotdb/logs
 
-# WAII acting as VISS Protocol Data Server
-#WAII_SRC_PATH = ../cdsp/automotive-viss2
+# VISSR acting as VISS Data Server
 
   tmp:
     image: busybox
-    container_name: waii_container_volumes
+    container_name: vissr_container_volumes
     user: root
     command: >
       chmod -R 777 /tmp/docker/
@@ -48,20 +47,25 @@ services:
     user: root
     command: redis-server /etc/redis.conf
     volumes:
-      - ../cdsp/automotive-viss2/redis/redis.conf:/etc/redis.conf
+      - ../cdsp/vissr/redis/redis.conf:/etc/redis.conf
     volumes_from:
       - tmp
 
   vissv2server:
     container_name: vissv2server
+    depends_on:
+      - redis
+      - iotdb-service
     build:
-      context: ../cdsp/automotive-viss2  # context set to repo root
+      context: ../cdsp/vissr  # context set to repo root
+      dockerfile: Dockerfile.rlserver
       target: vissv2server
-    entrypoint: [ /app/vissv2server,-s,redis]
+    entrypoint: [ /app/vissv2server,-s,apache-iotdb]
     ports:
-      - "127.0.0.1:8081:8081"
+      - "0.0.0.0:8081:8081"
       - "127.0.0.1:8888:8888"
-      - "127.0.0.1:8887:8887"
+      - "0.0.0.0:8887:8887"
+      - "0.0.0.0:8600:8600"
     volumes:
       - ./logs:/app/logs
     volumes_from: