Merge branch 'fix/rest-api-infrastructure' into 'master'

docker: fix docker-compose

Closes #424

See merge request caimira/caimira!512

README.md

@@ -219,7 +219,7 @@ Keep the profiler page open. Then, in another window, navigate to any page in CA
 
 The sessions are stored in a local file in the `/tmp` folder. To share it across multiple web nodes, a shared storage should be added to all web nodes. The folder can be customized via the environment variable `CAIMIRA_PROFILER_CACHE_DIR`.
 
-### CAiMIRA REST API Usage
+### CAiMIRA API Usage
 
 From the root directory of the project:
 
@@ -229,9 +229,9 @@ From the root directory of the project:
     python -m caimira.api.app
     ```
 
-2. The Tornado server will run on port `8088`.
+2. The Tornado server will run on port `8081`.
 
-To test the API functionality, you can send a `POST` request to `http://localhost:8088/report` with the required inputs in the request body. For an example of the required inputs, see [this link](https://gitlab.cern.ch/caimira/caimira/-/blob/master/caimira/apps/calculator/model_generator.py?ref_type=heads#L492).
+To test the API functionality, you can send a `POST` request to `http://localhost:8081/virus_report` with the required inputs in the request body. For an example of the required inputs, see [the baseline raw form data](https://gitlab.cern.ch/caimira/caimira/blob/master/caimira/src/caimira/calculator/validators/virus/virus_validator.py#L565).
 
 The response format will be:
 
@@ -248,14 +248,16 @@ The response format will be:
 
 ### Building the whole environment for local development
 
-**Simulate the docker build that takes place on openshift with:**
-
 ```
-s2i build file://$(pwd) --copy --keep-symlinks --context-dir ./app-config/nginx/ centos/nginx-112-centos7 caimira-nginx-app
-docker build . -f ./app-config/calculator-app/Dockerfile -t calculator-app
+docker build -f app-config/api-app/Dockerfile -t api-app .
+docker build -f app-config/calculator-app/Dockerfile -t calculator-app .
 docker build ./app-config/auth-service -t auth-service
 ```
 
+If you are using a computer with ARM CPU (Mac M1/2/3), then add the arg `--platform linux/arm64` to the docker build cmd.
+
+If you need to debug the Docker build, add the args `--no-cache --progress=plain` to see a more verbose output in your terminal.
+
 Get the client secret from the CERN Application portal for the `caimira-test` app. See [CERN-SSO-integration](#cern-sso-integration) for more info.
 ```
 read CLIENT_SECRET
@@ -267,22 +269,23 @@ export COOKIE_SECRET=$(openssl rand -hex 50)
 export OIDC_SERVER=https://auth.cern.ch/auth
 export OIDC_REALM=CERN
 export CLIENT_ID=caimira-test
+export CLIENT_SECRET=$CLIENT_SECRET
 ```
 
-Run docker-compose:
+Run docker compose:
 ```
 cd app-config
-CURRENT_UID=$(id -u):$(id -g) docker-compose up
+CURRENT_UID=$(id -u):$(id -g) docker compose up
 ```
 
 Then visit http://localhost:8080/.
 
-### Setting up the application on openshift
+### Setting up the application on OpenShift
 
 The https://cern.ch/caimira application is running on CERN's OpenShift platform. In order to set it up for the first time, we followed the documentation at https://paas.docs.cern.ch/. In particular we:
 
  * Added the OpenShift application deploy key to the GitLab repository
- * Created a Python 3.6 (the highest possible at the time of writing) application in OpenShift
+ * Created a Python 3.12 (the highest possible at the time of writing) application in OpenShift
  * Configured a generic webhook on OpenShift, and call that from the CI of the GitLab repository
 
 ### Updating the caimira-test.web.cern.ch instance

app-config/api-app/Dockerfile

@@ -0,0 +1,41 @@
+FROM registry.cern.ch/docker.io/condaforge/mambaforge AS conda
+
+ARG PYTHON_VERSION=3.12
+RUN mamba create --yes -p /opt/app python=${PYTHON_VERSION}
+
+COPY . /opt/app-source
+WORKDIR /opt/app-source
+# install Python deps
+RUN cd caimira \
+    && conda run -p /opt/app python -m pip install .
+
+COPY app-config/api-app/app.sh /opt/app/bin/api-app.sh
+
+RUN cd /opt/app \
+    && find -name '*.a' -delete \
+    && rm -rf /opt/app/conda-meta \
+    && rm -rf /opt/app/include \
+    && find -name '__pycache__' -type d -exec rm -rf '{}' '+' \
+    && rm -rf /opt/app/lib/python*/site-packages/pip /opt/app/lib/python*/idlelib /opt/app/lib/python*/ensurepip \
+          /opt/app/bin/x86_64-conda-linux-gnu-ld \
+          /opt/app/bin/sqlite3 \
+          /opt/app/bin/openssl \
+          /opt/app/share/terminfo \
+    && find /opt/app/lib/ -name 'tests' -type d -exec rm -rf '{}' '+' \
+    && find /opt/app/lib -name '*.pyx' -delete \
+    ;
+
+FROM registry.cern.ch/docker.io/library/debian
+
+COPY --from=conda /opt/app /opt/app
+ENV PATH=/opt/app/bin/:$PATH
+# Make a convenient location to the installed CAiMIRA package (i.e. a directory called caimira in the CWD).
+# It is important that this directory is also writable by a non-root user.
+RUN mkdir -p /scratch \
+  && chmod a+wx /scratch
+# Set the HOME directory to something that anybody can write to (to support non root users, such as on openshift).
+ENV HOME=/scratch
+WORKDIR /scratch
+RUN CAIMIRA_INIT_FILE=$(python -c "import caimira; print(caimira.__file__)") \
+    && ln -s $(dirname ${CAIMIRA_INIT_FILE}) /scratch/caimira
+CMD [ "api-app.sh" ]

app-config/api-app/app.sh

@@ -0,0 +1,9 @@
+#!/bin/bash
+
+args=("$@")
+if [ "$DEBUG" != "true" ] && [[ ! "${args[@]}" =~ "--no-debug" ]]; then
+    args+=("--no-debug")
+fi
+
+echo "Starting the caimira api app with: python -m caimira.api.app ${args[@]}"
+python -m caimira.api.app "${args[@]}"

app-config/auth-service/Dockerfile

@@ -1,7 +1,8 @@
 FROM registry.cern.ch/docker.io/condaforge/mambaforge as conda
 
-RUN mamba create --yes -p /opt/app python=3.9
+RUN mamba create --yes -p /opt/app python=3.12
 COPY . /opt/app-source
+
 RUN conda run -p /opt/app python -m pip install /opt/app-source
 RUN cd /opt/app \
  && find -name '*.a' -delete \

app-config/calculator-app/app.sh

@@ -26,7 +26,7 @@ if [[ "$APP_NAME" == "calculator-app" ]]; then
     export "DATA_SERVICE_ENABLED"="${DATA_SERVICE_ENABLED:=0}"
     export "CAIMIRA_PROFILER_ENABLED"="${CAIMIRA_PROFILER_ENABLED:=0}"
 
-    echo "Starting the caimira webservice with: python -m cern_caimira.apps.calculator ${args[@]}"
+    echo "Starting the caimira calculator app with: python -m cern_caimira.apps.calculator ${args[@]}"
     python -m cern_caimira.apps.calculator "${args[@]}"
 
 else

app-config/docker-compose.yml

@@ -1,5 +1,9 @@
-version: "3.8"
+
 services:
+  api-app:
+    image: api-app
+    user: ${CURRENT_UID}
+
   calculator-app:
     image: calculator-app
     environment:
@@ -7,7 +11,7 @@ services:
       - APP_NAME=calculator-app
       - APPLICATION_ROOT=/
       - CAIMIRA_CALCULATOR_PREFIX=/calculator-cern
-      - CAIMIRA_THEME=ui/apps/templates/cern
+      - CAIMIRA_THEME=cern_caimira/apps/templates/cern
       - DATA_SERVICE_ENABLED=0
       - CAIMIRA_PROFILER_ENABLED=0
     user: ${CURRENT_UID}
@@ -34,9 +38,11 @@ services:
     user: ${CURRENT_UID}
 
   caimira-router:
-    image: caimira-nginx-app
+    image: nginx:1.27
     ports:
       - "8080:8080"
+    volumes:
+      - ./nginx/nginx.conf:/etc/nginx/conf.d/nginx.conf:ro
     depends_on:
       calculator-app:
         condition: service_started
@@ -44,4 +50,3 @@ services:
         condition: service_started
       auth-service:
         condition: service_started
-    user: ${CURRENT_UID}

app-config/nginx/nginx.conf

@@ -61,6 +61,12 @@ server {
         proxy_pass http://calculator-app:8080;
     }
 
+    location /api/ {
+        # The trailing / in the proxy_pass ensures that the /api/ part
+        # is stripped before the request is passed to http://api-app:8081.
+        proxy_pass http://api-app:8081/;
+    }
+
     location /calculator {
         return 302 /calculator-cern$is_args$args;
     }

caimira/src/caimira/api/app.py

@@ -2,33 +2,44 @@
 # Entry point for the CAiMIRA application
 # """
 
+import argparse
 import tornado.ioloop
 import tornado.web
 import tornado.log
-from tornado.options import define, options
 import logging
 
 from caimira.api.routes.report_routes import VirusReportHandler, CO2ReportHandler
 
-define("port", default=8088, help="Port to listen on", type=int)
-
 logging.basicConfig(format="%(message)s", level=logging.INFO)
 
 
 class Application(tornado.web.Application):
-    def __init__(self):
+    def __init__(self, debug):
         handlers = [
             (r"/co2_report", CO2ReportHandler),
             (r"/virus_report", VirusReportHandler),
         ]
         settings = dict(
-            debug=True,
+            debug=debug,
         )
         super().__init__(handlers, **settings)
 
 
 if __name__ == "__main__":
-    app = Application()
-    app.listen(options.port)
-    logging.info(f"Tornado server is running on port {options.port}")
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        "--no-debug", help="Don't enable debug mode",
+        action="store_false",
+    )
+    parser.add_argument(
+        "--port",
+        help="The port to listen on",
+        default="8081"
+    )
+    args = parser.parse_args()
+    debug = args.no_debug
+
+    app = Application(debug=debug)
+    app.listen(args.port)
+    logging.info(f"Tornado API server is running on port {args.port}")
     tornado.ioloop.IOLoop.current().start()