api: add Ddocker example

* partially closes #424
This commit is contained in:
Nicola Tarocco 2024-10-08 23:29:14 +02:00
parent 72a6afece2
commit 0ccbfe4f9a
No known key found for this signature in database
GPG key ID: A08DEF00BA54E806
7 changed files with 93 additions and 22 deletions

View file

@ -219,7 +219,7 @@ Keep the profiler page open. Then, in another window, navigate to any page in CA
The sessions are stored in a local file in the `/tmp` folder. To share it across multiple web nodes, a shared storage should be added to all web nodes. The folder can be customized via the environment variable `CAIMIRA_PROFILER_CACHE_DIR`. The sessions are stored in a local file in the `/tmp` folder. To share it across multiple web nodes, a shared storage should be added to all web nodes. The folder can be customized via the environment variable `CAIMIRA_PROFILER_CACHE_DIR`.
### CAiMIRA REST API Usage ### CAiMIRA API Usage
From the root directory of the project: From the root directory of the project:
@ -229,9 +229,9 @@ From the root directory of the project:
python -m caimira.api.app python -m caimira.api.app
``` ```
2. The Tornado server will run on port `8088`. 2. The Tornado server will run on port `8081`.
To test the API functionality, you can send a `POST` request to `http://localhost:8088/report` with the required inputs in the request body. For an example of the required inputs, see [this link](https://gitlab.cern.ch/caimira/caimira/-/blob/master/caimira/apps/calculator/model_generator.py?ref_type=heads#L492). To test the API functionality, you can send a `POST` request to `http://localhost:8081/virus_report` with the required inputs in the request body. For an example of the required inputs, see [the baseline raw form data](https://gitlab.cern.ch/caimira/caimira/blob/master/caimira/src/caimira/calculator/validators/virus/virus_validator.py#L565).
The response format will be: The response format will be:
@ -248,13 +248,13 @@ The response format will be:
### Building the whole environment for local development ### Building the whole environment for local development
**Simulate the docker build that takes place on openshift with:** ```
docker build -f app-config/api-app/Dockerfile -t api-app .
docker build -f app-config/calculator-app/Dockerfile -t calculator-app .
docker build -f app-config/auth-service/Dockerfile -t auth-service .
```
``` If you are using a computer with ARM CPU (Mac M1/2/3), then add the arg `--platform linux/arm64` to the docker build cmd.
s2i build file://$(pwd) --copy --keep-symlinks --context-dir ./app-config/nginx/ centos/nginx-112-centos7 caimira-nginx-app
docker build . -f ./app-config/calculator-app/Dockerfile -t calculator-app
docker build ./app-config/auth-service -t auth-service
```
Get the client secret from the CERN Application portal for the `caimira-test` app. See [CERN-SSO-integration](#cern-sso-integration) for more info. Get the client secret from the CERN Application portal for the `caimira-test` app. See [CERN-SSO-integration](#cern-sso-integration) for more info.
``` ```
@ -269,20 +269,20 @@ export OIDC_REALM=CERN
export CLIENT_ID=caimira-test export CLIENT_ID=caimira-test
``` ```
Run docker-compose: Run docker compose:
``` ```
cd app-config cd app-config
CURRENT_UID=$(id -u):$(id -g) docker-compose up CURRENT_UID=$(id -u):$(id -g) docker compose up
``` ```
Then visit http://localhost:8080/. Then visit http://localhost:8080/.
### Setting up the application on openshift ### Setting up the application on OpenShift
The https://cern.ch/caimira application is running on CERN's OpenShift platform. In order to set it up for the first time, we followed the documentation at https://paas.docs.cern.ch/. In particular we: The https://cern.ch/caimira application is running on CERN's OpenShift platform. In order to set it up for the first time, we followed the documentation at https://paas.docs.cern.ch/. In particular we:
* Added the OpenShift application deploy key to the GitLab repository * Added the OpenShift application deploy key to the GitLab repository
* Created a Python 3.6 (the highest possible at the time of writing) application in OpenShift * Created a Python 3.12 (the highest possible at the time of writing) application in OpenShift
* Configured a generic webhook on OpenShift, and call that from the CI of the GitLab repository * Configured a generic webhook on OpenShift, and call that from the CI of the GitLab repository
### Updating the caimira-test.web.cern.ch instance ### Updating the caimira-test.web.cern.ch instance

View file

@ -0,0 +1,41 @@
FROM registry.cern.ch/docker.io/condaforge/mambaforge AS conda
ARG PYTHON_VERSION=3.12
RUN mamba create --yes -p /opt/app python=${PYTHON_VERSION}
COPY . /opt/app-source
WORKDIR /opt/app-source
# install Python deps
RUN cd caimira \
&& conda run -p /opt/app python -m pip install .
COPY app-config/api-app/app.sh /opt/app/bin/api-app.sh
RUN cd /opt/app \
&& find -name '*.a' -delete \
&& rm -rf /opt/app/conda-meta \
&& rm -rf /opt/app/include \
&& find -name '__pycache__' -type d -exec rm -rf '{}' '+' \
&& rm -rf /opt/app/lib/python*/site-packages/pip /opt/app/lib/python*/idlelib /opt/app/lib/python*/ensurepip \
/opt/app/bin/x86_64-conda-linux-gnu-ld \
/opt/app/bin/sqlite3 \
/opt/app/bin/openssl \
/opt/app/share/terminfo \
&& find /opt/app/lib/ -name 'tests' -type d -exec rm -rf '{}' '+' \
&& find /opt/app/lib -name '*.pyx' -delete \
;
FROM registry.cern.ch/docker.io/library/debian
COPY --from=conda /opt/app /opt/app
ENV PATH=/opt/app/bin/:$PATH
# Make a convenient location to the installed CAiMIRA package (i.e. a directory called caimira in the CWD).
# It is important that this directory is also writable by a non-root user.
RUN mkdir -p /scratch \
&& chmod a+wx /scratch
# Set the HOME directory to something that anybody can write to (to support non root users, such as on openshift).
ENV HOME=/scratch
WORKDIR /scratch
RUN CAIMIRA_INIT_FILE=$(python -c "import caimira; print(caimira.__file__)") \
&& ln -s $(dirname ${CAIMIRA_INIT_FILE}) /scratch/caimira
CMD [ "api-app.sh" ]

9
app-config/api-app/app.sh Executable file
View file

@ -0,0 +1,9 @@
#!/bin/bash
args=("$@")
if [ "$DEBUG" != "true" ] && [[ ! "${args[@]}" =~ "--no-debug" ]]; then
args+=("--no-debug")
fi
echo "Starting the caimira api app with: python -m caimira.api.app ${args[@]}"
python -m caimira.api.app "${args[@]}"

View file

@ -26,7 +26,7 @@ if [[ "$APP_NAME" == "calculator-app" ]]; then
export "DATA_SERVICE_ENABLED"="${DATA_SERVICE_ENABLED:=0}" export "DATA_SERVICE_ENABLED"="${DATA_SERVICE_ENABLED:=0}"
export "CAIMIRA_PROFILER_ENABLED"="${CAIMIRA_PROFILER_ENABLED:=0}" export "CAIMIRA_PROFILER_ENABLED"="${CAIMIRA_PROFILER_ENABLED:=0}"
echo "Starting the caimira webservice with: python -m cern_caimira.apps.calculator ${args[@]}" echo "Starting the caimira calculator app with: python -m cern_caimira.apps.calculator ${args[@]}"
python -m cern_caimira.apps.calculator "${args[@]}" python -m cern_caimira.apps.calculator "${args[@]}"
else else

View file

@ -1,5 +1,9 @@
services: services:
api-app:
image: api-app
user: ${CURRENT_UID}
calculator-app: calculator-app:
image: calculator-app image: calculator-app
environment: environment:

View file

@ -61,6 +61,12 @@ server {
proxy_pass http://calculator-app:8080; proxy_pass http://calculator-app:8080;
} }
location /api/ {
# The trailing / in the proxy_pass ensures that the /api/ part
# is stripped before the request is passed to http://api-app:8081.
proxy_pass http://api-app:8081/;
}
location /calculator { location /calculator {
return 302 /calculator-cern$is_args$args; return 302 /calculator-cern$is_args$args;
} }

View file

@ -2,33 +2,44 @@
# Entry point for the CAiMIRA application # Entry point for the CAiMIRA application
# """ # """
import argparse
import tornado.ioloop import tornado.ioloop
import tornado.web import tornado.web
import tornado.log import tornado.log
from tornado.options import define, options
import logging import logging
from caimira.api.routes.report_routes import VirusReportHandler, CO2ReportHandler from caimira.api.routes.report_routes import VirusReportHandler, CO2ReportHandler
define("port", default=8088, help="Port to listen on", type=int)
logging.basicConfig(format="%(message)s", level=logging.INFO) logging.basicConfig(format="%(message)s", level=logging.INFO)
class Application(tornado.web.Application): class Application(tornado.web.Application):
def __init__(self): def __init__(self, debug):
handlers = [ handlers = [
(r"/co2_report", CO2ReportHandler), (r"/co2_report", CO2ReportHandler),
(r"/virus_report", VirusReportHandler), (r"/virus_report", VirusReportHandler),
] ]
settings = dict( settings = dict(
debug=True, debug=debug,
) )
super().__init__(handlers, **settings) super().__init__(handlers, **settings)
if __name__ == "__main__": if __name__ == "__main__":
app = Application() parser = argparse.ArgumentParser()
app.listen(options.port) parser.add_argument(
logging.info(f"Tornado server is running on port {options.port}") "--no-debug", help="Don't enable debug mode",
action="store_false",
)
parser.add_argument(
"--port",
help="The port to listen on",
default="8081"
)
args = parser.parse_args()
debug = args.no_debug
app = Application(debug=debug)
app.listen(args.port)
logging.info(f"Tornado API server is running on port {args.port}")
tornado.ioloop.IOLoop.current().start() tornado.ioloop.IOLoop.current().start()