api: add Ddocker example

* partially closes #424
This commit is contained in:
Nicola Tarocco 2024-10-08 23:29:14 +02:00
parent 72a6afece2
commit 0ccbfe4f9a
No known key found for this signature in database
GPG key ID: A08DEF00BA54E806
7 changed files with 93 additions and 22 deletions

View file

@ -219,7 +219,7 @@ Keep the profiler page open. Then, in another window, navigate to any page in CA
The sessions are stored in a local file in the `/tmp` folder. To share it across multiple web nodes, a shared storage should be added to all web nodes. The folder can be customized via the environment variable `CAIMIRA_PROFILER_CACHE_DIR`.
### CAiMIRA REST API Usage
### CAiMIRA API Usage
From the root directory of the project:
@ -229,9 +229,9 @@ From the root directory of the project:
python -m caimira.api.app
```
2. The Tornado server will run on port `8088`.
2. The Tornado server will run on port `8081`.
To test the API functionality, you can send a `POST` request to `http://localhost:8088/report` with the required inputs in the request body. For an example of the required inputs, see [this link](https://gitlab.cern.ch/caimira/caimira/-/blob/master/caimira/apps/calculator/model_generator.py?ref_type=heads#L492).
To test the API functionality, you can send a `POST` request to `http://localhost:8081/virus_report` with the required inputs in the request body. For an example of the required inputs, see [the baseline raw form data](https://gitlab.cern.ch/caimira/caimira/blob/master/caimira/src/caimira/calculator/validators/virus/virus_validator.py#L565).
The response format will be:
@ -248,13 +248,13 @@ The response format will be:
### Building the whole environment for local development
**Simulate the docker build that takes place on openshift with:**
```
docker build -f app-config/api-app/Dockerfile -t api-app .
docker build -f app-config/calculator-app/Dockerfile -t calculator-app .
docker build -f app-config/auth-service/Dockerfile -t auth-service .
```
```
s2i build file://$(pwd) --copy --keep-symlinks --context-dir ./app-config/nginx/ centos/nginx-112-centos7 caimira-nginx-app
docker build . -f ./app-config/calculator-app/Dockerfile -t calculator-app
docker build ./app-config/auth-service -t auth-service
```
If you are using a computer with ARM CPU (Mac M1/2/3), then add the arg `--platform linux/arm64` to the docker build cmd.
Get the client secret from the CERN Application portal for the `caimira-test` app. See [CERN-SSO-integration](#cern-sso-integration) for more info.
```
@ -269,20 +269,20 @@ export OIDC_REALM=CERN
export CLIENT_ID=caimira-test
```
Run docker-compose:
Run docker compose:
```
cd app-config
CURRENT_UID=$(id -u):$(id -g) docker-compose up
CURRENT_UID=$(id -u):$(id -g) docker compose up
```
Then visit http://localhost:8080/.
### Setting up the application on openshift
### Setting up the application on OpenShift
The https://cern.ch/caimira application is running on CERN's OpenShift platform. In order to set it up for the first time, we followed the documentation at https://paas.docs.cern.ch/. In particular we:
* Added the OpenShift application deploy key to the GitLab repository
* Created a Python 3.6 (the highest possible at the time of writing) application in OpenShift
* Created a Python 3.12 (the highest possible at the time of writing) application in OpenShift
* Configured a generic webhook on OpenShift, and call that from the CI of the GitLab repository
### Updating the caimira-test.web.cern.ch instance

View file

@ -0,0 +1,41 @@
FROM registry.cern.ch/docker.io/condaforge/mambaforge AS conda
ARG PYTHON_VERSION=3.12
RUN mamba create --yes -p /opt/app python=${PYTHON_VERSION}
COPY . /opt/app-source
WORKDIR /opt/app-source
# install Python deps
RUN cd caimira \
&& conda run -p /opt/app python -m pip install .
COPY app-config/api-app/app.sh /opt/app/bin/api-app.sh
RUN cd /opt/app \
&& find -name '*.a' -delete \
&& rm -rf /opt/app/conda-meta \
&& rm -rf /opt/app/include \
&& find -name '__pycache__' -type d -exec rm -rf '{}' '+' \
&& rm -rf /opt/app/lib/python*/site-packages/pip /opt/app/lib/python*/idlelib /opt/app/lib/python*/ensurepip \
/opt/app/bin/x86_64-conda-linux-gnu-ld \
/opt/app/bin/sqlite3 \
/opt/app/bin/openssl \
/opt/app/share/terminfo \
&& find /opt/app/lib/ -name 'tests' -type d -exec rm -rf '{}' '+' \
&& find /opt/app/lib -name '*.pyx' -delete \
;
FROM registry.cern.ch/docker.io/library/debian
COPY --from=conda /opt/app /opt/app
ENV PATH=/opt/app/bin/:$PATH
# Make a convenient location to the installed CAiMIRA package (i.e. a directory called caimira in the CWD).
# It is important that this directory is also writable by a non-root user.
RUN mkdir -p /scratch \
&& chmod a+wx /scratch
# Set the HOME directory to something that anybody can write to (to support non root users, such as on openshift).
ENV HOME=/scratch
WORKDIR /scratch
RUN CAIMIRA_INIT_FILE=$(python -c "import caimira; print(caimira.__file__)") \
&& ln -s $(dirname ${CAIMIRA_INIT_FILE}) /scratch/caimira
CMD [ "api-app.sh" ]

9
app-config/api-app/app.sh Executable file
View file

@ -0,0 +1,9 @@
#!/bin/bash
args=("$@")
if [ "$DEBUG" != "true" ] && [[ ! "${args[@]}" =~ "--no-debug" ]]; then
args+=("--no-debug")
fi
echo "Starting the caimira api app with: python -m caimira.api.app ${args[@]}"
python -m caimira.api.app "${args[@]}"

View file

@ -26,7 +26,7 @@ if [[ "$APP_NAME" == "calculator-app" ]]; then
export "DATA_SERVICE_ENABLED"="${DATA_SERVICE_ENABLED:=0}"
export "CAIMIRA_PROFILER_ENABLED"="${CAIMIRA_PROFILER_ENABLED:=0}"
echo "Starting the caimira webservice with: python -m cern_caimira.apps.calculator ${args[@]}"
echo "Starting the caimira calculator app with: python -m cern_caimira.apps.calculator ${args[@]}"
python -m cern_caimira.apps.calculator "${args[@]}"
else

View file

@ -1,5 +1,9 @@
services:
api-app:
image: api-app
user: ${CURRENT_UID}
calculator-app:
image: calculator-app
environment:

View file

@ -61,6 +61,12 @@ server {
proxy_pass http://calculator-app:8080;
}
location /api/ {
# The trailing / in the proxy_pass ensures that the /api/ part
# is stripped before the request is passed to http://api-app:8081.
proxy_pass http://api-app:8081/;
}
location /calculator {
return 302 /calculator-cern$is_args$args;
}

View file

@ -2,33 +2,44 @@
# Entry point for the CAiMIRA application
# """
import argparse
import tornado.ioloop
import tornado.web
import tornado.log
from tornado.options import define, options
import logging
from caimira.api.routes.report_routes import VirusReportHandler, CO2ReportHandler
define("port", default=8088, help="Port to listen on", type=int)
logging.basicConfig(format="%(message)s", level=logging.INFO)
class Application(tornado.web.Application):
def __init__(self):
def __init__(self, debug):
handlers = [
(r"/co2_report", CO2ReportHandler),
(r"/virus_report", VirusReportHandler),
]
settings = dict(
debug=True,
debug=debug,
)
super().__init__(handlers, **settings)
if __name__ == "__main__":
app = Application()
app.listen(options.port)
logging.info(f"Tornado server is running on port {options.port}")
parser = argparse.ArgumentParser()
parser.add_argument(
"--no-debug", help="Don't enable debug mode",
action="store_false",
)
parser.add_argument(
"--port",
help="The port to listen on",
default="8081"
)
args = parser.parse_args()
debug = args.no_debug
app = Application(debug=debug)
app.listen(args.port)
logging.info(f"Tornado API server is running on port {args.port}")
tornado.ioloop.IOLoop.current().start()