OGC API Process Client

Goal

Create a container and run the ogc-api-processes-client step in the container image. This container will interact with the a deployrd process (e.g. water-bodies) on the OGC API endpint, submit a job with the recived inputs, monitor the job for any upcoming event(e.g running, successful, failed), and finally create STAC ItemCollection with assets pointing to the results.

Lab

This step has a dedicated lab available at execute-monitor-process.md. But it is very important to run 01-Deploy-an-application-package.ipynb before proceeding with job execution step.

Container

This ogc-api-processes-client has its own recipe to build the container image. The recipe is provided in the cell below:

# Stage 1: Build stage
FROM rockylinux:9.3-minimal AS build

# Install necessary build tools
RUN microdnf install -y curl tar && \
    microdnf clean all

# Download the hatch tar.gz file from GitHub
RUN curl -L https://github.com/pypa/hatch/releases/latest/download/hatch-x86_64-unknown-linux-gnu.tar.gz -o /tmp/hatch-x86_64-unknown-linux-gnu.tar.gz

# Extract the hatch binary
RUN tar -xzf /tmp/hatch-x86_64-unknown-linux-gnu.tar.gz -C /tmp/

# Stage 2: Final stage
FROM rockylinux:9

# Install runtime dependencies
RUN dnf install -y --nodocs which expat && \
    dnf clean all

# Set up a default user and home directory
ENV HOME=/home/ogc

# Create a user with UID 1001, group root, and a home directory
RUN useradd -u 1001 -r -g 0 -m -d ${HOME} -s /sbin/nologin \
        -c "Default ogc User" ogc && \
    mkdir -p /code && \
    mkdir -p /prod && \
    chown -R 1001:0 /code && \
    chmod g+rwx ${HOME} /code

# Copy the hatch binary from the build stage
COPY --from=build /tmp/hatch /usr/bin/hatch

# Ensure the hatch binary is executable
RUN chmod +x /usr/bin/hatch

# Install runtime dependencies
RUN dnf install -y which jq git && \
    dnf clean all


# Copy the application files into the /code directory
COPY --chown=1001:0 . /code

ARG STARS_VERSION

RUN if [[ -n "$STARS_VERSION" ]] ; then dnf install -y https://github.com/Terradue/Stars/releases/download/$STARS_VERSION/Stars.$STARS_VERSION.linux-x64.rpm; Stars --help;  fi 

# Switch to the non-root user
USER ogc
WORKDIR /code

# Set up virtual environment paths
ENV VIRTUAL_ENV=/code/envs/ogc
ENV PATH="$VIRTUAL_ENV/bin:$PATH"

# Prune any existing environments and create a new production environment
RUN hatch env prune && \
    hatch env create prod && \
    hatch run prod:ogc-api-processes-client --help && \
    chmod +rx -R /code/envs/ogc && \
    chmod +rx -R /home/ogc && \
    rm -fr /code/.git /code/.pytest_cache


WORKDIR /code

# Set the default command to run when the container starts
CMD ["ogc-api-processes-client"]

Build the container image with:

#!/bin/bash
export WORKSPACE=$PWD
docker_tag=$(yq eval '
  ."$graph"[]
  | select(.id == "ogc-api-processes-client")
  | .requirements[]
  | select(.class == "DockerRequirement")
  | .dockerPull
' cwl-workflows/eoap-api-cli.cwl)

echo $docker_tag
docker build -t $docker_tag $WORKSPACE/containers/ogc-api-processes-client

How to run a step in a container

We'll use podman container engine (docker is also fine).

Before running the container using podman it is very important to deploy the application package such as water-bodies on the OGC endpoint. The deployment of water-bodies application package is explained in 01-Deploy-an-application-package.ipynb. Once the deployment was successful, the command to run the ogc-api-processes-client step in the container is:

docker_tag=$(yq eval '
  ."$graph"[]
  | select(.id == "ogc-api-processes-client")
  | .requirements[]
  | select(.class == "DockerRequirement")
  | .dockerPull
' cwl-workflows/eoap-api-cli.cwl)

podman \
    run \
    -i \
    --userns=keep-id \
    --mount=type=bind,source=/workspace/ogc-api-processes-with-zoo/runs,target=/runs \
    --workdir=/runs \
    --read-only=true \
    --user=1001:100 \
    --rm \
    --env=TMPDIR=/tmp \
    --env=HOME=/runs \
    $docker_tag \
    ogc-api-processes-client \
    --api-endpoint \
    http://zoo-project-dru-service/acme/ogc-api/ \
    --process-id \
    water-bodies \
    --execute-request \
    containers/ogc-api-processes-client/execute_request.json \
    --output \
    feature-collection.json

Let's break down what this command does:

• podman run: This is the command to run a container.
• -i: This flag makes the container interactive, allowing you to interact with it via the terminal.
• --userns=keep-id: It instructs podman to keep the user namespace ID. --mount=type=bind,source=/workspace/ogc-api-processes-with-zoo/runs,target=/runs: This option mounts a directory from the host system to the container. In this case, it mounts the /workspace/ogc-api-processes-with-zoo/runs directory on the host to the /runs directory inside the container.
• --workdir=/runs: Sets the working directory inside the container to /runs.
• --read-only=true: Makes the file system inside the container read-only, meaning you can't write or modify files inside the container.
• --user=1001:100: Specifies the user and group IDs to be used within the container.
• --rm: This flag tells podman to remove the container after it has finished running.
• --env=HOME=/runs: Sets the HOME environment variable inside the container to /runs.
• $docker_tag: This is the name of the container image that you were built earlier and want to run.
• ogc-api-processes-client: This is the command to run inside the container. It runs a Python module named "ogc-api-processes-client"
• --api-endpoint "http://zoo-project-dru-service/acme/ogc-api/": This provides command-line arguments to the Python module. It specifies the address to the OGC API endpoint where the service is running.
• --process-id: Specifies the id of process we deployed (e.g. water-bodies) in 01-Deploy-an-application-package.ipynb.
• --execute-request: This input point to the JSON file containing the information of two sentinel-2/landsat products from planetarycomputer are going to pass to the water-bodies application pacakge. An expample of this JSON file is mentioned below:

{
    "inputs": {
        "stac_items": [
            "https://planetarycomputer.microsoft.com/api/stac/v1/collections/landsat-c2-l2/items/LC08_L2SP_044032_20231208_02_T1",
            "https://planetarycomputer.microsoft.com/api/stac/v1/collections/landsat-c2-l2/items/LC08_L2SP_043033_20231201_02_T1"
        ],
        "aoi": "-121.399,39.834,-120.74,40.472",
        "epsg": "EPSG:4326",
        "bands": [
            "green",
            "nir08"
        ]
    }
}

• --output: This input would pass the JSON file name containing the result of wate-bodies detection in STAC ItemCollection format

Expected outcome

The folder /workspace/ogc-api-processes-with-zoo/runs contains:

(base) jovyan@coder-mrossi:~/runs$ tree .
.
└── feature-collection.json

0 directories, 1 file