Docker: Unable to locate package libthemis-dev

[albertbuchard]

Describe the bug
When trying to dockerize themis using debian (buster or bullseye) - the dockerization stops at the installation of themis with error:
Unable to locate package libthemis-dev

To Reproduce
Steps to reproduce the behavior:

1. Create Dockerfile

FROM node:18-buster
RUN apt update && \
    apt -y install apt-transport-https

# Install themis
RUN wget -qO - https://pkgs-ce.cossacklabs.com/gpg | apt-key add -
RUN echo "deb https://pkgs-ce.cossacklabs.com/stable/debian buster main" | tee /etc/apt/sources.list.d/cossacklabs.list
RUN apt update && apt install -y libthemis-dev

2. Run docker build .

error
 > [ 5/11] RUN apt update && apt install -y libthemis-dev:
#9 0.223 
#9 0.223 WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
#9 0.223 
#9 0.326 Hit:1 http://deb.debian.org/debian buster InRelease
#9 0.330 Hit:2 http://deb.debian.org/debian-security buster/updates InRelease
#9 0.342 Hit:3 http://deb.debian.org/debian buster-updates InRelease
#9 0.350 Get:4 https://pkgs-ce.cossacklabs.com/stable/debian buster InRelease [1709 B]
#9 0.816 Fetched 1709 B in 1s (3049 B/s)
#9 0.816 Reading package lists...
#9 1.159 Building dependency tree...
#9 1.223 Reading state information...
#9 1.235 21 packages can be upgraded. Run 'apt list --upgradable' to see them.
#9 1.241 
#9 1.241 WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
#9 1.241 
#9 1.246 Reading package lists...
#9 1.595 Building dependency tree...
#9 1.666 Reading state information...
#9 1.715 E: Unable to locate package libthemis-dev
------
executor failed running [/bin/sh -c apt update && apt install -y libthemis-dev]: exit code: 100

Expected behavior
Should find libthemis-dev

Environment (please complete the following information):

• OS: MacOS ventura
• Hardware: 64-bit, Macbook pro M1
• Themis version: latest?
• Installation way:
  • via package manager
  • built from source

Additional context
.

[shadinua]

Hi, @albertbuchard!

Thank you for the detailed description of the problem. We have built and added packages for the arm64 architecture and tested according to the process you have described above.

Could you please check if it solve the issue and confirm that?

[yen936]

Hi @shadinua,

I have the exact same issue bug report as @albertbuchard.

Am I using the wrong base image? Thank you so much for your help.

Describe the bug
Docker image fails to build

To Reproduce
Steps to reproduce the behavior:

1. Here is my docker file

ARG FUNCTION_DIR="/function"

FROM python:3.10 as build-image

# Include global arg in this stage of the build
ARG FUNCTION_DIR

# Copy function code
RUN mkdir -p ${FUNCTION_DIR}
COPY . ${FUNCTION_DIR}

# Install the function's dependencies
RUN pip install \
    --target ${FUNCTION_DIR} \
        awslambdaric

RUN apt update && \
    apt -y install apt-transport-https

# Install themis
RUN wget -qO - https://pkgs-ce.cossacklabs.com/gpg | apt-key add -
RUN echo "deb https://pkgs-ce.cossacklabs.com/stable/debian buster main" | tee /etc/apt/sources.list.d/cossacklabs.list
RUN apt update && apt install -y libthemis-dev

# Use a slim version of the base Python image to reduce the final image size
FROM python:3.10-slim

# Include global arg in this stage of the build
ARG FUNCTION_DIR
# Set working directory to function root directory
WORKDIR ${FUNCTION_DIR}

# Copy in the built dependencies
COPY --from=build-image ${FUNCTION_DIR} ${FUNCTION_DIR}

# Set runtime interface client as default command for the container runtime
ENTRYPOINT [ "/usr/local/bin/python", "-m", "awslambdaric" ]
# Pass the name of the function handler as an argument to the runtime
CMD [ "lambda_function.handler" ]

3. Run docker build -t docker-image:test .
4. See the following error:

------
dockerfile:24
--------------------
  22 |     RUN wget -qO - https://pkgs-ce.cossacklabs.com/gpg | apt-key add -
  23 |     RUN echo "deb https://pkgs-ce.cossacklabs.com/stable/debian buster main" | tee /etc/apt/sources.list.d/cossacklabs.list
  24 | >>> RUN apt update && apt install -y libthemis-dev
  25 |     
  26 |     # Use a slim version of the base Python image to reduce the final image size
--------------------
ERROR: failed to solve: process "/bin/sh -c apt update && apt install -y libthemis-dev" did not complete successfully: exit code: 100

Expected behavior
Docker to build

Environment (please complete the following information):

• OS: OSX 13.3.1 Ventura
• Hardware: 64-bit Mac
• Themis version: 0.15.0 Newest
• Installation way:
  • via package manager
  • built from docker

Additional context
Add any other relevant context for the problem here. Share an example project, if you can.

[Lagovas]

python:3.10 image uses debian:bookworm as base image that is not supported yet. please, try to use python:3.10-bullseye

[yen936]

FROM public.ecr.aws/lambda/python:3.10

# Copy requirements.txt
COPY requirements.txt ${LAMBDA_TASK_ROOT}

# Copy function code
COPY lambda_function.py ${LAMBDA_TASK_ROOT}

RUN yum update -y

# RUN yum install -y httpd

RUN rpm --import https://pkgs-ce.cossacklabs.com/gpg
RUN wget -qO - https://pkgs-ce.cossacklabs.com/stable/centos/cossacklabs.repo | tee /etc/yum.repos.d/cossacklabs.repo
RUN yum install libthemis-devel


# Install the specified packages
RUN pip install -r requirements.txt

# Set the CMD to your handler (could also be done as a parameter override outside of the Dockerfile)
CMD [ "lambda_function.handler" ]

that worked thank you @Lagovas.

now I need to transition the dockerfile to one of AWS's base images. This image is Amazon Linux based on RedHat to my knowledge. Do you have an idea why this fails?

• I have tried to install other packages with yum: successful.

I am not clear why yum cannot find libthemis-devel

[+] Building 4.4s (11/12)                                                                                                                                                                                   
 => [internal] load build definition from dockerfile                                                                                                                                                   0.0s
 => => transferring dockerfile: 2.05kB                                                                                                                                                                 0.0s
 => [internal] load .dockerignore                                                                                                                                                                      0.0s
 => => transferring context: 2B                                                                                                                                                                        0.0s
 => [internal] load metadata for public.ecr.aws/lambda/python:3.10                                                                                                                                     0.9s
 => [1/8] FROM public.ecr.aws/lambda/python:3.10@sha256:19b2ea7fc3484f112f63b8dbc836847c1cd5ac41cf0305af8fdfa09b4470c9b6                                                                               0.0s
 => [internal] load build context                                                                                                                                                                      0.0s
 => => transferring context: 75B                                                                                                                                                                       0.0s
 => CACHED [2/8] COPY requirements.txt /var/task                                                                                                                                                       0.0s
 => CACHED [3/8] COPY lambda_function.py /var/task                                                                                                                                                     0.0s
 => CACHED [4/8] RUN yum update -y                                                                                                                                                                     0.0s
 => [5/8] RUN rpm --import https://pkgs-ce.cossacklabs.com/gpg                                                                                                                                         1.4s
 => [6/8] RUN wget -qO - https://pkgs-ce.cossacklabs.com/stable/centos/cossacklabs.repo | tee /etc/yum.repos.d/cossacklabs.repo                                                                        0.3s
 => ERROR [7/8] RUN yum install libthemis-devel                                                                                                                                                        1.8s
------                                                                                                                                                                                                      
 > [7/8] RUN yum install libthemis-devel:                                                                                                                                                                   
#0 0.389 Loaded plugins: ovl
#0 1.683 No package libthemis-devel available.
#0 1.801 Error: Nothing to do
------
dockerfile:15
--------------------
  13 |     RUN rpm --import https://pkgs-ce.cossacklabs.com/gpg
  14 |     RUN wget -qO - https://pkgs-ce.cossacklabs.com/stable/centos/cossacklabs.repo | tee /etc/yum.repos.d/cossacklabs.repo
  15 | >>> RUN yum install libthemis-devel
  16 |     
  17 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c yum install libthemis-devel" did not complete successfully: exit code: 1

[Lagovas]

I'm unfamiliar with AWS and centos too much, and I'm not sure what repo structure AWS expects. We support, test and maintain repositories for centos 7/8, and temporarily latest version deployed only for x86_64. You can try to download .rpm manually from https://pkgs-ce.cossacklabs.com/stable/centos/8/x86_64/ and try to install it locally from the file. It may works if AWS expects different repo structure in comparison with centos 7/8 but okay with .rpm packages packed for them. And it may work only for x86_64, not arm64.

As I see, base image uses scratch base image and install FS structure from archives - https://github.com/aws/aws-lambda-base-images/blob/python3.10/Dockerfile.python3.10 . So, it needs a time to investigate and debug.

[yen936]

Thanks for the response--and thanks for working on this lib. Yes, the base image was the key. By pivoting into the base image that you mentioned I finally got it, thank God. So, here's what I did for future reference:

# Define custom function directory
ARG FUNCTION_DIR="/function"

FROM python:3.10-bullseye as build-image

# Include global arg in this stage of the build
ARG FUNCTION_DIR

# Copy function code
RUN mkdir -p ${FUNCTION_DIR}
COPY . ${FUNCTION_DIR}

# Install the function's dependencies
RUN pip install --target ${FUNCTION_DIR} awslambdaric boto3 botocore pythemis

RUN apt update && apt install -y apt-transport-https wget gnupg

# Install themis
RUN wget -qO - https://pkgs-ce.cossacklabs.com/gpg | apt-key add -
RUN echo "deb https://pkgs-ce.cossacklabs.com/stable/debian bullseye main" | tee /etc/apt/sources.list.d/cossacklabs.list
RUN apt update && apt install -y libthemis-dev

WORKDIR ${FUNCTION_DIR}