Installing cmdstanr in Docker image

[jberry47]

Describe the bug
The task is to get cmdstan installed during a docker build. Following the example below, during the docker build, the logs indicate that the package is installed but when the package is called within library, it indicates that the package cannot be found.

cmd to build: docker build . --file Dockerfile --tag docker_test
log of build:

> (base) jb@WUST062456 docker_dev % docker build . --file Dockerfile --tag docker_test
[+] Building 1.9s (8/8) FINISHED                                                                 docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                             0.4s
 => => transferring dockerfile: 847B                                                                             0.4s
 => [internal] load metadata for docker.io/rocker/r-ver:latest                                                   0.9s
 => [auth] rocker/r-ver:pull token for registry-1.docker.io                                                      0.0s
 => [internal] load .dockerignore                                                                                0.1s
 => => transferring context: 2B                                                                                  0.1s
 => [1/4] FROM docker.io/rocker/r-ver:latest@sha256:45ac2c19f431efdd48d8adc59c51f56b43332c413f7486f148a4def7f8a  0.0s
 => CACHED [2/4] RUN apt-get update                                                                              0.0s
 => CACHED [3/4] RUN R -e "install.packages('cmdstanr', repos = c('https://stan-dev.r-universe.dev', getOption(  0.0s
 => ERROR [4/4] RUN R -e "library(cmdstanr)"                                                                     0.5s
------
 > [4/4] RUN R -e "library(cmdstanr)":
0.358 
0.358 R version 4.4.2 (2024-10-31) -- "Pile of Leaves"
0.358 Copyright (C) 2024 The R Foundation for Statistical Computing
0.358 Platform: x86_64-pc-linux-gnu
0.358 
0.358 R is free software and comes with ABSOLUTELY NO WARRANTY.
0.358 You are welcome to redistribute it under certain conditions.
0.358 Type 'license()' or 'licence()' for distribution details.
0.358 
0.358   Natural language support but running in an English locale
0.358 
0.358 R is a collaborative project with many contributors.
0.358 Type 'contributors()' for more information and
0.358 'citation()' on how to cite R or R packages in publications.
0.358 
0.358 Type 'demo()' for some demos, 'help()' for on-line help, or
0.358 'help.start()' for an HTML browser interface to help.
0.358 Type 'q()' to quit R.
0.358 
0.484 > library(cmdstanr)
0.486 Error in library(cmdstanr) : there is no package called ‘cmdstanr’
0.486 Execution halted
------
Dockerfile:6
--------------------
   4 |     
   5 |     RUN R -e "install.packages('cmdstanr', repos = c('https://stan-dev.r-universe.dev', getOption('repos')))"
   6 | >>> RUN R -e "library(cmdstanr)"
   7 |     
   8 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c R -e \"library(cmdstanr)\"" did not complete successfully: exit code: 1

To Reproduce
Minimum Dockerfile to reproduce

FROM rocker/r-ver:latest
RUN apt-get update
RUN R -e "install.packages('cmdstanr', repos = c('https://stan-dev.r-universe.dev', getOption('repos')))"
RUN R -e "library(cmdstanr)"

Expected behavior
Expectation is that library command executes successfully which means the functions in the package become available to use in the session

Operating system
Pulling from rocker/r-ver:latest

CmdStanR version number
Latest release from stan-dev.r-universe.dev

Additional context
None

[jgabry]

Hmm, strange. I'm not super experienced with docker, but recently @tillahoffmann created a docker image that included cmdstanr (to help me debug an issue). @tillahoffmann do you have any suggestions for successfully installing and loading cmdstanr in a docker image? Thanks!

[tillahoffmann]

It would be interesting to see the output from the following cached step. Maybe that has some further logs that could be useful.

CACHED [3/4] RUN R -e "install.packages('cmdstanr', repos = c('https://stan-dev.r-universe.dev',

We used a docker image to diagnose an issue in #1050. The Dockerfile is here and might be a good starting point. Does that setup work for you? The only difference I can spot is the different base image.

[jgabry]

Thanks @tillahoffmann!

[andrjohns]

I just built your example Dockerfile without any issues. Can you try building with the --no-cache flag/option so we can see the actual error?

[jberry47]

Ah my apologies I meant to send the log using --no-cache, it results in the same error. Very interesting that it works for you @tillahoffmann .

(base) jb@WUST062456 docker_dev % docker build . --file Dockerfile --tag docker_test --no-cache
[+] Building 14.6s (8/8) FINISHED                                                                docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                             0.0s
 => => transferring dockerfile: 847B                                                                             0.0s
 => [internal] load metadata for docker.io/rocker/r-ver:latest                                                   0.9s
 => [auth] rocker/r-ver:pull token for registry-1.docker.io                                                      0.0s
 => [internal] load .dockerignore                                                                                0.4s
 => => transferring context: 2B                                                                                  0.4s
 => CACHED [1/4] FROM docker.io/rocker/r-ver:latest@sha256:45ac2c19f431efdd48d8adc59c51f56b43332c413f7486f148a4  0.0s
 => [2/4] RUN apt-get update                                                                                    10.8s
 => [3/4] RUN R -e "install.packages('cmdstanr', repos = c('https://stan-dev.r-universe.dev', getOption('repos'  2.1s
 => ERROR [4/4] RUN R -e "library(cmdstanr)"                                                                     0.4s
------
 > [4/4] RUN R -e "library(cmdstanr)":
0.249 
0.249 R version 4.4.2 (2024-10-31) -- "Pile of Leaves"
0.249 Copyright (C) 2024 The R Foundation for Statistical Computing
0.249 Platform: x86_64-pc-linux-gnu
0.249 
0.249 R is free software and comes with ABSOLUTELY NO WARRANTY.
0.249 You are welcome to redistribute it under certain conditions.
0.249 Type 'license()' or 'licence()' for distribution details.
0.249 
0.249   Natural language support but running in an English locale
0.249 
0.249 R is a collaborative project with many contributors.
0.249 Type 'contributors()' for more information and
0.249 'citation()' on how to cite R or R packages in publications.
0.249 
0.249 Type 'demo()' for some demos, 'help()' for on-line help, or
0.249 'help.start()' for an HTML browser interface to help.
0.249 Type 'q()' to quit R.
0.249 
0.334 > library(cmdstanr)
0.335 Error in library(cmdstanr) : there is no package called ‘cmdstanr’
0.335 Execution halted
------
Dockerfile:6
--------------------
   4 |     
   5 |     RUN R -e "install.packages('cmdstanr', repos = c('https://stan-dev.r-universe.dev', getOption('repos')))"
   6 | >>> RUN R -e "library(cmdstanr)"
   7 |     
   8 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c R -e \"library(cmdstanr)\"" did not complete successfully: exit code: 1

View build details: docker-desktop://dashboard/build/desktop-linux/desktop-linux/kx1t6haji2dw6cs9rq5wnnoii

I'm thinking that is has something to do with the image from rocker. I'm using a different image from them for a different process rocker/shiny-verse:latest and I have the same issue with that image as well.

My plan this afternoon is to use Docker in debug mode and try to install cmdstanr to see if there are any more clues as to what is going on. I'll be sure to post anything I find here!

Thank you for the quick replies!

[andrjohns]

I don't think it's the container, since it's working for me. To verify, can you try running the container interactively (i.e., docker run -it rocker/r-ver:latest) and then calling the commands manually?

[jberry47]

Hi @andrjohns I tried to run it interactively with the command you supplied and via the debugger within Docker Desktop itself but I think there is some networking issues that don't allow it to reach outside the container. I tried installing ggplot2 and it also failed in both interactive environments.

However I have a much larger build that cmdstanr will go into and that build grabs all the other packages perfectly fine and I found that when it comes time for cmdstanr to install, I get this in the build log:

R version 4.4.2 (2024-10-31) -- "Pile of Leaves"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> install.packages('cmdstanr', repos = c('https://stan-dev.r-universe.dev', getOption('repos')))
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
Warning: unable to access index for repository https://stan-dev.r-universe.dev/src/contrib:
cannot open URL 'https://stan-dev.r-universe.dev/src/contrib/PACKAGES'
Warning: unable to access index for repository https://p3m.dev/cran/__linux__/noble/latest/src/contrib:
cannot open URL 'https://p3m.dev/cran/__linux__/noble/latest/src/contrib/PACKAGES'
>
>
Warning message:
package ‘cmdstanr’ is not available for this version of R
A version of this package for your version of R might be available elsewhere,
see the ideas at
https://cran.r-project.org/doc/manuals/r-patched/R-admin.html#Installing-packages

I can confirm that when I try to install cmdstanr on the physical machine using the same install.packages() command, the installation is successful and runs perfectly normally.

The next step I'm going to try is to get the tar file itself and install it as a local package.

[jberry47]

Successfully installed the package from source!

Updated minimum repro Dockerfile

FROM rocker/r-ver:latest

RUN apt-get update && apt-get install -y \
    libcurl4-gnutls-dev \
    curl
    
RUN curl -LOk https://stan-dev.r-universe.dev/bin/linux/noble/4.4/src/contrib/cmdstanr_0.8.1.tar.gz
RUN R CMD INSTALL --build /cmdstanr_0.8.1.tar.gz
RUN R -e "install.packages('posterior',repos='http://cran.rstudio.com/')"
RUN R -e "install.packages('R6',repos='http://cran.rstudio.com/')"

RUN R -e "library(cmdstanr)"

Build log confirms

R version 4.4.2 (2024-10-31) -- "Pile of Leaves"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> library(cmdstanr)
This is cmdstanr version 0.8.1
- CmdStanR documentation and vignettes: mc-stan.org/cmdstanr
- Use set_cmdstan_path() to set the path to CmdStan
- Use install_cmdstan() to install CmdStan

In the bigger build I can also confirm that I can use both set_cmdstan_path() and cmdstan_model() .

I'm still unsure why I'm unable to install cmdstanr the recommended way but in case anyone else out there is having a similar issue, installing it this way is successful.

[jberry47]

I think I know what the problem is.

I'm on a company computer that injects new SSL certificates between connections (MITMA), and the domain stan-dev.r-universe.dev is using HTTPS protocol whereas the domain cran.rstudio.com uses HTTP. Therefore I can get packages without SSL issues from CRAN, but when a connection is attempted using any HTTPS protocol, the certificate handshake is unsuccessful unless proper cacerts are in place.

A clue to this behavior is also in the repro curl command that works for me. I must use the -k flag to ignore certificate verification to download the package. If the flag is not used, the connection breaks and the build fails.

Really appreciate everyone's response to this issue! This is a problem specific to me and my use case, and not a problem with the package in any way.