FIX: undesired python version used for reticulate when working with radian

[code-instable]

problem

When using Radian IDE, which default used python version may differ from the selected one, reticulate will force the use of the python version used for radian instead of the one provided by RETICULATE_PYTHON. The proposed solution is to fetch the radian matching the desired python version for reticulate.

the rix configuration (minimal)

here is an almost minimal setup to reproduce the error:

rix(
    date = "2025-04-11",
    r_pkgs = c("reticulate"),
    git_pkgs = list(list(package_name = "somebm", repo_url = "https://github.com/cran/somebm", commit = "03584b24dee8081f08a0cde456edcc8419aba494")),
    py_conf = list(
        py_version = "3.13",
        py_pkgs = c("numpy")
    ),
    ide = "radian",
    project_path = ".",
    overwrite = FALSE,
    print = TRUE,
    message_type = "simple"
)

the following code is then executed inside the nix environment

Sys.getenv("RETICULATE_PYTHON") # we will see that loading reticulate changes the variable

require(reticulate)

reticulate::py_config()
reticulate::py_version()
reticulate::py_exe()
reticulate::py_list_packages()

Sys.getenv("RETICULATE_PYTHON")

collision between radian's python version and reticulate's python version

When entering nix-shell while using radian IDE, reticulate automatically falls back to the python version used by radian, as we can see in the following example:

(note that RETICULATE_PYTHON switches from the correct desired version to radian's one on loading the reticulate package)

❯ nix-shell

[nix-shell:~/github-contributions/rix/custom/with_py_pkgs-old]$ radian
R version 4.5.0 (2025-04-11) -- "How About a Twenty-Six"
Platform: aarch64-apple-darwin24.5.0 (64-bit)

r$> Sys.getenv("RETICULATE_PYTHON")

    require(reticulate)

    reticulate::py_config()
    reticulate::py_version()
    reticulate::py_exe()
    reticulate::py_list_packages()

    Sys.getenv("RETICULATE_PYTHON")
[1] "/nix/store/f55hdkj3px5v288jv6yai2gh5xrlj848-python3-3.13.2/bin/python"
Loading required package: reticulate
python:         /nix/store/3p3agnh4792y8bgjp0s8xx5bym1ahpks-python3-3.12.9/bin/python3.12
libpython:      /nix/store/3p3agnh4792y8bgjp0s8xx5bym1ahpks-python3-3.12.9/lib/libpython3.12.dylib
pythonhome:     /nix/store/3p3agnh4792y8bgjp0s8xx5bym1ahpks-python3-3.12.9:/nix/store/3p3agnh4792y8bgjp0s8xx5bym1ahpks-python3-3.12.9
version:        3.12.9 (main, Feb  4 2025, 14:38:38) [Clang 19.1.7 ]
numpy:           [NOT FOUND]

NOTE: Python version was forced by the current process
[1] ‘3.12’
[1] "/nix/store/3p3agnh4792y8bgjp0s8xx5bym1ahpks-python3-3.12.9/bin/python3.12"
             package     version                  requirement
1            appnope       0.1.3               appnope==0.1.3
2          asttokens       3.0.0             asttokens==3.0.0
3               comm       0.2.2                  comm==0.2.2
4          decorator       5.1.1             decorator==5.1.1
5          executing       2.2.0             executing==2.2.0
6          ipykernel      6.29.5            ipykernel==6.29.5
7            ipython      8.32.0              ipython==8.32.0
8               jedi      0.19.2                 jedi==0.19.2
9     jupyter_client       8.6.3        jupyter_client==8.6.3
10      jupyter_core       5.7.2          jupyter_core==5.7.2
11 matplotlib-inline       0.1.7     matplotlib-inline==0.1.7
12      nest_asyncio       1.6.0          nest_asyncio==1.6.0
13             numpy       2.2.3                 numpy==2.2.3
14         packaging        24.2              packaging==24.2
15             parso       0.8.4                 parso==0.8.4
16           pexpect       4.9.0               pexpect==4.9.0
17      platformdirs       4.3.6          platformdirs==4.3.6
18    prompt_toolkit      3.0.50       prompt_toolkit==3.0.50
19            psutil       7.0.0                psutil==7.0.0
20        ptyprocess       0.7.0            ptyprocess==0.7.0
21         pure_eval       0.2.3             pure_eval==0.2.3
22          Pygments      2.19.1             Pygments==2.19.1
23   python-dateutil 2.9.0.post0 python-dateutil==2.9.0.post0
24             pyzmq      26.2.1                pyzmq==26.2.1
25               six      1.17.0                  six==1.17.0
26        stack_data       0.6.3            stack_data==0.6.3
27              toml      0.10.2                 toml==0.10.2
28           tornado       6.4.2               tornado==6.4.2
29         traitlets      5.14.3            traitlets==5.14.3
30           wcwidth      0.2.13              wcwidth==0.2.13
[1] "/nix/store/3p3agnh4792y8bgjp0s8xx5bym1ahpks-python3-3.12.9/bin/python3.12"

only when using radian

However, it still works perfectly fine when using Rscript:

=== Executing R script in with_py_pkgs-old at Tue Jun  3 21:02:50 CEST 2025 ===
Working directory: /Users/instable/github-contributions/rix/custom/with_py_pkgs-old
=== Output ===
[1] "/nix/store/f55hdkj3px5v288jv6yai2gh5xrlj848-python3-3.13.2/bin/python"
Loading required package: reticulate
python:         /nix/store/f55hdkj3px5v288jv6yai2gh5xrlj848-python3-3.13.2/bin/python
libpython:      /nix/store/f55hdkj3px5v288jv6yai2gh5xrlj848-python3-3.13.2/lib/libpython3.13.dylib
pythonhome:     /nix/store/f55hdkj3px5v288jv6yai2gh5xrlj848-python3-3.13.2:/nix/store/f55hdkj3px5v288jv6yai2gh5xrlj848-python3-3.13.2
version:        3.13.2 (main, Feb  4 2025, 14:51:09) [Clang 19.1.7 ]
numpy:          /nix/store/wd5v6f1j439170iszqb0n5ln8qvyl31g-python3.13-numpy-2.2.3/lib/python3.13/site-packages/numpy
numpy_version:  2.2.3

NOTE: Python version was forced by RETICULATE_PYTHON
[1] ‘3.13’
[1] "/nix/store/f55hdkj3px5v288jv6yai2gh5xrlj848-python3-3.13.2/bin/python"
             package     version                  requirement
1            appnope       0.1.3               appnope==0.1.3
2          asttokens       3.0.0             asttokens==3.0.0
3               comm       0.2.2                  comm==0.2.2
4          decorator       5.1.1             decorator==5.1.1
5          executing       2.2.0             executing==2.2.0
6          ipykernel      6.29.5            ipykernel==6.29.5
7            ipython      8.32.0              ipython==8.32.0
8               jedi      0.19.2                 jedi==0.19.2
9     jupyter_client       8.6.3        jupyter_client==8.6.3
10      jupyter_core       5.7.2          jupyter_core==5.7.2
11 matplotlib-inline       0.1.7     matplotlib-inline==0.1.7
12      nest_asyncio       1.6.0          nest_asyncio==1.6.0
13             numpy       2.2.3                 numpy==2.2.3
14         packaging        24.2              packaging==24.2
15             parso       0.8.4                 parso==0.8.4
16           pexpect       4.9.0               pexpect==4.9.0
17      platformdirs       4.3.6          platformdirs==4.3.6
18    prompt_toolkit      3.0.50       prompt_toolkit==3.0.50
19            psutil       7.0.0                psutil==7.0.0
20        ptyprocess       0.7.0            ptyprocess==0.7.0
21         pure_eval       0.2.3             pure_eval==0.2.3
22          Pygments      2.19.1             Pygments==2.19.1
23   python-dateutil 2.9.0.post0 python-dateutil==2.9.0.post0
24             pyzmq      26.2.1                pyzmq==26.2.1
25               six      1.17.0                  six==1.17.0
26        stack_data       0.6.3            stack_data==0.6.3
27              toml      0.10.2                 toml==0.10.2
28           tornado       6.4.2               tornado==6.4.2
29         traitlets      5.14.3            traitlets==5.14.3
30           wcwidth      0.2.13              wcwidth==0.2.13
[1] "/nix/store/f55hdkj3px5v288jv6yai2gh5xrlj848-python3-3.13.2/bin/python"
=== Execution completed with exit status: 0 ===

proposed solution

The proposed solution is to match radian's python version with the python version from the user-defined config within the override of radianWrapper:

wrapped_pkgs = pkgs.radianWrapper.override {
    packages = [ somebm rpkgs ];
    radian = pkgs.python313Packages.radian
};

the code produces the following default.nix:

# This file was generated by the {rix} R package v0.17.0 on 2025-06-03
# with following call:
# >rix(date = "2025-04-11",
#  > r_pkgs = c("reticulate"),
#  > git_pkgs = list(list(package_name = "somebm",
#  > repo_url = "https://github.com/cran/somebm",
#  > commit = "03584b24dee8081f08a0cde456edcc8419aba494")),
#  > py_conf = list(py_version = "3.13",
#  > py_pkgs = c("numpy")),
#  > ide = "radian",
#  > project_path = ".",
#  > overwrite = FALSE,
#  > print = TRUE,
#  > message_type = "simple",
#  > r_ver = "4.5.0")
# It uses the `rstats-on-nix` fork of `nixpkgs` which provides improved
# compatibility with older R versions and R packages for Linux/WSL and
# Apple Silicon computers.
# Report any issues to https://github.com/ropensci/rix
let
 pkgs = import (fetchTarball "https://github.com/rstats-on-nix/nixpkgs/archive/2025-04-11.tar.gz") {};
 
  rpkgs = builtins.attrValues {
    inherit (pkgs.rPackages) 
      reticulate;
  };
 
    somebm = (pkgs.rPackages.buildRPackage {
      name = "somebm";
      src = pkgs.fetchgit {
        url = "https://github.com/cran/somebm";
        rev = "03584b24dee8081f08a0cde456edcc8419aba494";
        sha256 = "sha256-QqSk/bmbDpquHJfHqimCcJcj/4QKdqQLBLqKCRZzn6s=";
      };
      propagatedBuildInputs = builtins.attrValues {
        inherit (pkgs.rPackages) ;
      };
    });
   
  pyconf = builtins.attrValues {
    inherit (pkgs.python313Packages) 
      pip
      ipykernel
      numpy;
  };
   
  system_packages = builtins.attrValues {
    inherit (pkgs) 
      glibcLocales
      nix
      python313
      R;
  };
 
  wrapped_pkgs = pkgs.radianWrapper.override {
    packages = [ somebm rpkgs ];
    radian = pkgs.python313Packages.radian
  };
 
  shell = pkgs.mkShell {
    LOCALE_ARCHIVE = if pkgs.system == "x86_64-linux" then "${pkgs.glibcLocales}/lib/locale/locale-archive" else "";
    LANG = "en_US.UTF-8";
    LC_ALL = "en_US.UTF-8";
    LC_TIME = "en_US.UTF-8";
    LC_MONETARY = "en_US.UTF-8";
    LC_PAPER = "en_US.UTF-8";
    LC_MEASUREMENT = "en_US.UTF-8";
    RETICULATE_PYTHON = "${pkgs.python313}/bin/python";

    buildInputs = [ somebm rpkgs pyconf system_packages wrapped_pkgs ];
    
  }; 
in
  {
    inherit pkgs shell;
  }

running the nix-shell now produces the desired result as "forced by the current process":

(notice that the RETICULATE_PYTHON does not change here)

❯ nix-shell

[nix-shell:~/github-contributions/rix/custom/with_py_pkgs]$ radian
R version 4.5.0 (2025-04-11) -- "How About a Twenty-Six"
Platform: aarch64-apple-darwin24.5.0 (64-bit)

r$> Sys.getenv("RETICULATE_PYTHON")

    require(reticulate)

    reticulate::py_config()
    reticulate::py_version()
    reticulate::py_exe()
    reticulate::py_list_packages()

    Sys.getenv("RETICULATE_PYTHON")
[1] "/nix/store/f55hdkj3px5v288jv6yai2gh5xrlj848-python3-3.13.2/bin/python"
Loading required package: reticulate
python:         /nix/store/f55hdkj3px5v288jv6yai2gh5xrlj848-python3-3.13.2/bin/python3.13
libpython:      /nix/store/f55hdkj3px5v288jv6yai2gh5xrlj848-python3-3.13.2/lib/libpython3.13.dylib
pythonhome:     /nix/store/f55hdkj3px5v288jv6yai2gh5xrlj848-python3-3.13.2:/nix/store/f55hdkj3px5v288jv6yai2gh5xrlj848-python3-3.13.2
version:        3.13.2 (main, Feb  4 2025, 14:51:09) [Clang 19.1.7 ]
numpy:          /nix/store/wd5v6f1j439170iszqb0n5ln8qvyl31g-python3.13-numpy-2.2.3/lib/python3.13/site-packages/numpy
numpy_version:  2.2.3

NOTE: Python version was forced by the current process
[1] ‘3.13’
[1] "/nix/store/f55hdkj3px5v288jv6yai2gh5xrlj848-python3-3.13.2/bin/python3.13"
             package     version                  requirement
1            appnope       0.1.3               appnope==0.1.3
2          asttokens       3.0.0             asttokens==3.0.0
3               comm       0.2.2                  comm==0.2.2
4          decorator       5.1.1             decorator==5.1.1
5          executing       2.2.0             executing==2.2.0
6          ipykernel      6.29.5            ipykernel==6.29.5
7            ipython      8.32.0              ipython==8.32.0
8               jedi      0.19.2                 jedi==0.19.2
9     jupyter_client       8.6.3        jupyter_client==8.6.3
10      jupyter_core       5.7.2          jupyter_core==5.7.2
11 matplotlib-inline       0.1.7     matplotlib-inline==0.1.7
12      nest_asyncio       1.6.0          nest_asyncio==1.6.0
13             numpy       2.2.3                 numpy==2.2.3
14         packaging        24.2              packaging==24.2
15             parso       0.8.4                 parso==0.8.4
16           pexpect       4.9.0               pexpect==4.9.0
17      platformdirs       4.3.6          platformdirs==4.3.6
18    prompt_toolkit      3.0.50       prompt_toolkit==3.0.50
19            psutil       7.0.0                psutil==7.0.0
20        ptyprocess       0.7.0            ptyprocess==0.7.0
21         pure_eval       0.2.3             pure_eval==0.2.3
22          Pygments      2.19.1             Pygments==2.19.1
23   python-dateutil 2.9.0.post0 python-dateutil==2.9.0.post0
24             pyzmq      26.2.1                pyzmq==26.2.1
25               six      1.17.0                  six==1.17.0
26        stack_data       0.6.3            stack_data==0.6.3
27              toml      0.10.2                 toml==0.10.2
28           tornado       6.4.2               tornado==6.4.2
29         traitlets      5.14.3            traitlets==5.14.3
30           wcwidth      0.2.13              wcwidth==0.2.13
[1] "/nix/store/f55hdkj3px5v288jv6yai2gh5xrlj848-python3-3.13.2/bin/python3.13"

if no py_conf is provided, then we do not add the python version override:

# This file was generated by the {rix} R package v0.17.0 on 2025-06-03
# with following call:
# >rix(date = "2025-04-11",
#  > r_pkgs = c("reticulate"),
#  > git_pkgs = list(list(package_name = "somebm",
#  > repo_url = "https://github.com/cran/somebm",
#  > commit = "03584b24dee8081f08a0cde456edcc8419aba494")),
#  > ide = "radian",
#  > project_path = ".",
#  > overwrite = FALSE,
#  > print = TRUE,
#  > message_type = "simple",
#  > r_ver = "4.5.0")
# It uses the `rstats-on-nix` fork of `nixpkgs` which provides improved
# compatibility with older R versions and R packages for Linux/WSL and
# Apple Silicon computers.
# Report any issues to https://github.com/ropensci/rix
let
 pkgs = import (fetchTarball "https://github.com/rstats-on-nix/nixpkgs/archive/2025-04-11.tar.gz") {};
 
  rpkgs = builtins.attrValues {
    inherit (pkgs.rPackages) 
      reticulate;
  };
 
    somebm = (pkgs.rPackages.buildRPackage {
      name = "somebm";
      src = pkgs.fetchgit {
        url = "https://github.com/cran/somebm";
        rev = "03584b24dee8081f08a0cde456edcc8419aba494";
        sha256 = "sha256-QqSk/bmbDpquHJfHqimCcJcj/4QKdqQLBLqKCRZzn6s=";
      };
      propagatedBuildInputs = builtins.attrValues {
        inherit (pkgs.rPackages) ;
      };
    });
      
  system_packages = builtins.attrValues {
    inherit (pkgs) 
      glibcLocales
      nix
      R;
  };
 
  wrapped_pkgs = pkgs.radianWrapper.override {
    packages = [ somebm rpkgs ];
    
  };
 
  shell = pkgs.mkShell {
    LOCALE_ARCHIVE = if pkgs.system == "x86_64-linux" then "${pkgs.glibcLocales}/lib/locale/locale-archive" else "";
    LANG = "en_US.UTF-8";
    LC_ALL = "en_US.UTF-8";
    LC_TIME = "en_US.UTF-8";
    LC_MONETARY = "en_US.UTF-8";
    LC_PAPER = "en_US.UTF-8";
    LC_MEASUREMENT = "en_US.UTF-8";
    
    buildInputs = [ somebm rpkgs system_packages wrapped_pkgs ];
    
  }; 
in
  {
    inherit pkgs shell;
  }

note that I don't know how to properly remove the added newline when there is no py_conf without using a if/else statement, so any suggestion is much appreciated

suggested edits

(+) function: set_radian_python_version_override
(~) generate_wrapped_pkgs

• (+) added py_conf and flag_py_conf as arguments
• (+) added string format '%s' bellow the packages

TODO: remove whitespaces and empty line when there is no py_conf
(I don't know how to do)

potential edge case

the desired python version is much older than the one used for radian

[b-rodrigues]

The PR is still in draft, is that because you want to continue to work on it, or did you forget to set it to ready? In any case, could you fix the unit tests please?

[code-instable]

The PR is still in draft because I want to know whether things have to change, and if it is a good approach as well

I am not good with unit test, so I don't really know what to change, I will take a look at it

Also I don't know how to trim the added white spaces when there is no pyconf, helping me on this would be much appreciated

[b-rodrigues]

Ok let’s forget the unit tests then: the overall approach seems good. Put the PR in ready for review, and I will thouroughly test it. If it’s fine, I’ll merge and fix the tests and the whitespace issue myself.

[code-instable]

the radian package has been promoted to a top package 3 days ago. I may have to try if this resolved by itself the issue, which means the code provided is now breaking when it comes to the unstable branch

[b-rodrigues]

Have you had a chance to check whether this is still an issue?

[code-instable]

There is still the need to change something, I will do it later, as radian moved from a python package to a regular package

Which also breaks compatibility with different releases of nixpkgs (Should it be taken into account ?)