Very ugly attempt to fix issues with pickling functions for test_doc_examples.py in github CI.

Part 1: run example code by writing it to a file and loading it as a module with importlib instead of using exec.

Part 2: In the example notebook, add PWD to the PYTHONPATH in each remote_info object or dict, counting on the fact that the test framework will chdir to tmp_path, where it wrote the example code as a file (see Part 1) before the example is run, so PWD is the right place for the extracted code's module.

Pickling works locally, but remote jobs are using the global wfl instead of the one in the directory where pytest is being run from.  The remoteinfo_env fixture deals with this by modifying the PYTHONPATH in the remote_info env_vars field for other remote run tests, but adding that to the example would make it even uglier than these modifications already do.

Should the RemoteInfo constructor itself be able to add such information (e.g. extra bits for the PYTHONPATH env var) from an env var (maybe something like PYTEST_REMOTE_INFO_EXTRAS, or a more PYTHONPATH specific var name), which pytest can then use without polluting the example itself?

Author: bernstei

docs/source/examples.daisy_chain_mlip_fitting.ipynb

@@ -80,6 +80,7 @@
     "from wfl.utils.configs import atomization_energy\n",
     "from wfl.fit import error\n",
     "import wfl.map\n",
+    "from pathlib import Path\n",
     "\n",
     "from expyre.resources import Resources"
    ]
@@ -244,7 +245,8 @@
     "    \"check_interval\": 5,\n",
     "    \"num_inputs_per_queued_job\" :20,\n",
     "    \"pre_cmds\": ['echo \"-------------------------------------\"', \"which wfl\"],\n",
-    "    \"post_cmds\": ['echo \"_______________________________\"', \"which wfl\"]\n",
+    "    \"post_cmds\": ['echo \"_______________________________\"', \"which wfl\"],\n",
+    "    \"env_vars\": [f'PYTHONPATH={Path.cwd()}:$PYTHONPATH']\n",
     "}\n"
    ]
   },
@@ -464,6 +466,7 @@
     "        \"num_cores\" : 2,\n",
     "        \"partitions\" : \"stndard\"}, \n",
     "    \"check_interval\": 5, \n",
+    "    \"env_vars\": [f'PYTHONPATH={Path.cwd()}:$PYTHONPATH']\n",
     "}\n"
    ]
   },
@@ -548,7 +551,8 @@
     "    job_name = \"gap-eval\",\n",
     "    resources = resources,\n",
     "    check_interval=10, \n",
-    "    input_files=[\"gap.xml*\"])\n"
+    "    input_files=[\"gap.xml*\"],\n",
+    "    env_vars=[f'PYTHONPATH={Path.cwd()}:$PYTHONPATH'])\n",
    ]
   },
   {

tests/test_doc_examples.py

@@ -39,7 +39,15 @@ def test_example(tmp_path, nb_file, idx_execute, monkeypatch, needs_expyre, expy
 
     monkeypatch.chdir(tmp_path)
     try:
-        exec(code, globals())
+        ## exec(code, globals())
+        run_test_mod_name = f"run_test_{nb_file.replace('.', '_')}"
+        import sys, importlib
+        with open(run_test_mod_name + ".py", "w") as fout:
+            fout.write(code)
+        sys.path.append(str(tmp_path))
+        run_test_mod = importlib.import_module(run_test_mod_name)
+        sys.path.pop()
+
     except Exception as exc:
         import traceback, re
         tb_str = traceback.format_exc()