Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Grid locator #20

Merged
merged 13 commits into from
Mar 15, 2024
Merged

Grid locator #20

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
ba34313
feat(gridsearch): Define GridLocator object
Jacob-Stevens-Haas Mar 6, 2024
e341495
cln: Extract gridsearch functionality (mostly) to subpackage
Jacob-Stevens-Haas Mar 6, 2024
c51cea3
type(gridsearch): Clarify types in SkinnySpec
Jacob-Stevens-Haas Mar 6, 2024
f3425fe
feat(gridsearch): Added find_gridpoints()
Jacob-Stevens-Haas Mar 7, 2024
abfa77f
test(gridsearch): Test find_gridpoints with GridLocator
Jacob-Stevens-Haas Mar 10, 2024
7872478
types: Check more files in CI
Jacob-Stevens-Haas Mar 10, 2024
b0b3517
test: move gridsearch tests to their own file, move NestedDict into core
Jacob-Stevens-Haas Mar 10, 2024
ff0a8bc
test(gridsearch): Clarify and fix test_find_gridpoints
Jacob-Stevens-Haas Mar 10, 2024
855458d
fix: Apply OR correctly in matching param dicts
Jacob-Stevens-Haas Mar 11, 2024
dc97a46
fix(gridsearch): enable matching callable criteria
Jacob-Stevens-Haas Mar 11, 2024
3b954fc
feat(gridsearch): Enable locating a set of metrics or keep_axes
Jacob-Stevens-Haas Mar 12, 2024
adf2d08
tst(gridsearch): Add integration test
Jacob-Stevens-Haas Mar 12, 2024
d17275e
feat(gridsearch): Use find_gridpoints() in gridsearch.run()
Jacob-Stevens-Haas Mar 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,13 @@ addopts = '-m "not slow"'
markers = ["slow"]

[tool.mypy]
files = ["src/gen_experiments/__init__.py", "src/gen_experiments/utils.py"]
files = [
"src/gen_experiments/__init__.py",
"src/gen_experiments/utils.py",
"src/gen_experiments/gridsearch/typing.py",
"tests/test_all.py",
"tests/test_gridsearch.py",
]

[[tool.mypy.overrides]]
module="auto_ks.*"
Expand Down
2 changes: 1 addition & 1 deletion src/gen_experiments/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

import numpy as np
from numpy.typing import NDArray
from pysindy import BaseDifferentiation, FiniteDifference, SINDy # type: ignore
from pysindy import BaseDifferentiation, FiniteDifference, SINDy

from . import gridsearch, odes, pdes
from .utils import SINDyTrialData, make_model # noqa: F401
Expand Down
104 changes: 39 additions & 65 deletions src/gen_experiments/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,19 @@
from collections.abc import Iterable
from typing import TypeVar

import numpy as np
import pysindy as ps

from gen_experiments.data import _signal_avg_power
from gen_experiments.gridsearch.typing import (
GridLocator,
SeriesDef,
SeriesList,
SkinnySpecs,
)
from gen_experiments.plotting import _PlotPrefs
from gen_experiments.utils import FullSINDyTrialData, NestedDict, SeriesDef, SeriesList
from gen_experiments.typing import NestedDict
from gen_experiments.utils import FullSINDyTrialData

T = TypeVar("T")
U = TypeVar("U")
Expand Down Expand Up @@ -41,77 +49,43 @@ def addn(x):


plot_prefs = {
"test": _PlotPrefs(True, False, ({"sim_params.t_end": 10},)),
"test": _PlotPrefs(),
"test-absrel": _PlotPrefs(
True, _convert_abs_rel_noise, ({"sim_params.noise_abs": 1},)
True, _convert_abs_rel_noise, GridLocator(..., {("sim_params.noise_abs", (1,))})
),
"test-absrel2": _PlotPrefs(
True,
_convert_abs_rel_noise,
(
{"sim_params.noise_abs": 0.1},
{"sim_params.noise_abs": 0.5},
{"sim_params.noise_abs": 1},
{"sim_params.noise_abs": 2},
{"sim_params.noise_abs": 4},
{"sim_params.noise_abs": 8},
GridLocator(
...,
(..., ...),
Copy link
Collaborator

@yb6599 yb6599 Mar 15, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does ... signify here in the GridLocator arguments? Is it meant to be taking all the metrics or is it empty?

Edit: I've read about the Ellipsis object.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Yeah, I wanted to borrow some notation from numpy that means "all elements". They're both what are called "singletons". Only one NoneType object exists in a python program, just like only one EllipsisType object ever exists.

I suppose I could have used None instead. That may be an arguably better way, I just wanted to use an object that people associate with "all" of something.

(
{"sim_params.noise_abs": 0.1},
{"sim_params.noise_abs": 0.5},
{"sim_params.noise_abs": 1},
{"sim_params.noise_abs": 2},
{"sim_params.noise_abs": 4},
{"sim_params.noise_abs": 8},
),
),
),
"test-absrel3": _PlotPrefs(
"absrel-newloc": _PlotPrefs(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For running the experiments, which one of the plot_prefs do I use? test_absrel12 or absrel-newloc?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

absrel-newloc, but double check me that it's meaning is clear, given the GridLocator argument

True,
_convert_abs_rel_noise,
(
{
"sim_params.noise_abs": 1,
"diff_params.smoother_kws.window_length": 15,
},
{"sim_params.noise_abs": 1, "diff_params.meas_var": 1},
{"sim_params.noise_abs": 1, "diff_params.alpha": 1e-2},
GridLocator(
["coeff_mse", "coeff_f1"],
(..., (2, 3, 4)),
(
{"diff_params.kind": "kalman", "diff_params.alpha": None},
{
"diff_params.kind": "kalman",
"diff_params.alpha": lambda a: isinstance(a, int),
},
{"diff_params.kind": "trend_filtered"},
{"diff_params.diffcls": "SmoothedFiniteDifference"},
),
),
),
"test-absrel4": _PlotPrefs(
True,
_convert_abs_rel_noise,
(
{
"sim_params.noise_abs": 1,
"diff_params.smoother_kws.window_length": 15,
},
{"sim_params.noise_abs": 1, "diff_params.meas_var": 1},
{"sim_params.noise_abs": 1, "diff_params.alpha": 1e0},
{
"sim_params.noise_abs": 2,
"diff_params.smoother_kws.window_length": 15,
},
{"sim_params.noise_abs": 2, "diff_params.meas_var": 4},
{"sim_params.noise_abs": 2, "diff_params.alpha": 1e-1},
),
),
"test-absrel5": _PlotPrefs(
True,
_convert_abs_rel_noise,
(
{
"sim_params.noise_abs": 1,
"diff_params.diffcls": "SmoothedFiniteDifference",
},
{"sim_params.noise_abs": 1, "diff_params.kind": "kalman"},
{"sim_params.noise_abs": 1, "diff_params.kind": "trend_filtered"},
{
"sim_params.noise_abs": 2,
"diff_params.diffcls": "SmoothedFiniteDifference",
},
{"sim_params.noise_abs": 2, "diff_params.kind": "kalman"},
{"sim_params.noise_abs": 2, "diff_params.kind": "trend_filtered"},
{
"sim_params.noise_abs": 4,
"diff_params.diffcls": "SmoothedFiniteDifference",
},
{"sim_params.noise_abs": 4, "diff_params.kind": "kalman"},
{"sim_params.noise_abs": 4, "diff_params.kind": "trend_filtered"},
),
{(0, 2), (3, 2), (0, 3), (3, 3), (0, 4), (3, 4)},
),
}
sim_params = {
"test": ND({"n_trajectories": 2}),
Expand Down Expand Up @@ -301,7 +275,7 @@ def addn(x):
"duration-absnoise": ["sim_params.t_end", "sim_params.noise_abs"],
"rel_noise": ["sim_params.t_end", "sim_params.noise_rel"],
}
grid_vals = {
grid_vals: dict[str, list[Iterable]] = {
"test": [[5, 10, 15, 20]],
"abs_noise": [[0.1, 0.5, 1, 2, 4, 8]],
"abs_noise-kalman": [[0.1, 0.5, 1, 2, 4, 8], [0.1, 0.5, 1, 2, 4, 8]],
Expand All @@ -319,7 +293,7 @@ def addn(x):
"lorenzk": ["plot", "plot", "max"],
"plot2": ["plot", "plot"],
}
diff_series = {
diff_series: dict[str, SeriesDef] = {
"kalman1": SeriesDef(
"Kalman",
diff_params["kalman"],
Expand Down Expand Up @@ -375,7 +349,7 @@ def addn(x):
[[5, 8, 12, 15]],
),
}
series_params = {
series_params: dict[str, SeriesList] = {
"test": SeriesList(
"diff_params",
"Differentiation Method",
Expand Down Expand Up @@ -434,7 +408,7 @@ def addn(x):
}


skinny_specs = {
skinny_specs: dict[str, SkinnySpecs] = {
"exp3": (
("sim_params.noise_abs", "diff_params.meas_var"),
((identity,), (identity,)),
Expand Down
5 changes: 3 additions & 2 deletions src/gen_experiments/data.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@
import numpy as np
import scipy

from gen_experiments.utils import Float1D, Float2D, GridsearchResultDetails
from gen_experiments.gridsearch.typing import GridsearchResultDetails
from gen_experiments.utils import Float1D, Float2D

INTEGRATOR_KEYWORDS = {"rtol": 1e-12, "method": "LSODA", "atol": 1e-12}
TRIALS_FOLDER = Path(__file__).parent.absolute() / "trials"
Expand Down Expand Up @@ -56,7 +57,7 @@ def gen_data(
noise_abs = 0.1
rng = np.random.default_rng(seed)
if x0_center is None:
x0_center = np.zeros((n_coord))
x0_center = np.zeros((n_coord), dtype=np.float_)
t_train = np.arange(0, t_end, dt, dtype=np.float_)
t_train_span = (t_train[0], t_train[-1])
if nonnegative:
Expand Down
25 changes: 0 additions & 25 deletions src/gen_experiments/debug.py
View file
Open in desktop

This file was deleted.

Loading
Loading