Feedback Survey: The New UI for Optuna Dashboard

[c-bata]

We are considering replacing the current UI of Optuna Dashboard with the new UI that is now available in beta. It’s available as an opt-in feature - simply launch the dashboard as usual and click the link to the beta UI. Please try it out and share your thoughts with us.

$ pip install optuna-dashboard>=0.9.0
$ optuna-dashboard sqlite:///db.sqlite3

If you have any comments or suggestions for improvement regarding the new UI, please let us know by replying to this post.

[okaikov]

when running the new UI there is a bug in the visualization of order of the trials.

[okaikov]

I'm unable to upload a screenshot, but for example I get the following studies order:
193, 194
197, 199,
198, 196

when max study number is: 391

[c-bata]

Hmm... Could you share more detailed information and/or steps to reproduce with us?
https://github.com/optuna/optuna-dashboard/issues/new?assignees=&labels=bug&template=bug_report.yml

[okaikov]

I will try to reproduce as a simple/basic scenario.
in the meantime as a workaround, can you please add an option to view the studies as a list?

[c-bata]

@okaikov Sorry for the late reply. I confirmed the same problem when using MySQL backend. It will be fixed at the next optuna-dashboard release.

• Sort studies by study_id optuna#4414
• Sort studies by study_id #391

[okaikov]

I can confirm that the issue is resolved in v0.9.0b6.
thx for the fix :)

[nurettin]

This is how I installed the new version of optuna-dashboard:

pip install "optuna-dashboard>=0.9.0b2"

This is how I run it:

optuna-dashboard sqlite:///db.sqlite3 --port 8081 --host 0.0.0.0

This is the error I get:

python3.11/site-packages/optuna_dashboard/_serializer.py", line 17, in <module>
    from .artifact._backend import list_trial_artifacts
ModuleNotFoundError: No module named 'optuna_dashboard.artifact'

[c-bata]

Thank you for reporting!
I found an issue in sdist package. I will release the fixed version tomorrow 🙇

[c-bata]

@nurettin Thank you again for your report.
I've issued a new release v0.9.0b5 to resolve the problem.
https://github.com/optuna/optuna-dashboard/releases/tag/v0.9.0b5

[nurettin]

It seems to be working for me (after deleting browser cache with ctrl-shift-r)
Thanks for the release. Looking good, especially like the best trials section.

[nurettin]

I left my studies running overnight and

in the morning I wasn't able to start optuna dashboard (the 0.9.0b5 version)

I have this many trials in the postgresql database:

scan_service=# select count(1) from trial_values;
 33634

After I drop back to version <0.9 I was able to open the dashboard.

[c-bata]

Thank you for reporting!
Could you also share the following results with us?

• Number of studies: select count(1) from studies;
• Number of trials: select count(1) from trials;
• Number of trials in the study that you tried to open: select count(1) from trials where study_id=XXX

[nurettin]

Number of studies

scan_service=# select count(1) from studies;
    17

Number of trials

scan_service=# select count(1) from trials;
 27023

Number of trials per study

scan_service=# select count(1), study_id from trials group by study_id;
    20 |       36
  1620 |       31
    20 |       34
  1100 |       41
 10473 |       40
  9900 |       37
    40 |       32
    40 |       35
  1050 |       45
   160 |       38
   240 |        2
    30 |       15
   160 |        6
   660 |       27
   160 |       39
   990 |       44
   390 |       24

[c-bata]

Thanks!
I will try to make it work for 10,000 trials. I may disable some features depending on the number of trials though.

[hdx-18]

Is there a way to name the objectives? That would improve readability of the plots.

[c-bata]

@hdx-18 Thank you for your question! Could you try using set_objective_names(study, ["objective 1", "objective 2"]) like the following?

import optuna
from optuna_dashboard import set_objective_names

def objective(trial):
    ...

study = optuna.create_study(..., directions=["minimize", "maximize"])
set_objective_names(study, ["objective 1", "objective 2"])
study.optimize(objective, ...)

[hdx-18]

Yes, that worked! Thanks for the quick reply 😄

[lebigot]

The new UI looks slicker and more importantly, information is much better organized (through the left-hand side menu bar).

Now, the trial table is less legible than before: it would be useful to have the parameters in a table (instead of listed as a long string associated with each trial). This makes comparing trials much easier.

[c-bata]

Thank you for your feedback!

TrialTable is actually unchanged from the previous UI. Whether the parameters are displayed as JSON strings or as independent columns for each parameter depends on whether the search space changes dynamically. By the next release, we will address this so that each parameter can be displayed as an independent column even if the search space changes dynamically.

[lebigot]

It would also be useful if the trial table remembered the sorting order of the columns (otherwise we have to constantly change it when we get back to it).

[c-bata]

Sounds good! Thank you for your feedback.

[lebigot]

Also, the parallel plot is very hard to use: the lines are so thin that it's very hard to see their color. Maybe using thicker lines would help?

Also, offering as an option to have a more varied palette (with multiple colors) would help spot lines associated to better objective values.

[c-bata]

@keisuke-umezawa May I have your opinion on this?

[keisuke-umezawa]

Actually, the implementation is aligned with optuna. But, as you said, we can use more bold line for both optuna and dashboard.

[lebigot]

There are a few usability issues with the Intermediate Values pane:

1. Zooming on part of the graph is cancelled as soon as a new value is displayed. This is a real problem when running many jobs in parallel. It would be more useful if the zoom remained even as values are added to the graph.
2. Selecting the first curve of intermediate values is made impossible by the plotly overlay that gets displays when the cursor enter the Intermediate Values pane. It would be thus useful to have the list of graphed lines somewhere else.

[nucflash]

Some feedback:

• Contour plots are super useful, but I need them in the vicinity of the parameter importance plot; I first observe what are important parameters in the parameter importance plot and then pick a couple to draw in the contour plot. Having the two plots in separate pages, introduces a lot of back-n-forth
• Trials (list) I'd like the ability to a) reverse sort the by trial id, and b) filter and show only Best Trials
• Trials (table) not sure if I'm fond of the details page navigating to a different page. In the current version I expand and look at a few trials details simultaneously but the new UI makes it difficult (and adds more back-and-forth)
• Trials (table) it would nice if we could apply filters on the values of the objectives (like >, <) (on top the of state filters)
• In the Pareto Front plot I seem unable to get the info of a certain point, but I get this issue in the standard version too so this is not an issue with beta
• Parameter importance plot; I like the fact that I can see the parameter importance over multiple objectives, but I really miss the detail I get per individual objective. Also: a) depending on how the importances are distributed we can get 2-3 parameters with very long bars and all others with negligible bars, this reduces readability (and usability); b) When we have many hyperparameters it is very difficult to point to a bar and see what is the label of the bar (we don't have axis labels for all parameters since they don't fit), it would be nice if this plot was laid out as vertical (portrait; taller) instead of horizontal (landscape; wider) that it is currently; it would be extra nice if we could resize the plot from the UI

[2-5]

In the slice dropdown, the word Param is repeated in front of every parameter and it makes them hard to scan quickly. The Param prefix should be removed.

[2-5]

The new beta dashboard doesn't have an auto-refresh disable control, so if I zoom in on a chart, after 10 seconds the zoom resets due to refreshing.

In general, I don't think auto-refresh should reset chart settings.

[c-bata]

Thanks for the feedback! You can disable auto-refresh by clicking "Live Update" button as shown below:

In general, I don't think auto-refresh should reset chart settings.

Agree. This is just an implementation issue. I cannot find the solution to prevent it when using plotly.js.

[2-5]

Agree. This is just an implementation issue. I cannot find the solution to prevent it when using plotly.js.

plotly.js supports a uirevision attribute which is supposed to fix this problem. If this attribute will not change on an update, the UI will not be reset.

https://iotespresso.com/preserving-zoom-settings-across-data-refresh-in-dash/
https://plotly.com/javascript/uirevision/

[c-bata]

Thank you for letting me know! I just tried it and it seems to work fine. I will make a new patch release with this fix soon.

[c-bata]

I've just release optuna-dashboard v0.9.1. Thank you again for your help!
#436

[2-5]

Great, thanks for the quick fix!

There is a small regression - if you change the active displayed parameter, for example by selecting a different one in the Slice dropdown, in this case the plotly UI state should be reset, because the new param can have a vastly different value range, and if you keep the old zoom you will be looking at empty space.

[tonyofa]

Really like the new UI, and the tool overall is fantastic! Using Optuna has helped me understand the training process "under the hood'. Just wanted to say thanks, really but like the new UI better than the old.

[okaikov]

when moving to analytics tab, getting a blank white page

[lebigot]

Thanks for the Intermediate Values curves that remain selected even with live updates!

Now, the Analytics selections (at least for the Parallel Coordinates) get reset each time we go back to the page. This is inconvenient, as it is quite natural to switch between the History and Analytics sections so as to check and then select the most impactful hyper-parameters. Currently, this cannot be done easily because the Parallel Coordinate selection gets reset each time.

[lebigot]

In the Trials List screen, using Shift allows multiple trials to be displayed side by side.

I strongly suggest to move the Intermediate Values to the end of the displayed information: the most interesting part of this side-by-side feature is to compare models, and this is very difficult when comparable information (such as the model parameters) is not aligned between trials (currently due to the varying number of Intermediate Values between trials).

A better option yet would be to display the side-by-side information in a table, so that each entry (Parameters, User Parameters, Duration, etc.) can be compared between trials. Parameter that are not set during a trial can have an empty value in this table.

[lebigot]

Similarly, comparing the parameters in the Trials (Table) view is very hard, as soon as they are displayed as a list ("log2_bn_size: 3, num_hidden_head_layers: 7, num_init_features: 3, num_layers_block0: 6…"): what about putting them in a table, or at least giving the option of doing so?

Some cells in the table can be empty, if the corresponding parameters are not set for a specific trial (such as the number of neurons in layer 3 if there are only 2 layers).

[miykael]

The new UI looks great! And optuna overall is a great tool!

With regards to the visualizations, it would be nice if the Slice would also have a button to turn on "Log x scale" and that the Contour plot would have both "Log x" and "Log y" scale. Otherwise these visualizations become less useful for parameters that are covering multiple magnitudes. Similar log-scale feature would be needed for the Parallel Coordinate figure.

Not sure if for any of these approaches, the log=True parameter from the suggest_float function could be used to infer log-scaling.

[arealglitterboy]

I had to fail a trial yesterday and I couldn't find where to do that in the new UI. The trial table in the old UI had a toggle state where I could fail a trial if I needed to, but I can't seem to find that option here

[c-bata]

Thank you for your feedback! I will add a button to fail a trial until our next beta release 👍

[c-bata]

I addressed your feedback at #458.

[c-bata]

Released at https://github.com/optuna/optuna-dashboard/releases/tag/v0.10.0b1.

$ pip install "optuna-dashboard==0.10.0b1"

[AMMJ93]

Really like the new layout, but after 6000 trials the dashboard is just unusable. If it loads, it takes forever. Maybe it a good idea to have a page for such big studies that only shows one plot at a time. For example only the hyperparameter importance. Might that improve the usability?

[arealglitterboy]

I would also like more thorough filtering options, it would be nice to filter plots to only running/top 10 runs. Could help improve performance. Filtering out pruned trials is quite limited in my opinion

[BoGGoG]

I second this

[raquelhortab]

Hi! I like the new style although it'd be nice to show the best value on the studies list

[raquelhortab]

Also, on the "Intermediate values" chart, all trials are selected by default (good) but there is no option to deselect all of them so you cannot show just a few if you got many trials

[raquelhortab]

I saw today that by clicking one it automatically deselects the others. However, I observed that scrolling to fins the desired trial can be a nightmare, I user my touchpad and it scrolles so much at once that I can get it to where I need.

[srupprecht-tt]

When selecting the "Analytics" tab, I only get a blank white page. The console tells me that there is an array length error...
Any ideas what could be the problem?

The rest of the dashboard works and looks pretty nice, good job!

[raquelhortab]

It'd be good if the study sorting choice was remembered. For instance, I am always looking for the latest studies and I always need to re-sort them or go all the way down.

[keisuke-umezawa]

@raquelhortab Thank you for your feedback! It will be implemented by this PR. #543

[hokamoto]

I would like to express my deepest appreciation to the development team for developing and releasing such a great tool. Thank you so much.

I'm currently working with Optuna 3.2.0 and Optuna Dashboard 0.10.3, and I have a few queries.

1) Detail of data points in the Pareto Front graph
When I visualize a multi-objective problem and hovering my mouse over each data point in the Pareto Front graph, I've noticed that details are displayed for some trials, but not for others. What determines whether these details are shown or not? I've observed this symptom on various web browsers on my Mac.

• Google Chrome 115
• Safari 16.6
• Firefox 116.0

2) Time zone of Started At and Completed At
When viewing the list of trials, Started At and Completed At fields show like Tue Aug 01 2023 14:38:08 GMT+0900 (Japan Standard Time), but the actual time of this trial is not 14:38:08 (JST) but 14:38:08 (UTC). The time zone of the machine connected to Optuna Dashboard is JST, but the time zone of the server where Optuna and Optuna Dashboard are running is UTC. Is there any way to display the correct time?

Considering that both Optuna and Optuna Dashboard are running in a UTC environment, I am speculating that the dates are being stringified incorrectly on the frontend. FWIW, I'm using SQLite for storage.

I've observed this symptom on the web browsers same as the above.

[max-accenta]

Hi, and thanks for working on this new design. I believe it will be much easier to navigate through the results as more and more visualization tools are available.

A small improvement that could be made: on my (small) screen, there is a horizontal scrollbar as the view is slightly wider than the available space.