Inverse Design Plugin

[tylerflex]

Frontend PR:
#94

[lucas-flexcompute]

This is looking really great! Very intuitive to use!

There's a comment about **kwargs in the post-process function that I didn't really understand and I couldn't find in the code. Maybe it is outdated…

One suggestion that I have is to change the result.sim_final to something like result.get_sim(step_index: int = -1) so that it is easy to check the simulation at any point in history. The index can be used as a list index, so passing -1 would grab the last simulation. The same goes for result.get_sim_data(step_index: int = -1).

Also, if we keep it, maybe change sim_final to sim_last (just because it is the last in history, but maybe not the final design, as the optimization might continue later).

Finally, regarding the last comment in the notebook, I'd keep the methods returning different data types as you currently have. I know it's not usually a good idea, but seeing that this is a function used mostly at the end of the workflow (not something that will interfere with the whole processing flow), I don't think the extra complexity of 2 separate functions will pay off. From the user's perspective, they will have to keep track of the use case either way (by correctly treating the return value or calling the right function).

[tomflexcompute]

invdes is a very nice wrapper on top of adjoint! It will keep lowering the entrance barrier of adjoint optimization further and make it accessible to users even without any experience. The notebook looks very nice as well.

A few more minor comments:

• Could you add the notebook to docs/features/adjoint.rst?
• In the title we only capitalize the first word. Also maybe consider a longer title like "Introduction to inverse design plugin"?
• The capitalization of the sub headers is a bit inconsistent. For example "Inverse Design object" -> "Inverse Design Object". Could you double check others.
• Some typos:
  "conveniencen" -> "convenience"
  "funciton" -> "function"
  "optmizer" -> "optimizer"

[weiliangjin2021]

Looks very good. I wish I had used it in my PhD!

One minor comment:

# transformations on the parameters that lead to the material density array (0,1)
filter_project = tdi.FilterProject(radius=0.120, beta=10.0)

# penalties applied to the state of the material density, after these transformations are applied
penalty = tdi.ErosionDilationPenalty(weight=0.8, length_scale=0.120)

Would be good to briefly explain how you choose and what's the rule of thumb of the numbers here, as well as the units.

[momchil-flex]

One more typo from me in the very last note: Therefore it's sim_final and sim_data_final methods -> its.

Is the plan to eventually convert some of the existing adjoint notebooks too?

[tylerflex]

@lucas-flexcompute

There's a comment about **kwargs in the post-process function that I didn't really understand and I couldn't find in the code. Maybe it is outdated…

Yea, the comment was referring to the step_index and history arguments passed to the objective function, but I think the wording was a bit confusing. For now I just left the post processing function call signature as accepting (sim_data, **kwargs) and changed the note to just point out that we can accept other information into the post-processing function, which we'll talk about in a future tutorial.

One suggestion that I have is to change the result.sim_final to something like result.get_sim(step_index: int = -1) so that it is easy to check the simulation at any point in history. The index can be used as a list index, so passing -1 would grab the last simulation. The same goes for result.get_sim_data(step_index: int = -1).

sounds good. I will add the get_sim() get_sim_data() and then I'll just change all mentions of final to last as there are still some other convenience methods that have final in their name.

Thanks @tomflexcompute

Could you add the notebook to docs/features/adjoint.rst?

Done

In the title we only capitalize the first word. Also maybe consider a longer title like "Introduction to inverse design plugin"?

I fixed the capitalization. My opinion (not strongly held) is that our titles are a bit long generally and it makes them hard to parse when looking at all of the notebooks. I might leave it as inverse design plugin for now until we maybe have future notebooks, at which point, maybe I'll change it to inverse design plugin: introduction?

The capitalization of the sub headers is a bit inconsistent. For example "Inverse Design object" -> "Inverse Design Object". Could you double check others.

Made all of them title case.

Some typos:
"conveniencen" -> "convenience"
"funciton" -> "function"
"optmizer" -> "optimizer"

Fixed all.

@momchil-flex

One more typo from me in the very last note: Therefore it's sim_final and sim_data_final methods -> its.

This section was just an internal note, so I ended up removing it anyway.

Is the plan to eventually convert some of the existing adjoint notebooks too?

Yea I'd like to do that for the ones that use topology optimization. I'm not sure whether to do it within the existing notebooks, or just to create invdes copies of the adjoint ones. Any preferences?

[tylerflex]

Looks very good. I wish I had used it in my PhD!

One minor comment:

# transformations on the parameters that lead to the material density array (0,1)
filter_project = tdi.FilterProject(radius=0.120, beta=10.0)

# penalties applied to the state of the material density, after these transformations are applied
penalty = tdi.ErosionDilationPenalty(weight=0.8, length_scale=0.120)

Would be good to briefly explain how you choose and what's the rule of thumb of the numbers here, as well as the units.

Thanks @weiliangjin2021 I've added some more explanation in that cell.

[tylerflex]

FYI I changed the API a bit so that instead of the free floating functions, tdi.get_amps, tda.sum_abs_squared, I made a class tda.Utilities that contains these as @classmethods, so it would be

tdi.Utilities.get_amps(sim_data, ...)

I think it makes it easier to document in the API reference and users can also help(Utilities), which I'll put in the notebook to demonstrate.

[daquinteroflex]

I really like how straightforward this is for people getting started on inverse design, it just makes sense and interfaces very well with the tidy3d API. I look forward to sharing this with some of our research group when we were interested in designing some basic components with invdes but didn't have the computers or adjoint experience then.

Just a suggestion to improve some docs for beginners:

• Adding figures of how the Discretizations, Transformations and Penalties work in very visual terms, might be pretty handy to teach people for both the notebook and the corresponding classes in the API.

It's really well explained anyways!

Just also brainstorming (not sure it has application value exactly), it might be a pretty cool idea in the future if we use this tool to do the multi-inverse design optimisation between heat and fdtd. Say to maximise both EM transmission and heat transfer/phase shifting in a relevant application. Like in a way this tool enables doing this type of thing that maybe others wouldn't.

[e-g-melo]

Hi @tylerflex! It is a fascinating improvement to our inverse design project workflow! Thanks for this work!

General comments:

1. I wonder if you have included that buffer layer to improve fabricability at the interfaces between the design region and the external geometries.

2. Could we have different grid sizes for the design region and FDTD? For example, include an optional fdtd_grid_size parameter. So the user can specify grid_size only to have the same resolution to design region and FDTD, or grid_size and fdtd_grid_size to set respectively the design region and FDTD resolutions. That would allow us to specify a finer mesh for the design region to apply all the fabrication constraints and create a dedicated GDS export function that uses that finer mesh to improve the smoothness of the sidewall contours.

I have some comments more focused on GUI implementation now:

1. It would be required to show more visual information on the progress of the optimization. I suppose we can read the objective function, FOM, and gradient progress from the history. But it would be interesting to include an optional list of monitors to save to the history as well, so the users can include whatever monitors they want to track the device response during the optimization.

2. Besides num_steps, we could include a threshold value for stopping the iteration. For example, if the post_process_fn function returns a value higher than xxx, we stop the iteration to save user credits.

3. In addition to the post_process_fn, it would be interesting to have an alternative objective function object, as you mentioned before in a Slack discussion. So, the users could create, for example:

objective = tdi.Objective(function=-tdi.Softmax(tdi.Power(monitor_name=mon_a, f=freq0, mode_num=0, direction="+"), tdi.Power(monitor_name=mon_b, f=freq0, mode_num=0, direction="+"), tdi.Power(monitor_name=mon_c, f=freq0, mode_num=0, direction="+")))

design = tdi.InverseDesign(
    simulation=simulation,
    design_region=design_region,
    post_process_fn=None,
    objective_fn=objective,
    task_name="invdes",
    output_monitor_names=[mnt.name for mnt in monitors_out],
)

That would help us to create/show the objective function using this interface, which we already use in post-run and Design plugin to define such custom values.

In addition to these operators in the image, it would be nice to have:

• Min
• Max
• Softmax
• Sum
• Mean
• StandardDeviation

4. Another interesting convenience function would be an initial structure. So, if the user supplies that initial structure, which could be a td.GeometryGroup, td.Box, etc, we could map the initial parameter array to that structure and then start the optimization from it. I don't know if it will be very useful in the case of topology, but as we should have such a feature in level-set and shape, it would be interesting to include it in topology as well to keep uniformity.

[tylerflex]

@daquinteroflex

Adding figures of how the Discretizations, Transformations and Penalties work in very visual terms, might be pretty handy to teach people for both the notebook and the corresponding classes in the API.

Thanks, I went ahead and made some images to add to the API reference. Can you double check that I did it correctly in this commit. and also if you think I should edit the images or the text at all to make it clearer?

it might be a pretty cool idea in the future if we use this tool to do the multi-inverse design optimisation between heat and fdtd

This would be cool, I'll give that some thought. Maybe I can one day support HeatSimulation in the simulations field and then the post processing function can be a function of the fdtd data and the heat data together.

[tylerflex]

@e-g-melo thanks for your suggestions. they are all really good ones. I think many of them might be good as enhancements we can add in later versions. I don't see them breaking backwards compatibility, so it can be a good idea to just target them later, also depending on whether users find them lacking. flexcompute/tidy3d#1581

I did want to address something in your comments and see if I understand

Could we have different grid sizes for the design region and FDTD? For example, include an optional fdtd_grid_size parameter. So the user can specify grid_size only to have the same resolution to design region and FDTD, or grid_size and fdtd_grid_size to set respectively the design region and FDTD resolutions. That would allow us to specify a finer mesh for the design region to apply all the fabrication constraints and create a dedicated GDS export function that uses that finer mesh to improve the smoothness of the sidewall contours.

So the DesignRegion currently has a pixel_size field, that is set independently from the FDTD grid. The JaxSimulation will include a MeshOverrideStructure with a dl set by this pixel_size unless set differently (or removed) through the optional DesignRegion.override_structure_dl field.

Do you think this solves the issue you raise here? The things I don't love about the current API:

1. There's not a way to automatically set DesignRegion.pixel_size from the Simulation. Or at least I haven't thought through this. Maybe we can use the minimum dl in the DesignRegion.simulation to set it automatically?
2. If the pixel_size is very large, there can be some trouble, for example the Simulation override structure can become very low resolution. I'm not sure a great way to validate it.

If you have any general thoughts about the API for setting resolutions as described above, especially with regards to your comment, let me know and we can adjust how it works.

[e-g-melo]

So the DesignRegion currently has a pixel_size field, that is set independently from the FDTD grid. The JaxSimulation will include a MeshOverrideStructure with a dl set by this pixel_size unless set differently (or removed) through the optional DesignRegion.override_structure_dl field.

Do you think this solves the issue you raise here? The things I don't love about the current API:

1. There's not a way to automatically set DesignRegion.pixel_size from the Simulation. Or at least I haven't thought through this. Maybe we can use the minimum dl in the DesignRegion.simulation to set it automatically?
2. If the pixel_size is very large, there can be some trouble, for example the Simulation override structure can become very low resolution. I'm not sure a great way to validate it.

Thanks for the clarification @tylerflex. That should work!

For 1, maybe we could consider a simple expression like lda0/(15 * eps_max**0.5), which we can easily explain in docs and GUI.
For 2, what about raising a warning if the pixel_size > lda0/(10 * eps_max**0.5)?

[tylerflex]

For 1, maybe we could consider a simple expression like lda0/(15 * eps_max0.5), which we can easily explain in docs and GUI. For 2, what about raising a warning if the pixel_size > lda0/(10 * eps_max0.5)?

@e-g-melo what is lda0 here?

I could try adding these validators, there's some complications regarding how to compute eps_max, for example at what frequency, but this is something we handle in other parts of the code, so maybe I can re-use bits. Maybe another idea is to use the Simulation.grid to compute the smallest (or average) grid size and then compare this value to pixel_size. Maybe @momchil-flex has some ideas too?

[tylerflex]

@e-g-melo I added a validation to InverseDesign and InverseDesignMulti to make it more user friendly when setting the design region pixel size.

flexcompute/tidy3d@52467cc

The logic is to check the minimum wavelength in the material within the simulation(s) and compare this to the pixel size. If the pixel size is larger than 0.1 * the minimum wavelength, the validator logs a warning, telling the user to set a lower pixel size or set the mesh override DL.

Do you think this works?

[e-g-melo]

@e-g-melo I added a validation to InverseDesign and InverseDesignMulti to make it more user friendly when setting the design region pixel size.

flexcompute/tidy3d@52467cc

The logic is to check the minimum wavelength in the material within the simulation(s) and compare this to the pixel size. If the pixel size is larger than 0.1 * the minimum wavelength, the validator logs a warning, telling the user to set a lower pixel size or set the mesh override DL.

Do you think this works?

Yes, this is a good strategy.