Project Meeting 2024.06.06

Agenda

• Phase 9a update

Action Items

• Create a matrix-like list of runs that provide results, listed by the different specs
  • Latest Run Results
  • Note to add more detailed information about the hardware for each run.
• Runs to do
  • RSG
    • Rerunning the explicit chunking tests, keeping the environment variables consistent between the two
  • WSP *
• WSP to try to rewrite the tour mode choice expression that is causing a crash.
• WSP to test Jeff's code that may address the np.where situation, discussed in the previous week. If that doesn't work, WSP will work on replacing the np.where expressions and test.

Meeting Notes

• Full Scale Performance: Single Process, Sharrow On, Explicit Chunking
  • Explicit chunking for 4 models (ones with higher memory usage), with chunking set to 0.5 (2 chunks)
  • Trip destination did not have much memory usage reduction, which could be a bug. Other models tested had 30-50% reduction in RAM.
  • There are changes in RAM usage for components where there wasn’t explicit chunking. There was also a difference between the two runs where the environment variables were not consistent, which could explain that difference. They are doing another run to make sure everything is the same.
  • There could also be a sharrow memory release difference between the two runs.
  • Note that there are only differences in the impacts of environment variables are changed when sharrow is on or multiprocess is on.
  • There’s are a few very, thin memory spikes. It would be interesting to see any differences if there were more chunks.
• Full Scale Performance: Single Process, Sharrow On
• Sharrow test crashes with utility not aligned error
  • Sijia looked into sharrow test mode because the model crashed due to the utilities not matching. She found the expression in tour mode choice that was causing the problem. Sijia will try to rewrite the tour mode choice expression so that it doesn't crash. She doesn't think this expression is in the MTC model, it's specific to SANDAG ABM3.
• Is there some places where we can memorialize what expressions do and do not work with sharrow? How can that information be translated to other agencies?
  • Guidance on how to write expressions that are sharrow-compatible should be in the Users Guide.
  • For the canonical models, need guidance on how to run in the most efficient ways. Potentially, the Users Guide could add a section on “common errors.”