Q4 Reduced workload goal - Oct 18 Sprint 1 tracking issue

[consideRatio]

Timeline

Wed-Fri Oct 18-20

The sprint was introduced, we triaged issues in 2i2c-org/infrastructure. As goal lead I was still catching up with planning and adjustments to the available capacity allocations.

In direct communication with Yuvi and Georgiana, a focus towards resolving node sharing related aspects was established.

Oct 23-31

Me and @GeorgianaElena met to discuss and overview node sharing related topics to work, notes available in this hack.md document.

• @yuvipanda fixed things in Add script to generate resource allocation (nodeshare) choices #3030 that we merged with the intent to let me and Georgiana work on it from there over time during this goal.
• @GeorgianaElena:
  • [Agreement needed]: add docs about the notebook node pool default choices #3304
  • Make all GCP clusters support the instance types 4, 16, and 64 CPU highmem nodes #3319, closes Let all clusters cloud infra support the instance types 4, 16, and 64 CPU highmem nodes #3256
  • [deployer] Relocate resource allocation script #3335, closes Resource allocation script: relocate commands under "generate"? #3310
• @consideRatio:
  • Stop using continuous prePuller #3313
    Initialized but not worked directly to completion because the work to do it grew and could be delayed without consequence to prioritize work that blocks other work
  • Add script to collect information about daemonsets cpu/memory requests in various clusters #3330, Refine script to collect information about instances capacity #3338, closed by deployer: add info generating commands under resource-allocation #3337
    This blocks other refinements for the resource allocation logic
  • daskhub / leap: optimize dask-gateway options for a 16 CPU / 128 GB mem worker #2364
    Pending work about daemonset requests in misc clusters

Upcoming:

• [deployer] Allow skipping and forcing the creation/update of a cluster config file #3294
• Upgrade our k8s clusters to k8s 1.25+ #3224
• Let node resource-allocation script handle optimizations for events #3293
• Document how to pre-pull images to pre-started nodes during events #2541

[GeorgianaElena]

@consideRatio, hope it's ok that I added #3256 to the upcoming section above, since once #3319 it's done, I think we can close that one.

[consideRatio]

Thank you @GeorgianaElena you are most welcome to contribute to updating this issue!!! :D

[damianavila]

Sprint 1 is done a long time ago, so closing it.