Skip to content

Latest commit

 

History

History
80 lines (53 loc) · 4.02 KB

infrastructure.md

File metadata and controls

80 lines (53 loc) · 4.02 KB

Infrastructure & Computing

Note

Some links in this document require Broad Institute access. These are marked with 🔒.

Computing Infrastructure

Overview

  • Macbook: For most daily tasks
  • DGX-1 / spirit / oppy servers: For GPU computing FIXME: Now DGX-1 owned by C-lab
  • AWS: For specific project needs (use judiciously)
  • Broad's compute cluster: For easily parallelizable jobs

DGX-1 Server (GPU Computing) FIXME: Now DGX-1 owned by C-lab, move this to deprecated

Spirit/Oppy Server (GPU Computing)

Broad's Compute Cluster

  • Best for: Easily parallelizable jobs without interactivity or cloud data access needs
  • Getting started: https://intranet.broadinstitute.org/node/5811 🔒
  • Support: #bits Slack channel
  • Also see #uger and #devnull-gpu-discuss Slack channels for discussions

AWS

  • Use cases: Projects requiring interactivity, large-scale cloud data access, or specific GPU needs
  • Getting started:
    • Request an account through CloudLab for $500 in free credits with a 90-day limit, which will allow you to experiment freely
    • After initial experimentation, ask Shantanu to set up your project account
  • Required reading:
    • Beth's talk 🔒 and slides 🔒 on Imaging Platform AWS Use. This is mostly relevant to Cimini Lab's usage of AWS but the concepts are useful to know.
    • AWS Usage Policy 🔒
  • We recommend interacting via Skypilot; see this Slack thread 🔒

Development Environment Setup

Package Management

  • Install applications using brew
  • Use pixi for project environment management (conda + pip dependencies, GPU support) — see workflows for details
  • Use uv for standalone Python tooling outside of project environments

Project Setup & Initialization

Important

Complete these steps before starting any new project. Most links below require Broad access 🔒.

Project Initialization

You will rarely need to initialize a project yourself, but here are the instructions if needed:

For Non-Cimini Lab Projects

  1. Create Imaging Platform Project List Item 🔒
  2. Send startup email 🔒 - See this blog post for context
  3. Continue with "All Projects" steps below

For Cimini Lab Projects

  • Find project_tag and project_name_full in Imaging Platform Project List 🔒
  • CC img-emailcatcher in email correspondence
  • Continue with "All Projects" steps below

All Projects

  1. Create GitHub repo following wiki guidelines 🔒 for lab notebook
  2. Create folder in ProjectPlanning 🔒: YYYY_MM_DD_Project_description
  3. Create S3 folder: s3://imaging-platform/YYYY_MM_DD_Project_description

Project Naming Convention

  • Format: YYYY_MM_DD_Short_description
  • Example: 2021_08_23_Batch_effect_correction