Code

[pull] master from Netflix:master#104

Code

Merged

savingoyal merged 97 commits intosavingoyal:mastersavingoyal/metaflow:masterfrom Netflix:masterNetflix/metaflow:masterCopy head branch name to clipboardAug 20, 2022

+31,410-3,776

Commits

Commits on Jan 18, 2022

• fix default resource math (fixes #467) (#810)

  oavdeev

  authored
  e6f1f00

  Copy full SHA for e6f1f00

• run datastore tests on master branch pushes too (#909)

  oavdeev

  authored
  7b948c6

  Copy full SHA for 7b948c6

• Fix datatest broken by previous commit (#912)

  romain-intel

  authored
  1f5afa4

  Copy full SHA for 1f5afa4

Commits on Jan 19, 2022

• Bump version to 2.4.9 (#913)

  oavdeev

  authored
  1c97086

  Copy full SHA for 1c97086

Commits on Jan 20, 2022

• Update README for metaflow/R (#915)

  savingoyal

  authored
  3f7bfa6

  Copy full SHA for 3f7bfa6

Commits on Jan 24, 2022

• Ns package support (#892)

  Show description for 45be4ac
  romain-intel

  authored
  45be4ac

  Copy full SHA for 45be4ac

• Fix issue with @Batch and num_parallel=1; improve status log. (#904)

  akyrola

  authored
  44f1ae7

  Copy full SHA for 44f1ae7

Commits on Jan 25, 2022

• Fix issue in Step Functions integration with CLI defined decorators (#920)

  savingoyal

  authored
  885810a

  Copy full SHA for 885810a

• add unit test for compute_resource_attributes (#922)

  oavdeev

  authored
  5a95ce6

  Copy full SHA for 5a95ce6

• Vendor importlib_metadata (#925)

  Show description for fea7d41
  savingoyal

  authored
  fea7d41

  Copy full SHA for fea7d41

• Fix compute_resources to take into account string values (#919)

  Show description for 01b7149
  

  romain-intel

  and

  oavdeev

  authored
  01b7149

  Copy full SHA for 01b7149

• version bump (#921)

  valayDave

  authored
  44cb29f

  Copy full SHA for 44cb29f

• Stacked @card v1.2 : multiple @card decorators (#893)

  Show description for c247f9f
  

  4 peopleauthored
  c247f9f

  Copy full SHA for c247f9f

Commits on Jan 26, 2022

• Bump follow-redirects in /metaflow/plugins/cards/ui (#927)

  Show description for c52c710
  dependabot[bot]

  authored
  c52c710

  Copy full SHA for c52c710

• Bump marked from 4.0.8 to 4.0.10 in /metaflow/plugins/cards/ui (#926)

  Show description for f73eb71
  dependabot[bot]

  authored
  f73eb71

  Copy full SHA for f73eb71

• Removing some code bits from metaflow.cards (#928)

  Show description for 8753440
  valayDave

  authored
  8753440

  Copy full SHA for 8753440

Commits on Jan 28, 2022

• Introduce Mamba as a dependency solver for @conda (#918)

  Show description for 7d1697d
  

  savingoyal

  and

  jimgoo

  authored
  7d1697d

  Copy full SHA for 7d1697d

Commits on Feb 1, 2022

• Added base64 Transformation for json object (#933)

  Show description for fed10b8
  valayDave

  authored
  fed10b8

  Copy full SHA for fed10b8

Commits on Feb 3, 2022

• Vendor click as dependency for Metaflow (#929)

  Show description for 62f5e52
  savingoyal

  authored
  62f5e52

  Copy full SHA for 62f5e52

Commits on Feb 15, 2022

• Python Multi Version Test Suite (#946)

  Show description for 8c2f776
  valayDave

  authored
  8c2f776

  Copy full SHA for 8c2f776

• bump version to 2.5.1 (#956)

  oavdeev

  authored
  a47dd19

  Copy full SHA for a47dd19

Commits on Feb 16, 2022

• follow symlinks in packaging (#958)

  Show description for 1ea38fd
  tuulos

  authored
  1ea38fd

  Copy full SHA for 1ea38fd

Commits on Feb 17, 2022

• bump version to 2.5.2 (#961)

  oavdeev

  authored
  fc06094

  Copy full SHA for fc06094

Commits on Feb 21, 2022

• Add README.md to PyPI package description (#965)

  Show description for e556481
  savingoyal

  authored
  e556481

  Copy full SHA for e556481

Commits on Feb 24, 2022

• Don't explicitly break py2 support (#962)

  Show description for 2ec907e
  romain-intel

  authored
  2ec907e

  Copy full SHA for 2ec907e

• Pass the paths used by the external interpreter to the one launching the Env Escape server (#960)

  Show description for 54aea15
  romain-intel

  authored
  54aea15

  Copy full SHA for 54aea15

• Bump follow-redirects in /metaflow/plugins/cards/ui (#966)

  Show description for b337606
  dependabot[bot]

  authored
  b337606

  Copy full SHA for b337606

• Adding export of graph to JSON for cli (#955)

  valayDave

  authored
  0939cca

  Copy full SHA for 0939cca

Commits on Feb 25, 2022

• Fix "Too many symbolic links" error when using Conda + Batch on MacOS (#972)

  Show description for 3e69f3f
  bishax

  authored
  3e69f3f

  Copy full SHA for 3e69f3f

Commits on Mar 5, 2022

• emit app tag for AWS Batch jobs (#970)

  savingoyal

  authored
  6b07668

  Copy full SHA for 6b07668

Commits on Mar 8, 2022

• Bump to 2.5.3 (#974)

  oavdeev

  authored
  9f737e6

  Copy full SHA for 9f737e6

Commits on Mar 16, 2022

• Extension packaging improvements (#959)

  Show description for b2c778a
  

  romain-intel

  and

  valayDave

  authored
  b2c778a

  Copy full SHA for b2c778a

• pass DEFAULT_AWS_CLIENT_PROVIDER to remote tasks (#982)

  oavdeev

  authored
  32e2f6f

  Copy full SHA for 32e2f6f

• Fixing some hacky plumbing in card test suite (#967)

  Show description for 1748175
  valayDave

  authored
  1748175

  Copy full SHA for 1748175

• Simplify mflog (#979)

  savingoyal

  authored
  7e210a2

  Copy full SHA for 7e210a2

Commits on Mar 21, 2022

• Fix extension root determination in some cases + card tweaks (#989)

  valayDave

  authored
  c903953

  Copy full SHA for c903953

Commits on Mar 23, 2022

• Use importlib_metadata 2.8.3 for Python 3.4, 3.5 but 4.8.3 for Python 3.6+ (#988)

  Show description for 7787f5e
  romain-intel

  authored
  7787f5e

  Copy full SHA for 7787f5e

• Allow configuring the root directory where artifacts go when pulling from S3. (#991)

  Show description for 0779300
  

  kgullikson88

  and

  kgullikson-sc

  authored
  0779300

  Copy full SHA for 0779300

• Add an option to pass a role argument to S3() to provide a role (#987)

  Show description for cbc4089
  romain-intel

  authored
  cbc4089

  Copy full SHA for cbc4089

Commits on Mar 24, 2022

• bump version to 2.5.4 (#993)

  oavdeev

  authored
  67cc8fb

  Copy full SHA for 67cc8fb

• Bug fix for datetime index in default card (#981)

  Show description for 9e69cfd
  valayDave

  authored
  9e69cfd

  Copy full SHA for 9e69cfd

Commits on Mar 25, 2022

• Bump minimist from 1.2.5 to 1.2.6 in /metaflow/plugins/cards/ui (#995)

  Show description for 540f211
  dependabot[bot]

  authored
  540f211

  Copy full SHA for 540f211

Commits on Apr 13, 2022

• Fix black - set upper bound for click version (#1006)

  cloudw

  authored
  7921d13

  Copy full SHA for 7921d13

Commits on Apr 22, 2022

• Add tags to current singleton (#1019)

  Show description for de6a3b5
  sam-watts

  authored
  de6a3b5

  Copy full SHA for de6a3b5

• Dispatch Metaflow flows to Argo Workflows (#992)

  Show description for aa24b59
  savingoyal

  authored
  aa24b59

  Copy full SHA for aa24b59

Commits on Apr 23, 2022

• Skip sleep if not retryting S3 operations (#1001)

  Show description for 23d07c6
  kevsmith

  authored
  23d07c6

  Copy full SHA for 23d07c6

Commits on Apr 25, 2022

• Bump minor version for release (#1022)

  savingoyal

  authored
  cc471c7

  Copy full SHA for cc471c7

Commits on Apr 27, 2022

• Alternate way of adding metadata to cloned tasks (#1003)

  Show description for 493d406
  romain-intel

  authored
  493d406

  Copy full SHA for 493d406

Commits on Apr 29, 2022

• Bump cross-fetch from 3.1.4 to 3.1.5 in /metaflow/plugins/cards/ui (#1027)

  Show description for aca8ba9
  dependabot[bot]

  authored
  aca8ba9

  Copy full SHA for aca8ba9

• fixed bug from #1023 (#1025)

  valayDave

  authored
  3acd855

  Copy full SHA for 3acd855

Commits on May 3, 2022

• current.pathspec to return None when used outside Flow (#1033)

  Show description for d977cd4
  jackie-ob

  authored
  d977cd4

  Copy full SHA for d977cd4

Commits on May 12, 2022

• Fixing bug in the card list command (#1044)

  Show description for b0238f9
  valayDave

  authored
  b0238f9

  Copy full SHA for b0238f9

Commits on May 13, 2022

• Enable MinIO/EMC-ECS as blob stores for Metaflow (#1045)

  Show description for ac3d265
  savingoyal

  authored
  ac3d265

  Copy full SHA for ac3d265

• [tag-mutation-project] Stash system info in task metadata (#1039)

  Show description for f7a56a7
  jackie-ob

  authored
  f7a56a7

  Copy full SHA for f7a56a7

• Address issues with S3 get and ranges (#1034)

  Show description for 2acbd9e
  romain-intel

  authored
  2acbd9e

  Copy full SHA for 2acbd9e

• Fix _new_task calling bug in LocalMetadataProvider (#1046)

  jackie-ob

  authored
  9443b53

  Copy full SHA for 9443b53

• Release 2.6.1 (#1047)

  oavdeev

  authored
  6cdd311

  Copy full SHA for 6cdd311

Commits on May 24, 2022

• Support default secrets (#1048)

  savingoyal

  authored
  27cce78

  Copy full SHA for 27cce78

Commits on May 25, 2022

• Run id file should be written prior to execution when resuming (#1051)

  Show description for 717731c
  jackie-ob

  authored
  717731c

  Copy full SHA for 717731c

Commits on May 26, 2022

• release 2.6.2 (#1052)

  oavdeev

  authored
  969d386

  Copy full SHA for 969d386

• Fix instance metadata calls for IMDSV2 (#1053)

  savingoyal

  authored
  5c55795

  Copy full SHA for 5c55795

• bump version to 2.6.3 (#1054)

  oavdeev

  authored
  4d02f2a

  Copy full SHA for 4d02f2a

Commits on May 29, 2022

• Small reorder of imports to make things more consistent (#1055)

  romain-intel

  authored
  1430af6

  Copy full SHA for 1430af6

Commits on Jun 1, 2022

• [tag-mutation-project] Local metadata provider - object read paths to return ancestral Run tags (#1043)

  Show description for ffff49b
  jackie-ob

  authored
  ffff49b

  Copy full SHA for ffff49b

Commits on Jun 14, 2022

• typo fix. (#1068)

  valayDave

  authored
  9b0aed9

  Copy full SHA for 9b0aed9

• This is the Jackie's tag changes with an additional test and a fix. (#1063)

  Show description for 57f7400
  

  romain-intel

  and

  jackie-ob

  authored
  57f7400

  Copy full SHA for 57f7400

Commits on Jun 15, 2022

• Add two options to resume: the ability to specify a run-id and the ab… (#1059)

  Show description for 38be8f4
  romain-intel

  authored
  38be8f4

  Copy full SHA for 38be8f4

Commits on Jun 16, 2022

• Improve sidecar message handling (#1057)

  Show description for e073608
  romain-intel

  authored
  e073608

  Copy full SHA for e073608

• Small typo in NullEventLogger.send (#1071)

  Show description for 126546f
  romain-intel

  authored
  126546f

  Copy full SHA for 126546f

• Bump version to 2.7.0

  romain-intel

  committed
  0f2826c

  Copy full SHA for 0f2826c

Commits on Jun 17, 2022

• Change behavior of env escape when module is present (#1072)

  Show description for 25182d3
  romain-intel

  authored
  25182d3

  Copy full SHA for 25182d3

• Bump version to 2.7.1 (#1073)

  romain-intel

  authored
  6441ed5

  Copy full SHA for 6441ed5

Commits on Jun 29, 2022

• Fix an issue with the environment escape server directory (#1074)

  Show description for cc3224e
  romain-intel

  authored
  cc3224e

  Copy full SHA for cc3224e

• Support session_vars and client_params in S3() (#1069)

  Show description for f9a4968
  romain-intel

  authored
  f9a4968

  Copy full SHA for f9a4968

Commits on Jul 5, 2022

• Support M1 Macs (#1077)

  savingoyal

  authored
  b586358

  Copy full SHA for b586358

Commits on Jul 8, 2022

• changes to @kubernetes for sandbox (#1079)

  savingoyal

  authored
  6f097e3

  Copy full SHA for 6f097e3

Commits on Jul 13, 2022

• Changed error to warning in AWS retry (#1081)

  Show description for cbfe4f6
  

  pjoshi30

  and

  Preetam Joshi

  authored
  cbfe4f6

  Copy full SHA for cbfe4f6

Commits on Jul 14, 2022

• bump to 2.7.2 (#1083)

  oavdeev

  authored
  e4eb751

  Copy full SHA for e4eb751

• Fixed s3util test (#1084)

  Show description for 54cd7fc
  

  pjoshi30

  and

  Preetam Joshi

  authored
  54cd7fc

  Copy full SHA for 54cd7fc

Commits on Jul 18, 2022

• Bump svelte from 3.46.1 to 3.49.0 in /metaflow/plugins/cards/ui (#1086)

  Show description for 6722893
  dependabot[bot]

  authored
  6722893

  Copy full SHA for 6722893

Commits on Jul 20, 2022

• Metadata version check flag (#1088)

  mrfalconer

  authored
  c3718dd

  Copy full SHA for c3718dd

• Bump terser from 5.10.0 to 5.14.2 in /metaflow/plugins/cards/ui (#1090)

  Show description for 570c07c
  dependabot[bot]

  authored
  570c07c

  Copy full SHA for 570c07c

Commits on Jul 26, 2022

• fix r CI deps (#1092)

  Show description for c1ff469
  oavdeev

  authored
  c1ff469

  Copy full SHA for c1ff469

Commits on Jul 29, 2022

• fix fractional resoure handling for batch (#1089)

  oavdeev

  authored
  056d2db

  Copy full SHA for 056d2db

• bump version (#1093)

  oavdeev

  authored
  ca29bec

  Copy full SHA for ca29bec

Commits on Aug 1, 2022

• Fix docstrings for the API reference (no functional changes!) (#1076)

  Show description for ac017e1
  tuulos

  authored
  ac017e1

  Copy full SHA for ac017e1

Commits on Aug 2, 2022

• Move a sys.path modification in s3op to __main__ (#1095)

  Show description for e9ea3a4
  romain-intel

  authored
  e9ea3a4

  Copy full SHA for e9ea3a4

Commits on Aug 3, 2022

• Airflow Support (#1094)

  Show description for 925530b
  valayDave

  authored
  925530b

  Copy full SHA for 925530b

• Move sys.path insert earlier in s3op.py (#1098)

  romain-intel

  authored
  f11a372

  Copy full SHA for f11a372

• Update setup.py (#1099)

  savingoyal

  authored
  6cb54b5

  Copy full SHA for 6cb54b5

Commits on Aug 5, 2022

• Fix for env_escape bug when importing local packages (#1100)

  hunsdiecker

  authored
  eb8b373

  Copy full SHA for eb8b373

• Bump to 2.7.5 (#1102)

  romain-intel

  authored
  90ee1ed

  Copy full SHA for 90ee1ed

Commits on Aug 9, 2022

• Fix another issue with the escape hatch and paths (#1105)

  romain-intel

  authored
  b2d0005

  Copy full SHA for b2d0005

• Bump to 2.7.6 (#1106)

  romain-intel

  authored
  b0fab46

  Copy full SHA for b0fab46

Commits on Aug 11, 2022

• add a flag to overwrite config when running metaflow configure sandbox (#1103)

  oavdeev

  authored
  5bf7b60

  Copy full SHA for 5bf7b60

Commits on Aug 18, 2022

• card dev docs tiny fix. (#1108)

  valayDave

  authored
  c2d615e

  Copy full SHA for c2d615e

• Fix an issue with get_cards not respecting a Task's ds-root (#1111)

  Show description for eda2f2d
  romain-intel

  authored
  eda2f2d

  Copy full SHA for eda2f2d