Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RELEASE] rmm v24.06 #1571

Merged
merged 35 commits into from
Jun 5, 2024
Merged

[RELEASE] rmm v24.06 #1571

merged 35 commits into from
Jun 5, 2024

Conversation

raydouglass
Copy link
Member

❄️ Code freeze for branch-24.06 and v24.06 release

What does this mean?

Only critical/hotfix level issues should be merged into branch-24.06 until release (merging of this PR).

What is the purpose of this PR?

  • Update documentation
  • Allow testing for the new release
  • Enable a means to merge branch-24.06 into main for the release

raydouglass and others added 30 commits March 15, 2024 12:03
@raydouglass
DOC v24.06 Updates [skip ci]
fb23a13
@GPUtester
Merge pull request #1511 from rapidsai/branch-24.04
c60bcb1 
Forward-merge branch-24.04 into branch-24.06 [skip ci]
@galipremsagar
Enable all tests for arm arch (#1510)
bd3f0d8 
This PR enables running all pytests for `arm64` jobs.

Authors:
  - GALI PREM SAGAR (https://github.com/galipremsagar)

Approvers:
  - Jake Awe (https://github.com/AyodeAwe)
  - Mark Harris (https://github.com/harrism)
  - https://github.com/jakirkham

URL: #1510
@wence-
Explicitly use the current device resource in DeviceBuffer (#1514)
8675b2c 
Previously we were relying on the C++ and Python-level device resources to agree. But this need not be the case.

To avoid this, first get the current device resource and then use it when allocating the wrapped C++ device_buffer when creating DeviceBuffers.

- Closes #1506

Authors:
  - Lawrence Mitchell (https://github.com/wence-)

Approvers:
  - Mark Harris (https://github.com/harrism)

URL: #1514
@harrism
Remove deprecated supports_streams and get_mem_info methods. (#1519)
48820cb 
Fixes #1388 
Fixes #1389 
Fixes #1428
Fixes #1434

Removes the previously deprecated functions

 - `rmm::mr::device_memory_resource::supports_streams()`
 - `rmm::mr::device_memory_resource::supports_get_mem_info()`
 - `rmm::mr::device_memory_resource::get_mem_info()`

Authors:
  - Mark Harris (https://github.com/harrism)

Approvers:
  - Michael Schellenberger Costa (https://github.com/miscco)
  - Bradley Dice (https://github.com/bdice)
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1519
@galipremsagar
Remove a file not being used (#1521)
cdf20a6 
We removed using this file in : #1510, this pr deletes the file.

Authors:
  - GALI PREM SAGAR (https://github.com/galipremsagar)

Approvers:
  - https://github.com/jakirkham
  - Ray Douglass (https://github.com/raydouglass)

URL: #1521
@harrism
Fix ordering / heading levels in README.md and python example in guid…
6771b71 
…e.md (#1513)

This is a minor fix to reorder some of the C++ docs in README.md and fix heading levels. It also adds prompt characters to the pytorch allocator example in the Python guide.md to be consistent with other examples in the doc.

Authors:
  - Mark Harris (https://github.com/harrism)

Approvers:
  - Lawrence Mitchell (https://github.com/wence-)

URL: #1513
@GPUtester
Merge pull request #1525 from rapidsai/branch-24.04
af756c6 
Forward-merge branch-24.04 into branch-24.06
@wence-
Update multi-gpu discussion for device_buffer and device_vector dtors (…
7d7d65a 
…#1524)

Since #1370, the dtor for device_buffer ensures that the correct device is active when the deallocation occurs. We therefore update the example to discuss this. Since device_vector still requires the user to manage the active device correctly by hand, call this out explicitly in the documentation.

- Closes #1523

Authors:
  - Lawrence Mitchell (https://github.com/wence-)

Approvers:
  - Mark Harris (https://github.com/harrism)

URL: #1524
@vyasr
Move rmm Python package to subdirectory (#1526)
9e02f34 
This change is in preparation for #1512.

Authors:
  - Vyas Ramasubramani (https://github.com/vyasr)

Approvers:
  - Bradley Dice (https://github.com/bdice)
  - Ray Douglass (https://github.com/raydouglass)

URL: #1526
@harrism
Remove deprecated cuda_async_memory_resource constructor that takes t…
7ed529f 
…hrust::optional parameters (#1535)

Fixes #1534
Fixes #1530

Authors:
  - Mark Harris (https://github.com/harrism)

Approvers:
  - Bradley Dice (https://github.com/bdice)
  - Michael Schellenberger Costa (https://github.com/miscco)

URL: #1535
@wence-
Make thrust_allocator deallocate safe in multi-device setting (#1533)
588928f 
Previously, the user had to arrange that the device active when a
thrust_allocator object was created was also active when allocate and
deallocate was called. This is hard to manage if exceptions are
thrown. Instead, save the active device on construction and ensure
that it is active when calling deallocate and deallocate. This means
that device_vector is safe to destruct with RAII semantics in a
multi-device setting.

Add tests of this facility, and correct the parameterization usage in
the other thrust allocator tests such that we actually check the MRs
we're parameterizing over.

- Closes #1527

Authors:
  - Lawrence Mitchell (https://github.com/wence-)

Approvers:
  - Rong Ou (https://github.com/rongou)
  - Mark Harris (https://github.com/harrism)

URL: #1533
@robertmaynard
Always use a static gtest (#1532)
9e6db74 
Removes the need for us to install GTest in our rmm CI containers.

Authors:
  - Robert Maynard (https://github.com/robertmaynard)

Approvers:
  - Bradley Dice (https://github.com/bdice)
  - Mark Harris (https://github.com/harrism)
  - Jake Awe (https://github.com/AyodeAwe)

URL: #1532
@harrism
Remove deprecated functionality (#1537)
3caaa08 
Closes #1536

Authors:
  - Mark Harris (https://github.com/harrism)

Approvers:
  - Lawrence Mitchell (https://github.com/wence-)
  - Vyas Ramasubramani (https://github.com/vyasr)
  - Ray Douglass (https://github.com/raydouglass)

URL: #1537
@AyodeAwe
Remove unneeded update-version.sh update (#1520)
bdb7a59 
This shouldn't update `build_docs.sh` as this file already reads the version from an updated file.

Authors:
  - Jake Awe (https://github.com/AyodeAwe)
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - Bradley Dice (https://github.com/bdice)
  - Ray Douglass (https://github.com/raydouglass)

URL: #1520
@wence-
Fix Python install instruction (#1547)
26fa9ec 
In #1526 we moved the location of the python package, so should also update the install instructions.

- Closes #1546

Authors:
  - Lawrence Mitchell (https://github.com/wence-)

Approvers:
  - Mark Harris (https://github.com/harrism)
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1547
@trxcllnt
Fix cuda11.8 nvcc dependency (#1542)
c74dda2 
`cuda-nvcc` is the CUDA 12 package name but was being included in the CUDA 11 dependencies, making the conda solve impossible. This PR uses the same nvcc matrix as defined in the build dependencies list.

Authors:
  - Paul Taylor (https://github.com/trxcllnt)

Approvers:
  - Jake Awe (https://github.com/AyodeAwe)

URL: #1542
@vyasr
Only use functions in the limited API (#1545)
01ccf97 
This PR removes usage of the only method in rmm's Cython that is not part of the Python limited API. Contributes to rapidsai/build-planning#42

Authors:
  - Vyas Ramasubramani (https://github.com/vyasr)
  - https://github.com/jakirkham

Approvers:
  - https://github.com/jakirkham

URL: #1545
@bdice
Remove header existence tests. (#1550)
9d0a29a 
This PR removes header existence tests from the librmm conda recipe to reduce friction when code is moved. Closes #1549.

These changes are similar to rapidsai/cudf#14072.

Authors:
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - Mark Harris (https://github.com/harrism)
  - Lawrence Mitchell (https://github.com/wence-)
  - Ray Douglass (https://github.com/raydouglass)

URL: #1550
@mroeschke
Enable warnings as errors for Python tests (#1551)
363e4c6 
As part of rapidsai/build-planning#26, warnings in Python tests will cause that test to fail.

Authors:
  - Matthew Roeschke (https://github.com/mroeschke)
  - Lawrence Mitchell (https://github.com/wence-)

Approvers:
  - Lawrence Mitchell (https://github.com/wence-)

URL: #1551
@vyasr
Build C++ wheel (#1529)
beab71a 
This PR changes wheel building in RMM to create a separate C++ wheel that is then found from the Python wheel. The C++ wheel is now a hard dependency of the Python wheel. This allows Python packaging to more closely mirror the structure of our conda packaging, and the way we would normally wish to package these in any other manager. It also allows us to reduce package sizes by allowing better sharing of artifacts between different Python packages that rely on the same C++ components from other packages.

Contributes to rapidsai/build-planning#33

Authors:
  - Vyas Ramasubramani (https://github.com/vyasr)

Approvers:
  - Lawrence Mitchell (https://github.com/wence-)
  - Ray Douglass (https://github.com/raydouglass)
  - Bradley Dice (https://github.com/bdice)

URL: #1529
@vyasr
Add publish jobs for C++ wheels (#1554)
fdef5f9 
#1529 added new C++ wheels, but I forgot to update build.yaml there.

Depends on rapidsai/shared-workflows#209.

Authors:
  - Vyas Ramasubramani (https://github.com/vyasr)

Approvers:
  - AJ Schmidt (https://github.com/ajschmidt8)
  - Ray Douglass (https://github.com/raydouglass)

URL: #1554
@jameslamb
add RAPIDS copyright pre-commit hook (#1553)
c7fa7b1 
Contributes to rapidsai/build-planning#30.

Proposes adding the RAPIDS `verify-copyright` pre-commit hook, which automatically updates copyright dates for modified files.

This also fixes one date in a file from #1529.

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Bradley Dice (https://github.com/bdice)
  - Kyle Edwards (https://github.com/KyleFromNVIDIA)

URL: #1553
@hcho3
Migrate to {{ stdlib("c") }} (#1543)
e13b253 
The `sysroot*` syntax is getting phased out (conda-forge/conda-forge.github.io#2102).
The recommendation is to move to `{{ stdlib("c") }}`.

Ref rapidsai/build-planning#39

Authors:
  - Philip Hyunsu Cho (https://github.com/hcho3)
  - https://github.com/jakirkham

Approvers:
  - Bradley Dice (https://github.com/bdice)
  - https://github.com/jakirkham
  - Ray Douglass (https://github.com/raydouglass)

URL: #1543
@harrism
Refactor polymorphic allocator to use device_async_resource_ref (#1555)
4f01c55 
Fixes #1540.
Fixes #1453 by removing unused `polymorphic_allocator::resource()`.

Authors:
  - Mark Harris (https://github.com/harrism)

Approvers:
  - Michael Schellenberger Costa (https://github.com/miscco)
  - Rong Ou (https://github.com/rongou)

URL: #1555
@trxcllnt
add --rm and --name to devcontainer run args (#1539)
f11c8ca 
* Remove the devcontainer when the VSCode window closes
* Adds a descriptive name to the running container:
  ```shell
  $ docker ps -a
  CONTAINER ID   IMAGE        ...  NAMES
  0dbb364fe544   vsc-rmm-...  ...  rapids-rmm-24.06-cuda12.2-conda
  
  $ docker rm -f rapids-rmm-24.06-cuda12.2-conda
  ```

Authors:
  - Paul Taylor (https://github.com/trxcllnt)

Approvers:
  - Mark Harris (https://github.com/harrism)
  - Ray Douglass (https://github.com/raydouglass)

URL: #1539
@raydouglass
Overhaul ops-codeowners (#1561)
aa16588
@aaronmondal
Don't ignore GCC-specific warning under Clang (#1557)
32cd537 
Clang doesn't know `-Wnon-template-friend`.

Authors:
  - Aaron Siddhartha Mondal (https://github.com/aaronmondal)

Approvers:
  - Mark Harris (https://github.com/harrism)
  - Michael Schellenberger Costa (https://github.com/miscco)

URL: #1557
@wence-
Allow specifying mr in DeviceBuffer construction, and document owners…
8ee39ad 
…hip requirements in Python/C++ interfacing (#1552)

On the C++ side, device_buffers store raw pointers for the memory
resource that was used in their allocation. Consequently, it is unsafe
to take ownership of a device_buffer in Python unless we controlled
the provenance of the memory resource that was used in its allocation.
The only way to do that is to pass the memory resource from Python
into C++ and then use it when constructing the DeviceBuffer.

Add discussion of this with some examples and a section on pitfalls
if only relying on get_current_device_resource and
set_current_device_resource.

To allow Python users of `DeviceBuffer` objects to follow best practices,
introduce explicit (defaulting to the current device resource) `mr` 
arguments in both `c_from_unique_ptr` and the `DeviceBuffer` constructor.

- Closes #1492

Authors:
  - Lawrence Mitchell (https://github.com/wence-)

Approvers:
  - Mark Harris (https://github.com/harrism)
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1552
@lilohuang
Adding support for cupy.cuda.stream.ExternalStream (#1559)
cab7e06 
Cupy offers the `cupy.cuda.stream.ExternalStream` for utilizing external CUDA streams. Moreover, `cupy.cuda.get_current_stream()` will return an instance of `cupy.cuda.stream.ExternalStream` instead of `cupy.cuda.stream.Stream`, particularly when the current cuPy stream has been changed.

Therefore, we must verify both types of instances to avoid errors.

See details in the https://docs.cupy.dev/en/stable/user_guide/interoperability.html#cuda-stream-pointers

Authors:
  - Lilo Huang (https://github.com/lilohuang)
  - Mark Harris (https://github.com/harrism)

Approvers:
  - Mark Harris (https://github.com/harrism)
  - Leo Fang (https://github.com/leofang)

URL: #1559
jameslamb and others added 4 commits May 16, 2024 19:34
@jameslamb
update copyright dates (#1564)
91d529f 
Updates some copyright dates. These were updated in 2024, but their copyright dates were not automatically modified because #1553 hadn't been merged yet.

On each file, I made a trivial change to convince the `verify-copyright` pre-commit hook that the files have changed.

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1564
@raydouglass
Make sure rmm wheel dependency on librmm is updated [skip ci] (#1565)
46e153c
@harrism
Add NVTX support and RMM_FUNC_RANGE() macro (#1558)
4e9482f 
Let's get RMM allocate/deallocates showing up in profiler timelines.

Closes #495

Authors:
  - Mark Harris (https://github.com/harrism)
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - Lawrence Mitchell (https://github.com/wence-)
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1558
@robertmaynard
rmm needs to link to nvtx3::nvtx3-cpp to support installed nvtx3 (#1569)
dc1e17a 
If nvtx3 is already installed on the system via rapids-cmake rmm fails to link since it uses the non-namespaced target names.

Authors:
  - Robert Maynard (https://github.com/robertmaynard)

Approvers:
  - Vyas Ramasubramani (https://github.com/vyasr)
  - Paul Taylor (https://github.com/trxcllnt)

URL: #1569
@raydouglass raydouglass requested review from a team as code owners May 24, 2024 19:34
@raydouglass raydouglass requested review from wence- and miscco May 24, 2024 19:34
@copy-pr-bot copy-pr-bot
Copy link

copy-pr-bot bot commented May 24, 2024

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions bot added CMake Python Related to RMM Python API conda cpp Pertains to C++ code ci labels May 24, 2024
@raydouglass raydouglass merged commit 416afcc into main Jun 5, 2024
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@harrism harrism harrism approved these changes

@galipremsagar galipremsagar galipremsagar approved these changes

@wence- wence- Awaiting requested review from wence- wence- is a code owner automatically assigned from rapidsai/rmm-cpp-codeowners

@miscco miscco Awaiting requested review from miscco miscco is a code owner automatically assigned from rapidsai/rmm-cpp-codeowners

Assignees
No one assigned
Labels
ci CMake conda cpp Pertains to C++ code Python Related to RMM Python API
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.