Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Rolling Correlation Perf Enhancement #552

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

aa452948257
Copy link
Contributor

@aa452948257 aa452948257 commented Jun 29, 2023

What do these changes do?

The purpose of this modification is to address the slow computation speed of RollingCorrelation when the number of columns is large. It involves parallelizing the computation by dividing it into column-wise partitions.

Related issue number

#316

Check code requirements

  • tests added / passed (if needed)
  • Ensure all linting tests pass

@XprobeBot XprobeBot added this to the v0.3.2 milestone Jun 29, 2023
@aa452948257 aa452948257 changed the title Enh/df corr ENH: Jun 29, 2023
@XprobeBot XprobeBot added the enhancement New feature or request label Jun 29, 2023
@aa452948257 aa452948257 changed the title ENH: ENH: Rolling Correlation Perf Enhancement Jun 29, 2023
@codecov
Copy link

codecov bot commented Jun 29, 2023

Codecov Report

Merging #552 (37a512a) into main (4ae5f34) will decrease coverage by 11.72%.
The diff coverage is 33.00%.

@@             Coverage Diff             @@
##             main     #552       +/-   ##
===========================================
- Coverage   86.54%   74.83%   -11.72%     
===========================================
  Files        1008     1009        +1     
  Lines       77917    78115      +198     
  Branches    16170    16216       +46     
===========================================
- Hits        67437    58457     -8980     
- Misses       8976    17601     +8625     
- Partials     1504     2057      +553     
Flag Coverage Δ
unittests 74.71% <33.00%> (-11.74%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...hon/xorbits/_mars/dataframe/window/rolling/corr.py 32.29% <32.29%> (ø)
...hon/xorbits/_mars/dataframe/window/rolling/core.py 78.09% <42.85%> (-21.91%) ⬇️
python/xorbits/_mars/opcodes.py 100.00% <100.00%> (ø)

... and 166 files with indirect coverage changes

@XprobeBot XprobeBot modified the milestones: v0.3.2, v0.4.1 Jul 1, 2023
@XprobeBot XprobeBot modified the milestones: v0.4.1, Temp Jul 10, 2023
@aresnow1 aresnow1 removed this from the Temp milestone Jul 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants