Skip to content

为什么datajuicer在Ray模式下,不支持groupby算子呢? #920

@lsyel

Description

@lsyel

Before Asking

  • I have asked the Data-Juicer Q&A Copilot (available on Doc Site, DingTalk, or Discord), but the problem still persists.

  • I have read the README carefully.

  • I have pulled the latest code of main branch to run again and the problem still existed.

Search before asking

  • I have searched the Data-Juicer issues and found no similar questions.

Question

groupby算子仅支持default模式,不支持ray模式,是因为ray模式下大规模数据时可能会数据倾斜导致OOM吗?还是说和数据规模无关,ray模式下就是不能groupby?

Environment Information

No response

Additional Context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    questionFurther information is requested

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions