Skip to content

Pull requests: datajuicer/data-juicer

New pull request New
26 Open 582 Closed
26 Open 582 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

docs: add operator documentation
#930 opened Mar 5, 2026 by cmgzn • Draft
11
perf: optimize TokenNumFilter with batch tokenization
#929 opened Mar 3, 2026 by JohnGiorgi Loading…
3 tasks done
5
fix(ops): fix GeneralFusedOP discarding Mapper results in fused pipeline
#928 opened Mar 3, 2026 by dubin555 Loading…
8
fix(ops): fix NlpaugEnMapper only augmenting first sample in batch
#927 opened Mar 3, 2026 by dubin555 Loading…
6
fix(ops): prevent shared mutable _default_kwargs pollution across operator instances
#926 opened Mar 3, 2026 by dubin555 Loading…
6 tasks done
8
feat(mapper): add custom tokenizer support to RemoveRepeatSentencesMapper
#925 opened Feb 27, 2026 by JohnGiorgi Loading…
4
perf: cache redundant sum() calls in repetition filters
#924 opened Feb 26, 2026 by dubin555 Loading…
2 tasks done
5
feat: add latex_figure_context_extractor_mapper operator
#923 opened Feb 26, 2026 by liyuyi-2001 Loading…
6
Add support for json[l].gz, and make ray dataset support reading json…
#919 opened Feb 25, 2026 by HunterLine Loading…
9
[WIP] arXiv/PDF to Markdown mappers + dj-op one-shot runner dj:op issues/PRs about some specific OPs
#917 opened Feb 14, 2026 by yxdyc Loading…
7
[WIP] Multi-branch executor dj:core issues/PRs about the core functions of Data-Juicer enhancement New feature or request
#916 opened Feb 13, 2026 by yxdyc Loading…
1
6
[WIP] feat: Add combined_logical_filter operator with AND/OR support dj:op issues/PRs about some specific OPs
#914 opened Feb 13, 2026 by yxdyc Loading…
6
Feat: Support iceberg、hudi、delta、hdfs data source.
#911 opened Feb 11, 2026 by Dludora Loading…
7
[WIP] Feat: Add video_calibration_mapper and video_split_by_frame_mapper
#902 opened Feb 1, 2026 by 1van2ha0 Loading…
3 tasks
7
[WIP] Feat: Add RayImageBTSMinhashDeduplicator
#897 opened Jan 29, 2026 by Dludora Loading…
8
[WIP] feat: Pr 839 s3 download checkpoint resume and unittest for s3 download
#870 opened Dec 25, 2025 by Dludora Loading…
7
Depth seg new op dj:op issues/PRs about some specific OPs
#862 opened Dec 22, 2025 by archernsy Loading…
18
Add Operator-Level Parallel Data Processing with Ray Actors dj:dist issues/PRs about distributed data processing dj:efficiency regarding to efficiency issues and enhancements enhancement New feature or request
#761 opened Aug 19, 2025 by Cccccc0630 Loading…
12
[NewOp] Add group_diversity_filter op
#745 opened Jul 22, 2025 by lingzhq Loading…
5
Add lidar object segmentation op
#736 opened Jul 14, 2025 by Qirui-jiao Loading…
9
[WIP] add lidar object detection op
#721 opened Jun 26, 2025 by Cathy0908 Loading…
1
[NewOp] Add generate_challenging_qa_mapper based on MindGYM principles
#703 opened Jun 14, 2025 by Bat-Reality Loading…
12
Optimization framework dj:core issues/PRs about the core functions of Data-Juicer dj:efficiency regarding to efficiency issues and enhancements
#702 opened Jun 13, 2025 by cyruszhang • Draft
5
[NewOp] Add domain_diversity_selector based on DaaR principles
#699 opened Jun 12, 2025 by lingzhq Loading…
8
[WIP] deduping benchmark suite
#607 opened Mar 4, 2025 by cyruszhang Loading…
1
ProTip! Exclude everything labeled bug with -label:bug.