Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding sliding() implementation #158

Open
oleg-smith opened this issue Sep 10, 2017 · 9 comments
Open

Adding sliding() implementation #158

oleg-smith opened this issue Sep 10, 2017 · 9 comments

Comments

@oleg-smith
Copy link

oleg-smith commented Sep 10, 2017

Is it possible to add sliding() to StreamEx? Now I found it only in tests.

@amaembo
Copy link
Owner

amaembo commented Sep 16, 2017

I'm reluctant to implement it as built-in operation, because it's hard to implement efficient parallel version. Test demonstrates how you can do it in your project if you need it. If you want slides of two elements, use pairMap. Also if your source is List or array, you can use StreamEx.ofSubLists() and you will have very efficient sliding with a little allocations.

@oleg-smith
Copy link
Author

Ok, and could you tell why ofSublists() cannot work over ordered sequential stream? Why do I need to create collection first to use ofSublists() for such case?

@amaembo
Copy link
Owner

amaembo commented Sep 24, 2017

There's no thing like sequential stream in the Stream API type system: the parallel flag can be set and cleared at any point and I cannot provide sequential implementation only without providing a parallel implementation. It's possible to use "poor man parallelism", but I don't like to do it (though do it occasionally).

List.ofSubLists works completely fine for parallel stream if the source list provides fast random access, because different threads can get different list elements by an index.

@oleg-smith
Copy link
Author

Ok, there is no thing like that, how do I solve problem where I need to split stream by equally sized portions without loading all its content in memory?

@amaembo
Copy link
Owner

amaembo commented Sep 24, 2017

Currently you may create utility methods in your project like it's demonstrated in tests. Write in utility class:

private static <T> StreamEx<List<T>> sliding(StreamEx<List<T>> input, int size) {
    return input.headTail((head, tail) -> head.size() == size ? sliding(
            tail.mapFirst(next -> StreamEx.of(head.subList(1, size), next).toFlatList(l -> l)), size).prepend(head)
            : sliding(tail.mapFirst(next -> StreamEx.of(head, next).toFlatList(l -> l)), size));
}

public static <T> Function<StreamEx<T>, StreamEx<List<T>>> sliding(int size) {
    return s -> sliding(s.map(Collections::singletonList), size);
}

Use:

StreamEx.of(1,2,3,4,5).chain(sliding(3)).forEach(System.out::println);

Output:

[1, 2, 3]
[2, 3, 4]
[3, 4, 5]

@oleg-smith
Copy link
Author

yeah, and for non-overlapping case - just to set some other params?

@amaembo
Copy link
Owner

amaembo commented Sep 24, 2017

For batches there's another sample. In utility class:

private static <T> StreamEx<List<T>> batches(StreamEx<T> input, int size, List<T> cur) {
    return input.headTail((head, tail) -> cur.size() >= size
                    ? batches(tail, size, asList(head)).prepend(cur)
                    : batches(tail, size, StreamEx.of(cur).append(head).toList()),
            () -> Stream.of(cur));
}

public static <T> Function<StreamEx<T>, StreamEx<List<T>>> batches(int size) {
    return s -> batches(s, size, Collections.emptyList());
}

Use:

StreamEx.of(1,2,3,4,5,6,7,8).chain(batches(3)).forEach(System.out::println);

Output:

[1, 2, 3]
[4, 5, 6]
[7, 8]

@oleg-smith
Copy link
Author

Thank you!

@mkobit
Copy link

mkobit commented Apr 9, 2024

In Java 22 from JEP-461, the Gatherer API will support this type of aggregation and emission in java.util.Stream.

There looks to be some built-ins right now, similar to Collectors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants