Is Filtering/FlatMap supported? about parallel-collectors HOT 7 CLOSED

Hi @davidmav!

Thanks! When I started writing it... I was surprised that no one did that before!

So, Parallel Collectors are just... a set of collectors for existing Stream API. There's no reflection, bytecode injection, or any other black magic - just using Stream API public interfaces.

This makes it fully compatible with any construct of Stream API. Technically, you should be able to replace any Collectors.toList() with any of these, and that should just work :)

So, to answer directly your question, yes you can. Check the following copy-paste'able snippet:

    public static void main(String[] args) {

    ExecutorService executor = Executors.newFixedThreadPool(2);

    List<List<Integer>> lists = asList(asList(1), asList(2, 3), asList(4, 5, 6));

    List<String> join = lists.stream()
      .flatMap(list -> list.stream())
      .filter(i -> i % 2 == 0)
      .collect(ParallelCollectors.parallelToList(i -> foo(i), executor))
      .join();

    System.out.println(join);

    executor.shutdownNow();
}

private static String foo(Integer i) {
    try {
        Thread.sleep(1000);
    } catch (InterruptedException e) {
        throw new IllegalStateException(e);
    }
    return i.toString();
}

Does that help?

from parallel-collectors.

Let me know if you still have questions, closing for now

from parallel-collectors.

Hi @pivovarit I wanted to follow up on this question, in your example does .filter(i -> i % 2 == 0) get parallelized as well?

from parallel-collectors.

No, just the part that lands in the collector

from parallel-collectors.

Hi @pivovarit

Thank you for the quick response -- do you have any advice on how I can go about parallelizing filtering?

from parallel-collectors.

For now, you could try a trick in which you do the filtering inside a map() of a parallel collector so that you end up with Stream<Boolean> or some form of a pair <Stream<Pair<Boolean, T>> and then you can simply do the cheap filtering sequentially

from parallel-collectors.

from parallel-collectors.