[libc++] P2502R2: std::generator
#92213
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
xiaoyang-sde
added
the
libc++
|
@llvm/pr-subscribers-libcxx Author: Xiaoyang Liu (xiaoyang-sde) ChangesIntroductionThis patch attempts to implement std::generator<int> fib() {
int a = 0;
int b = 1;
while (true) {
co_yield std::exchange(a, std::exchange(b, a + b));
}
}
auto fib_vec = fib() | std::views::take(5) | std::ranges::to<std::vector<int>>();As of 05/14/2024, this patch is in its initial phase. During this stage, the ImplementationIn practice, each generator owns a coroutine. The coroutine uses the
If the generator can't be recursive and doesn't support custom allocators, it can be implemented in fewer than 100 lines. It's recommended to understand how it works before proceeding. The code is available on Compiler Explorer. Complexities arise when the generator can be recursive. It should be able to yield another generator, which in turn can yield another generator, and so forth. For example, this capability is useful in implementing a lazily-evaluated tree traversal algorithm: struct tree_node {
tree_node* left;
tree_node* right;
int element;
};
std::generator<int> traversal(tree_node* node) {
if (node == nullptr) {
co_return;
}
co_yield std::ranges::elements_of(traversal(node->left));
co_yield node->element;
co_yield std::ranges::elements_of(traversal(node->right));
}To illustrate the implementation, consider a scenario where generator The client code only has access to struct __root_data {
add_pointer_t<_Yielded> __value_ptr;
std::coroutine_handle<__gen_promise_base> __active;
};The promise type of struct __recursive_data {
std::coroutine_handle<__gen_promise_base> __parent;
std::coroutine_handle<__gen_promise_base> __root;
};Given that a generator can be either a root generator ( template <class _Yielded>
class __gen_promise_base {
private:
/* ... */
union __union {
__root_data __root;
__recursive_data __recursive;
} __data_;
bool __tag_;
/* ... */
}Given the information provided, it would be easier to read the code. Complicated logics, such as transferring control from a recursive generator to its parent, are well-documented in the code. ReferencePatch is 24.51 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/92213.diff 9 Files Affected:
diff --git a/libcxx/include/CMakeLists.txt b/libcxx/include/CMakeLists.txt
index 96d7b4037e106..73efd343f6ac0 100644
--- a/libcxx/include/CMakeLists.txt
+++ b/libcxx/include/CMakeLists.txt
@@ -646,6 +646,7 @@ set(files
__ranges/data.h
__ranges/drop_view.h
__ranges/drop_while_view.h
+ __ranges/elements_of.h
__ranges/elements_view.h
__ranges/empty.h
__ranges/empty_view.h
@@ -949,6 +950,7 @@ set(files
fstream
functional
future
+ generator
initializer_list
inttypes.h
iomanip
diff --git a/libcxx/include/__ranges/elements_of.h b/libcxx/include/__ranges/elements_of.h
new file mode 100644
index 0000000000000..f446d3c12d67e
--- /dev/null
+++ b/libcxx/include/__ranges/elements_of.h
@@ -0,0 +1,55 @@
+// -*- C++ -*-
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef _LIBCPP___RANGES_ELEMENTS_OF_H
+#define _LIBCPP___RANGES_ELEMENTS_OF_H
+
+#include <__config>
+#include <__memory/allocator.h>
+#include <__ranges/concepts.h>
+#include <__utility/move.h>
+#include <cstddef>
+
+#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
+# pragma GCC system_header
+#endif
+
+_LIBCPP_PUSH_MACROS
+#include <__undef_macros>
+
+_LIBCPP_BEGIN_NAMESPACE_STD
+
+#if _LIBCPP_STD_VER >= 23
+
+namespace ranges {
+
+template <range _Range, class _Allocator = allocator<byte>>
+struct elements_of {
+ _LIBCPP_NO_UNIQUE_ADDRESS _Range range;
+ _LIBCPP_NO_UNIQUE_ADDRESS _Allocator allocator;
+
+ // This explicit constructor is required because AppleClang 15 hasn't implement P0960R3
+ _LIBCPP_HIDE_FROM_ABI explicit constexpr elements_of(_Range __range, _Allocator __alloc = _Allocator())
+ : range(std::move(__range)), allocator(std::move(__alloc)) {}
+};
+
+template <class _Range, class _Allocator = allocator<byte>>
+// This explicit constraint is required because AppleClang 15 might not deduce the correct type for `_Range` without it
+ requires range<_Range&&>
+elements_of(_Range&&, _Allocator = _Allocator()) -> elements_of<_Range&&, _Allocator>;
+
+} // namespace ranges
+
+#endif // _LIBCPP_STD_VER >= 23
+
+_LIBCPP_END_NAMESPACE_STD
+
+_LIBCPP_POP_MACROS
+
+#endif // _LIBCPP___RANGES_ELEMENTS_OF_H
diff --git a/libcxx/include/generator b/libcxx/include/generator
new file mode 100644
index 0000000000000..f4e15fae46023
--- /dev/null
+++ b/libcxx/include/generator
@@ -0,0 +1,377 @@
+// -*- C++ -*-
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef _LIBCPP_GENERATOR
+#define _LIBCPP_GENERATOR
+
+#include <__assert>
+#include <__concepts/constructible.h>
+#include <__concepts/convertible_to.h>
+#include <__config>
+#include <__coroutine/coroutine_handle.h>
+#include <__coroutine/coroutine_traits.h>
+#include <__coroutine/noop_coroutine_handle.h>
+#include <__coroutine/trivial_awaitables.h>
+#include <__exception/exception_ptr.h>
+#include <__iterator/default_sentinel.h>
+#include <__memory/addressof.h>
+#include <__memory/allocator_arg_t.h>
+#include <__memory/allocator_traits.h>
+#include <__ranges/concepts.h>
+#include <__ranges/elements_of.h>
+#include <__ranges/view_interface.h>
+#include <__type_traits/add_pointer.h>
+#include <__type_traits/common_reference.h>
+#include <__type_traits/conditional.h>
+#include <__type_traits/is_nothrow_constructible.h>
+#include <__type_traits/is_object.h>
+#include <__type_traits/is_reference.h>
+#include <__type_traits/is_void.h>
+#include <__type_traits/remove_cvref.h>
+#include <__utility/exchange.h>
+#include <__utility/swap.h>
+
+#if _LIBCPP_STD_VER >= 23
+
+_LIBCPP_BEGIN_NAMESPACE_STD
+
+template <class _Ref, class _Val>
+using __gen_val = conditional_t<is_void_v<_Val>, remove_cvref_t<_Ref>, _Val>;
+
+template <class _Ref, class _Val>
+using __gen_ref = conditional_t<is_void_v<_Val>, _Ref&&, _Ref>;
+
+template <class _Ref, class _Val>
+using __gen_yielded =
+ conditional_t<is_reference_v<__gen_ref<_Ref, _Val>>, __gen_ref<_Ref, _Val>, const __gen_ref<_Ref, _Val>&>;
+
+template <class, class, class>
+class generator;
+
+template <class _Yielded>
+class __gen_promise_base {
+private:
+ template <class, class, class>
+ friend class generator;
+
+ template <class, class>
+ friend class __gen_iter;
+
+ // Each promise object stores either a `__root_data` when associated with a root generator, or a `__recursive_data`
+ // when associated with a generator that is yielded recursively.
+ struct __root_data {
+ // The client code has access only to the iterator of the root generator. Thus, the root generator must store the
+ // yielded values of recursively-yielded generators, which will then be returned when the client code dereferences
+ // the iterator.
+ add_pointer_t<_Yielded> __value_ptr;
+ // The client code has access only to the iterator of the root generator. Thus, the root generator needs to identify
+ // which generator is currently active. This active generator will then be resumed when the client code increments
+ // the iterator.
+ std::coroutine_handle<__gen_promise_base> __active;
+ };
+
+ struct __recursive_data {
+ std::exception_ptr __exception;
+ std::coroutine_handle<__gen_promise_base> __parent;
+ std::coroutine_handle<__gen_promise_base> __root;
+ };
+
+ union __union {
+ __root_data __root;
+ __recursive_data __recursive;
+
+ ~__union() noexcept {}
+ } __data_;
+
+ // The `__tag_` stores the active member of the `__data_` union:
+ // - `false` indicates that the active member is `__root`.
+ // - `true` indicates that the active member is `__recursive`.
+ // This field can be omitted because `__recursive_data.__root` can store the active member.
+ bool __tag_;
+
+ [[nodiscard]] _LIBCPP_HIDE_FROM_ABI bool __is_root() noexcept { return !__tag_; }
+ [[nodiscard]] _LIBCPP_HIDE_FROM_ABI __root_data& __get_root_data() noexcept {
+ _LIBCPP_ASSERT_INTERNAL(__is_root(), "the active member of `__data_` is not `__root`");
+ return __data_.__root;
+ }
+ _LIBCPP_HIDE_FROM_ABI void __set_root_tag() noexcept { __tag_ = false; }
+
+ [[nodiscard]] _LIBCPP_HIDE_FROM_ABI bool __is_recursive() noexcept { return __tag_; }
+ [[nodiscard]] _LIBCPP_HIDE_FROM_ABI __recursive_data& __get_recursive_data() noexcept {
+ _LIBCPP_ASSERT_INTERNAL(__is_recursive(), "the active member of `__data_` is not `__recursive`");
+ return __data_.__recursive;
+ }
+ _LIBCPP_HIDE_FROM_ABI void __set_recursive_tag() noexcept { __tag_ = true; }
+
+ [[nodiscard]] _LIBCPP_HIDE_FROM_ABI std::coroutine_handle<__gen_promise_base>& __active() noexcept {
+ _LIBCPP_ASSERT_INTERNAL(__is_root(), "the active member of `__data_` is not `__root`");
+ return __get_root_data().__active;
+ }
+
+ [[nodiscard]] _LIBCPP_HIDE_FROM_ABI add_pointer_t<_Yielded>& __value_ptr() noexcept {
+ _LIBCPP_ASSERT_INTERNAL(__is_root(), "the active member of `__data_` is not `__root`");
+ return __get_root_data().__value_ptr;
+ }
+
+ [[nodiscard]] _LIBCPP_HIDE_FROM_ABI add_pointer_t<_Yielded>& __root_value_ptr() noexcept {
+ if (__is_root()) {
+ return __value_ptr();
+ }
+ return __get_recursive_data().__root.promise().__value_ptr();
+ }
+
+ struct __element_awaiter {
+ [[nodiscard]] _LIBCPP_HIDE_FROM_ABI bool await_ready() noexcept { return false; }
+
+ template <class _Promise>
+ _LIBCPP_HIDE_FROM_ABI void await_suspend(coroutine_handle<_Promise> __current) noexcept {
+ __current.promise().__root_value_ptr() = std::addressof(__value);
+ }
+
+ _LIBCPP_HIDE_FROM_ABI void await_resume() noexcept {}
+
+ remove_cvref_t<_Yielded> __value;
+ };
+
+ struct __final_awaiter {
+ [[nodiscard]] _LIBCPP_HIDE_FROM_ABI bool await_ready() noexcept { return false; }
+
+ template <class _Promise>
+ _LIBCPP_HIDE_FROM_ABI coroutine_handle<> await_suspend(coroutine_handle<_Promise> __current) noexcept {
+ // Checks if the current generator is recursively-yielded
+ if (__current.promise().__is_recursive()) {
+ auto&& __recursive_data = __current.promise().__get_recursive_data();
+ auto __parent = __recursive_data.__parent;
+ // Updates the active generator to its parent, allowing the client code to resume it later
+ __recursive_data.__root.promise().__active() = __parent;
+ // Transfers execution to its parent, which is the generator that yields it
+ return __parent;
+ }
+ return std::noop_coroutine();
+ }
+
+ _LIBCPP_HIDE_FROM_ABI void await_resume() noexcept {}
+ };
+
+ template <class _Ref, class _Val, class _Allocator>
+ struct __recursive_awaiter {
+ generator<_Ref, _Val, _Allocator> __gen;
+
+ _LIBCPP_HIDE_FROM_ABI explicit __recursive_awaiter(generator<_Ref, _Val, _Allocator>&& __gen) noexcept
+ : __gen{std::move(__gen)} {}
+
+ [[nodiscard]] _LIBCPP_HIDE_FROM_ABI bool await_ready() noexcept { return !__gen.__coroutine_; }
+
+ template <class _Promise>
+ _LIBCPP_HIDE_FROM_ABI coroutine_handle<__gen_promise_base>
+ await_suspend(coroutine_handle<_Promise> __current) noexcept {
+ // Stores a `__recursive_data` in the promise object associated with `__gen`
+ auto __recursive = coroutine_handle<__gen_promise_base>::from_address(__gen.__coroutine_.address());
+ __recursive.promise().__set_recursive_tag();
+ auto&& __recursive_data = __recursive.promise().__get_recursive_data();
+
+ // Sets `__recursive_data.__parent` to the current generator
+ auto __parent = coroutine_handle<__gen_promise_base>::from_address(__current.address());
+ __recursive_data.__parent = __parent;
+
+ // Sets `__recursive_data.__root` to the current generator if it's a root generator, or to the root generator of
+ // the current generator otherwise
+ if (__parent.promise().__is_recursive()) {
+ __recursive_data.__root = __parent.promise().__get_recursive_data().__root;
+ } else {
+ __recursive_data.__root = __parent;
+ }
+
+ // Updates the active generator to `__gen`, allowing the client code to resume it later
+ __recursive_data.__root.promise().__active() = __recursive;
+
+ // Transfers execution to `__gen`
+ return __recursive;
+ }
+
+ _LIBCPP_HIDE_FROM_ABI void await_resume() {
+ auto __recursive = coroutine_handle<__gen_promise_base>::from_address(__gen.__coroutine_.address());
+ auto&& __recursive_data = __recursive.promise().__get_recursive_data();
+ if (__recursive_data.__exception) {
+ std::rethrow_exception(std::move(__recursive_data.__exception));
+ }
+ }
+ };
+
+public:
+ _LIBCPP_HIDE_FROM_ABI __gen_promise_base() noexcept
+ : __data_{.__root =
+ {
+ .__value_ptr = nullptr,
+ .__active = coroutine_handle<__gen_promise_base>::from_promise(*this),
+ }},
+ __tag_{false} {}
+
+ _LIBCPP_HIDE_FROM_ABI ~__gen_promise_base() noexcept {
+ if (__is_root()) {
+ __data_.__root.~__root_data();
+ } else {
+ __data_.__recursive.~__recursive_data();
+ }
+ }
+
+ _LIBCPP_HIDE_FROM_ABI suspend_always initial_suspend() const noexcept { return {}; }
+
+ _LIBCPP_HIDE_FROM_ABI __final_awaiter final_suspend() noexcept { return {}; }
+
+ _LIBCPP_HIDE_FROM_ABI suspend_always yield_value(_Yielded __value) noexcept {
+ __root_value_ptr() = addressof(__value);
+ return {};
+ }
+
+ _LIBCPP_HIDE_FROM_ABI auto yield_value(const remove_reference_t<_Yielded>& __value)
+ requires is_rvalue_reference_v<_Yielded> &&
+ constructible_from<remove_cvref_t<_Yielded>, const remove_reference_t<_Yielded>&>
+ {
+ return __element_awaiter{.__value = __value};
+ }
+
+ template <class _Ref2, class _Val2, class _Allocator2, class _Unused>
+ requires same_as<__gen_yielded<_Ref2, _Val2>, _Yielded>
+ _LIBCPP_HIDE_FROM_ABI auto
+ yield_value(ranges::elements_of<generator<_Ref2, _Val2, _Allocator2>&&, _Unused> __elements) noexcept {
+ return __recursive_awaiter<_Ref2, _Val2, _Allocator2>{std::move(__elements.range)};
+ }
+
+ template <ranges::input_range _Range, class _Allocator>
+ requires convertible_to<ranges::range_reference_t<_Range>, _Yielded>
+ _LIBCPP_HIDE_FROM_ABI auto yield_value(ranges::elements_of<_Range, _Allocator> __range) {
+ auto __lambda =
+ [](allocator_arg_t, _Allocator, ranges::iterator_t<_Range> __i, ranges::sentinel_t<_Range> __s) static
+ -> generator<_Yielded, ranges::range_value_t<_Range>, _Allocator> {
+ for (; __i != __s; ++__i) {
+ co_yield static_cast<_Yielded>(*__i);
+ }
+ };
+ return yield_value(ranges::elements_of(
+ __lambda(allocator_arg, __range.allocator, ranges::begin(__range.range), ranges::end(__range.range))));
+ }
+
+ _LIBCPP_HIDE_FROM_ABI void await_transform() = delete;
+
+ _LIBCPP_HIDE_FROM_ABI void return_void() const noexcept {}
+
+ _LIBCPP_HIDE_FROM_ABI void unhandled_exception() {
+ if (__is_root()) {
+ throw;
+ } else {
+ __get_recursive_data().__exception = std::current_exception();
+ }
+ }
+};
+
+template <class _Ref, class _Val>
+class __gen_iter {
+private:
+ using __val = __gen_val<_Ref, _Val>;
+ using __ref = __gen_ref<_Ref, _Val>;
+
+public:
+ using value_type = __val;
+ using difference_type = ptrdiff_t;
+
+ _LIBCPP_HIDE_FROM_ABI explicit __gen_iter(
+ coroutine_handle<__gen_promise_base<__gen_yielded<_Ref, _Val>>> __coroutine) noexcept
+ : __coroutine_{std::move(__coroutine)} {}
+
+ _LIBCPP_HIDE_FROM_ABI __gen_iter(__gen_iter&& __other) noexcept
+ : __coroutine_{std::exchange(__other.__coroutine_, {})} {}
+
+ _LIBCPP_HIDE_FROM_ABI __gen_iter& operator=(__gen_iter&& __other) noexcept {
+ __coroutine_ = std::exchange(__other.__coroutine_, {});
+ return *this;
+ }
+
+ [[nodiscard]] _LIBCPP_HIDE_FROM_ABI __ref operator*() const noexcept(is_nothrow_copy_constructible_v<__ref>) {
+ return static_cast<__ref>(*__coroutine_.promise().__value_ptr());
+ }
+
+ _LIBCPP_HIDE_FROM_ABI __gen_iter& operator++() {
+ __coroutine_.promise().__active().resume();
+ return *this;
+ }
+
+ _LIBCPP_HIDE_FROM_ABI void operator++(int) { ++*this; }
+
+ [[nodiscard]] _LIBCPP_HIDE_FROM_ABI bool operator==(default_sentinel_t) const noexcept { return __coroutine_.done(); }
+
+private:
+ coroutine_handle<__gen_promise_base<__gen_yielded<_Ref, _Val>>> __coroutine_;
+};
+
+template <class _Ref, class _Val = void, class _Allocator = void>
+class generator : public ranges::view_interface<generator<_Ref, _Val, _Allocator>> {
+private:
+ using __val = __gen_val<_Ref, _Val>;
+ static_assert(same_as<remove_cvref_t<__val>, __val> && is_object_v<__val>);
+
+ using __ref = __gen_ref<_Ref, _Val>;
+ static_assert(is_reference_v<__ref> ||
+ (same_as<remove_cvref_t<__ref>, __ref> && is_object_v<__ref> && copy_constructible<__ref>));
+
+ using __rref = conditional_t<is_lvalue_reference_v<_Ref>, remove_reference_t<_Ref>&&, _Ref>;
+ static_assert(common_reference_with<__ref&&, __val&> && common_reference_with<__ref&&, __rref&&> &&
+ common_reference_with<__rref&&, const __val&>);
+
+ template <class>
+ friend class __gen_promise_base;
+
+public:
+ using yielded = __gen_yielded<_Ref, _Val>;
+
+ struct promise_type : __gen_promise_base<yielded> {
+ [[nodiscard]] _LIBCPP_HIDE_FROM_ABI generator get_return_object() noexcept {
+ return generator{coroutine_handle<promise_type>::from_promise(*this)};
+ }
+ };
+
+ _LIBCPP_HIDE_FROM_ABI generator(const generator&) = delete;
+ _LIBCPP_HIDE_FROM_ABI generator(generator&& __other) noexcept
+ : __coroutine_{std::exchange(__other.__coroutine_, {})} {}
+
+ _LIBCPP_HIDE_FROM_ABI ~generator() {
+ if (__coroutine_) {
+ __coroutine_.destroy();
+ }
+ }
+
+ _LIBCPP_HIDE_FROM_ABI generator& operator=(generator __other) noexcept {
+ std::swap(__coroutine_, __other.__coroutine_);
+ return *this;
+ }
+
+ [[nodiscard]] _LIBCPP_HIDE_FROM_ABI __gen_iter<_Ref, _Val> begin() {
+ auto __coroutine = std::coroutine_handle<__gen_promise_base<yielded>>::from_promise(__coroutine_.promise());
+ __coroutine.resume();
+ return __gen_iter<_Ref, _Val>{__coroutine};
+ }
+
+ [[nodiscard]] _LIBCPP_HIDE_FROM_ABI default_sentinel_t end() const noexcept { return {}; }
+
+private:
+ _LIBCPP_HIDE_FROM_ABI explicit generator(coroutine_handle<promise_type> __coroutine) noexcept
+ : __coroutine_{__coroutine} {}
+
+ coroutine_handle<promise_type> __coroutine_ = nullptr;
+};
+
+_LIBCPP_END_NAMESPACE_STD
+
+#endif
+
+#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
+# pragma GCC system_header
+#endif
+
+#endif
diff --git a/libcxx/include/module.modulemap b/libcxx/include/module.modulemap
index 0ee0cdce61bc0..421963e0bc9a8 100644
--- a/libcxx/include/module.modulemap
+++ b/libcxx/include/module.modulemap
@@ -1699,6 +1699,7 @@ module std_private_ranges_dangling [system] { header "__ranges
module std_private_ranges_data [system] { header "__ranges/data.h" }
module std_private_ranges_drop_view [system] { header "__ranges/drop_view.h" }
module std_private_ranges_drop_while_view [system] { header "__ranges/drop_while_view.h" }
+module std_private_ranges_elements_of [system] { header "__ranges/elements_of.h" }
module std_private_ranges_elements_view [system] { header "__ranges/elements_view.h" }
module std_private_ranges_empty [system] { header "__ranges/empty.h" }
module std_private_ranges_empty_view [system] { header "__ranges/empty_view.h" }
diff --git a/libcxx/include/ranges b/libcxx/include/ranges
index 07a525ed8641f..a8fbfc462bf0d 100644
--- a/libcxx/include/ranges
+++ b/libcxx/include/ranges
@@ -116,6 +116,10 @@ namespace std::ranges {
// [range.dangling], dangling iterator handling
struct dangling;
+ // [range.elementsof], class template elements_of
+ template<range R, class Allocator = allocator<byte>>
+ struct elements_of;
+
template<range R>
using borrowed_iterator_t = see below;
@@ -392,6 +396,7 @@ namespace std {
#include <__ranges/data.h>
#include <__ranges/drop_view.h>
#include <__ranges/drop_while_view.h>
+#include <__ranges/elements_of.h>
#include <__ranges/elements_view.h>
#include <__ranges/empty.h>
#include <__ranges/empty_view.h>
diff --git a/libcxx/modules/std/ranges.inc b/libcxx/modules/std/ranges.inc
index f71efe948ede1..82055c2ddbfbb 100644
--- a/libcxx/modules/std/ranges.inc
+++ b/libcxx/modules/std/ranges.inc
@@ -83,8 +83,10 @@ export namespace std {
// [range.dangling], dangling iterator handling
using std::ranges::dangling;
+#if _LIBCPP_STD_VER >= 23
// [range.elementsof], class template elements_of
- // using std::ranges::elements_of;
+ using std::ranges::elements_of;
+#endif
using std::ranges::borrowed_iterator_t;
diff --git a/libcxx/test/std/ranges/coro.generator/generator.pass.cpp b/libcxx/test/std/ranges/coro.generator/generator.pass.cpp
new file mode 100644
index 0000000000000..c7e280f74b70a
--- /dev/null
+++ b/libcxx/test/std/ranges/coro.generator/generator.pass.cpp
@@ -0,0 +1,40 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// UNSUPPORTED: c++03, c++11, c++14, c++17, c++20
+
+// std::generator
+
+#include <generator>
+
+#include <cassert>
+#include <ranges>
+#include <utility>
+#include <vector>
+
+std::generator<int> fib() {
+ int a = 0;
+ int b = 1;
+ while (true) {
+ co_yield std::exchange(a...
[truncated]
|
|
This patch is based on #91414. The test cases are still under active development. |
H-G-Hristov
reviewed
May 15, 2024
H-G-Hristov
reviewed
May 15, 2024
Comment on lines
+354
to
+360
| [[nodiscard]] _LIBCPP_HIDE_FROM_ABI __gen_iter<_Ref, _Val> begin() { | ||
| auto __coroutine = std::coroutine_handle<__gen_promise_base<yielded>>::from_promise(__coroutine_.promise()); | ||
| __coroutine.resume(); | ||
| return __gen_iter<_Ref, _Val>{__coroutine}; | ||
| } | ||
|
|
||
| [[nodiscard]] _LIBCPP_HIDE_FROM_ABI default_sentinel_t end() const noexcept { return {}; } |
There was a problem hiding this comment.
I think you need to add [[nodiscard]] tests. I think they are located in /libcxx/tests/libcxx/diagnostics or something like that.
There was a problem hiding this comment.
Thanks! I'm still working on finalizing the test cases.
| class generator; | ||
|
|
||
| template <class _Yielded> | ||
| class __gen_promise_base { |
There was a problem hiding this comment.
Just a (personal) nit: IMO not abbreviating generator to gen would make it more readable and clear that these types are an implementation detail of generator, when reading the code randomly.
| } | ||
|
|
||
| [[nodiscard]] _LIBCPP_HIDE_FROM_ABI __gen_iter<_Ref, _Val> begin() { | ||
| auto __coroutine = std::coroutine_handle<__gen_promise_base<yielded>>::from_promise(__coroutine_.promise()); |
There was a problem hiding this comment.
FYI I am asked in reviews not to use auto usually. AFAIK I guess it is preferred to use the full type wherever possible.
hawkinsw
reviewed
May 15, 2024
| _LIBCPP_NO_UNIQUE_ADDRESS _Range range; | ||
| _LIBCPP_NO_UNIQUE_ADDRESS _Allocator allocator; | ||
|
|
||
| // This explicit constructor is required because AppleClang 15 hasn't implement P0960R3 |
There was a problem hiding this comment.
Suggested change
| // This explicit constructor is required because AppleClang 15 hasn't implement P0960R3 | |
| // This explicit constructor is required because AppleClang 15 hasn't implemented P0960R3 |
There was a problem hiding this comment.
Appreciate it! However, these files are cherry-picked from #91414. (It seems GitHub doesn't support stacking pull requests.)
| }; | ||
|
|
||
| template <class _Range, class _Allocator = allocator<byte>> | ||
| // This explicit constraint is required because AppleClang 15 might not deduce the correct type for `_Range` without it |
There was a problem hiding this comment.
Suggested change
| // This explicit constraint is required because AppleClang 15 might not deduce the correct type for `_Range` without it | |
| // This explicit constraint is required because AppleClang 15 might not deduce the correct type for `_Range` without it. |
xiaoyang-sde
force-pushed
the
generator
branch
3 times, most recently
from
4499587
to
a473480
Compare
Comment on lines
+289
to
+290
| template <class _Ref, class _Val> | ||
| class __gen_iter { |
There was a problem hiding this comment.
Perhaps the iterator type shouldn't be a class template, which makes _Ref and _Val interfer with ADL.
I changed the strategy of MSVC STL in microsoft/STL#4464. Note that it seems OK to keep __gen_promise_base (or _Gen_promise_base in MSVC STL) a class template after later thinking, but I didn't check this at the time of that PR.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Introduction
This patch attempts to implement
std::generatorfrom P2502R2.std::generatoris a view that represents a recursive coroutine-based generator.Implementation
In practice, each generator owns a coroutine. The coroutine uses the
co_yieldstatement to produce values for its consumer. Itsbegin()method returns an iterator with access to the coroutine handle.co_yieldstatement, at which point it suspends, storing the address of the yielded value within its promise.If the generator can't be recursive and doesn't support custom allocators, it can be implemented in fewer than 100 lines. It's recommended to understand how it works before proceeding. The code is available on Compiler Explorer.
Complexities arise when the generator can be recursive. It should be able to yield another generator, which in turn can yield another generator, and so forth. For example, this capability is useful in implementing a lazily-evaluated tree traversal algorithm:
To illustrate the implementation, consider a scenario where generator
Ayields another generatorB, which in turn yields a generatorC. Here,Ais the root generator because the client code has access to its iterator, whileBandCare recursive generators. Due to the lazily-evaluated nature of generators,Acannot consume all elements ofBat once, store them in a container, and yield them individually. Instead, when the iterator ofAis incremented, it should directly resume the coroutine ofCand transfer control back to the client code onceC's coroutine has yielded a value.The client code only has access to
A's iterator, which doesn't know the existence ofBandC. Consequently,BandCare responsible for writing their yielded values toA's promise, which will subsequently be returned to the client code upon dereferencingA's iterator. When incrementingA's iterator, merely resumingA's coroutine is insufficient, asB's orC's coroutine might still be yielding values. Consequently,A's promise must store the coroutine handle of the currently active coroutine, which will be resumed upon incrementingA's iterator.The promise type of
BandCneeds to store the coroutine handle of its parent, allowing the transfer of control back to the parent upon completion. Additionally, it must store the coroutine handle of its root generator, as it's responsible for writing the address of the yielded value to the root generator's promise, as explained earlier.Given that a generator can be either a root generator (
A) or a recursive generator (BandC), the promise type can use a tagged union to store the required data based on the generator's type. (In practice, additional fields are included to enable the propagation of exceptions out of recursive generators.)Given the information provided, it would be easier to read the code. Complicated logics, such as transferring control from a recursive generator to its parent, are well-documented in the code.
Reference
std::generator: Synchronous Coroutine Generator for Ranges