Lazy Auto-population of fields option

[pypoulp]

Hello there,

First, thanks a lot for this amazing project ! I'm quite familiar with Python, not that much with Mongo, but it looks super easy with Beanie 😉.

I was thinking about Projections, and ways to optimize queries a bit. Some kind of lazy auto-population of fields could be great in some cases.

To explain, lets take an example from the docs:

bar = await Product.find_one(Product.name == "Mars")
bar.price = 10
await bar.save()

• If the "bar" document is quite huge, we still need to fetch the full doc with find_one just to update the price.
• If we use projections to just get the price, it's not possible to save the new price in database / to use methods of Bar object.

I've used Ftrack Python API in the past, the default behavior is auto-population of fields, and you can use Projections to minimize the number of queries and fetch more data at once.

With the previous example, it may look a bit like:

# Following command would just fetch the _id of the bar, other fields are not initialized.
bar = await Product.find_one_lazy(Product.name == "Mars")
# If we access a specific non-initialized field, a query is performed on the fly to just get this value, and it's cached.
print(bar.price)
# When we set a value, it also initialize the field if not initialized yet, and do Pydantic validation.
bar.name = "foo"
await bar.save_changes()

So it's a bit like actual state managment, but with dynamic population of fields.

We can also think about a way to pass a Model to pre-fetch only some fields:

class BarPrice(BaseModel):
    price: int

# Following command would just fetch the _id and price of the bar, other fields are not initialized.
bar = await Product.find_one_lazy(Product.name == "Mars",  select=BarPrice)

In a lot of cases, it's fine to get the full document and minimize the number of queries, but IMO this approach would be great when you just need to manipulate few fields of a large document.

Maybe I'm misunderstanding something and there is already a workaround to address this thing. Using Motor directly can be a solution.

I can try to prototype something, but I'd like to have your feedback before :)

[roman-right]

Wow. THis looks really interesting. I didn't think about such a feature. Thank you! I'll add this to the todo