RFCs: Redesign ntast as Application API, abstract Notion API as low-level persistence layer

[phuctm97]

This discussion is created to discuss my new proposal for ntast to make it easier to work with Notion content at application layer while maintaining efficient communication with Notion API.

Why?

While Notion API block structure is brilliant to make partial query and transaction effective, it's too low-level from application-level constraints, which makes it hard and error-prone to work with at application layer.

Notion API block structure is already presented very well in notion-types, big shoutout to @transitive-bullshit 🙌🏻.

Glossary

• Application layer: where a Notion-based end-user application is implemented, e.g. 3rd apps, Notion web/desktop app. At this layer, an app should work on only data structures that directly describe Notion-application features, e.g. a heading 1 block, a text block can be bold, italic, etc.

• Persistence layer: where changes on application layer are persisted. This is done by communicating with Notion API.

Notion API is unaware of Application layer and has its own protocol for Persistence layer.

---

Goals

• ntast syntax trees directly represent how Notion content is being represented to end-users
• Communicate changes efficiently with Notion API
• Maintain compatibility with unist ecosystem

Notion API query objects ↔️ ntast

This can be done easily as ntast is a high-level representation of Notion query objects. However, this needs to be done in a way that changes can be detected.

In Notion, there're 3 types of changes:

• Block creation and deletion.
• Block's property change.
• Children's order change.

Changes should be detected as simple as:

const changes = diff(treeBefore, treeAfter);

Block creation and deletion

This can be done by maintaining an id of a block.

• A block that has an id is considered pre-existed block, which may be changed or deleted.
• A block that has no id is a new block.

Block property change

This can be done by checking deep equality between 2 instances of the same block id (using deep-equal library).

Children's order change

[WIP]

ntast ↔️ Notion API transaction

Thanks to the ability to compare and detect changes from 2 instances of the same tree, transactions can be created:

const originalTree = ntastParse(queryObjects);

const currTree = clone(originalTree);
// Make changes to currTree
// ...

const changes = ntastDiff(originalTree, currTree);
const transaction = ntastCreateTransaction(changes);

Syntax trees

List of syntax trees implementing above assumptions.

[WIP]

[phuctm97]

There're more to add, but the general idea is described, comments and thoughts are highly appreciated!

[phuctm97]

I need. to spend some time on Notion Tweet, will get back and update it after the early launch. Meanwhile, would love to see any feedback or idea.