Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot access a private page with custom unofficialNotionAPI argument #9

Open
jshingler opened this issue Dec 5, 2023 · 8 comments
Open

Cannot access a private page with custom unofficialNotionAPI argument #9

jshingler opened this issue Dec 5, 2023 · 8 comments

Comments

@jshingler
Copy link

Hi, ... Got the Private Pages Info, ... Thanks.

Does the graph navigation handle Circular Links between pages.

When I run the app, ... it just runs and runs. It never finishes.
@9oelM
Copy link
Member

9oelM commented Dec 5, 2023

Do you mind sharing your code? I have tested with a test page where I created circular links between two pages, but it still works. You also might want to juggle some optional configs defined by Typescript.

Here's my page that acts as the root node: https://lizard-flax-e38.notion.site/Test-bc95525429244d169cad7f6725cb5b34?pvs=4

Running the scraper gives me the following result, without hanging:

{
  "nodes": [
    {
      "id": "bc955254-2924-4d16-9cad-7f6725cb5b34",
      "title": "Test",
      "type": "page",
      "spaceId": "fda5cc70-a0d3-4f12-9b33-2ec0c26ab6ca",
      "parentId": "none",
      "cc": 2
    },
    {
      "id": "65786275-82a8-4c32-8bd7-11d49a3ff7c6",
      "parentId": "bc955254-2924-4d16-9cad-7f6725cb5b34",
      "spaceId": "fda5cc70-a0d3-4f12-9b33-2ec0c26ab6ca",
      "type": "page",
      "title": "A"
    },
    {
      "id": "175b9592-7cf5-4452-be34-da896731654b",
      "parentId": "bc955254-2924-4d16-9cad-7f6725cb5b34",
      "spaceId": "fda5cc70-a0d3-4f12-9b33-2ec0c26ab6ca",
      "type": "page",
      "title": "B"
    }
  ],
  "links": [
    {
      "source": "65786275-82a8-4c32-8bd7-11d49a3ff7c6",
      "target": "bc955254-2924-4d16-9cad-7f6725cb5b34"
    },
    {
      "source": "65786275-82a8-4c32-8bd7-11d49a3ff7c6",
      "target": "175b9592-7cf5-4452-be34-da896731654b"
    },
    {
      "source": "175b9592-7cf5-4452-be34-da896731654b",
      "target": "bc955254-2924-4d16-9cad-7f6725cb5b34"
    }
  ],
  "errors": []
}

@jshingler
Copy link
Author

I used the code as is from the info you gave me, .... the page id is the blog.
And I used the project from the example scapper and replaced index.js with this code

import { NotionGraph } from "@graphcentral/notion-graph-scraper"
import fs from "fs"
import { NotionAPI } from "notion-client";

/**

  • example of how to use @graphcentral/notion-graph-scraper
    */
    ;(async () => {
    const notionApi = new NotionAPI({
    activeUser: process.env.NOTION_ACTIVE_USER,
    authToken: process.env.NOTION_TOKEN_V2
    })
    const notionGraph = new NotionGraph({
    maxDiscoverableNodes: 10,
    maxDiscoverableNodesInOtherSpaces: 10,
    verbose: true,
    unofficialNotionAPI: notionApi
    })
    const graph = await notionGraph.buildGraphFromRootNode(
    // Some random Japanese blog
    '95fcfe03257541c5aaa21dd43bdbc381'
    )
    console.log(graph.nodes.length)
    console.log(graph.links.length)
    await new Promise((resolve, reject) => {
    fs.writeFile(test0.json, JSON.stringify(graph), (err) => {
    if (err) reject(err)
    else resolve(``)
    })
    });

process.exit(0)
})()

@9oelM
Copy link
Member

9oelM commented Dec 6, 2023

@jshingler but you configured process.env.NOTION_ACTIVE_USER and process.env.NOTION_TOKEN_V2 right?
what is the output when you run it?

@9oelM
Copy link
Member

9oelM commented Dec 6, 2023

and the japanese blog should be accessible without using the custom notionApi argument because it's already public

@jshingler
Copy link
Author

Yes I Configured the environment variables, ....
I am not using the Japanese Blog, ... I just didn't want to expose my private page id

Here is a portion of the output, it just keeps repeating

[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0

@9oelM
Copy link
Member

9oelM commented Dec 8, 2023 •
edited

Yeah if it keeps showing zeroes, it shouldn't be about circular links because there's no page to start with. There should be something about not being able to retrieve the information in the first place. Let me try with my own private pages and get back to you

@9oelM 9oelM changed the title Circular Links Cannot access a private page with custom unofficialNotionAPI argument Dec 8, 2023
@jshingler
Copy link
Author

Looking at this again tonight, ... here is the error message that shows up in test0.json.

d.map is not a function. in notion-utils/build/index.js

{"links":[],"nodes":[],"errors":[{"name":"TypeError","message":"d.map is not a function","stack":"TypeError: d.map is not a function\n at s (file:///Users/jshingler/Archive/Workspaces/graphcentral/notion-scrape-example/node_modules/notion-utils/build/index.js:1:4072)\n at s (file:///Users/jshingler/Archive/Workspaces/graphcentral/notion-scrape-example/node_modules/notion-utils/build/index.js:1:4308)\n at s (file:///Users/jshingler/Archive/Workspaces/graphcentral/notion-scrape-example/node_modules/notion-utils/build/index.js:1:4308)\n at s (file:///Users/jshingler/Archive/Workspaces/graphcentral/notion-scrape-example/node_modules/notion-utils/build/index.js:1:4308)\n at s (file:///Users/jshingler/Archive/Workspaces/graphcentral/notion-scrape-example/node_modules/notion-utils/build/index.js:1:4308)\n at s (file:///Users/jshingler/Archive/Workspaces/graphcentral/notion-scrape-example/node_modules/notion-utils/build/index.js:1:4308)\n at s (file:///Users/jshingler/Archive/Workspaces/graphcentral/notion-scrape-example/node_modules/notion-utils/build/index.js:1:4308)\n at s (file:///Users/jshingler/Archive/Workspaces/graphcentral/notion-scrape-example/node_modules/notion-utils/build/index.js:1:4308)\n at s (file:///Users/jshingler/Archive/Workspaces/graphcentral/notion-scrape-example/node_modules/notion-utils/build/index.js:1:4308)\n at s (file:///Users/jshingler/Archive/Workspaces/graphcentral/notion-scrape-example/node_modules/notion-utils/build/index.js:1:4308)"},{}]}

@jshingler
Copy link
Author

It works on VERY simple private pages, .... Seems to be React-Notion-X has issues with something on the more complex pages.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jshingler @9oelM