Library fails to parse google.com

[mergesort]

Hi @satoshi-takano, thank you so much for this library — I've been experimenting with it tonight and have found it very useful already! I'm testing out quite a few sites and noticed that OpenGraph fails to parse some bare domains like google.com and youtube.com.

---

I ran both of those through opengraph.xyz and they seem to work fine there.
https://www.opengraph.xyz/url/https%3A%2F%2Fgoogle.com
https://www.opengraph.xyz/url/https%3A%2F%2Fyoutube.com

The library though returns no metadata.

let metadata = try await OpenGraph.fetch(url: url)
print(metadata)

// Prints OpenGraph(source: [:])

---

I was wondering if there was something I need to configure to get these domains to parse, or possibly even special-case considerations that would help them work right.

Thank you again, I'm very excited to continue integrating your library. 😄

[Iomegan]

Did you try different user agents?

I.e.:

var headers = [String: String]()
headers["User-Agent"] = "facebookexternalhit/1.1"
headers["charset"] = "UTF-8"
OpenGraph.fetch(url: url, headers: headers) { result in
    (...)
}

[mergesort]

Thanks for the response @Iomegan, I did indeed try that beforehand but excluded it because I wanted to provide the simplest reproducible example.

An API call that looks like this fails to return any metadata as well.

let metadata = try await OpenGraph.fetch(
    url: URL(string: "https://google.com")!,
    headers: [
        "User-Agent" : "facebookexternalhit/1.1",
        "charset" : "UTF-8"
    ]
)

Is there anything else I should try or consider?

[Iomegan]

Hmm. With the header set I do get some metadata (on macOS with the network entitlement enabled) for youtube.com:
OpenGraph(source: [OpenGraph.OpenGraphMetadata.image: "https://www.youtube.com/img/desktop/yt_1200.png"])
For google.com I only got a response the first time I run the code:

OpenGraph(source: [OpenGraph.OpenGraphMetadata.description: "Zu Ehren von Mama Cax! #GoogleDoogle", OpenGraph.OpenGraphMetadata.imageWidth: "1087", OpenGraph.OpenGraphMetadata.image: "https://www.google.com/logos/doodles/2023/celebrating-mama-cax-6753651837110013.2-2x.png", OpenGraph.OpenGraphMetadata.imageHeight: "460"])

But now google only returns empty metadata.

OpenGraph(source: [:])

I'm not sure why, but looks like an Open Graph bug to me.