Project pages are not indexable by googlebot

[paulkaplan]

Project pages do not appear to be correctly indexed by google. Although the relevant meta tags appear, the bot still does not consider each page its own separate canonical url, it points them all to a single random "google selected" canonical.

Using the interactive google search console, you can tell that the project pages show the "woops browser unsupported" full screen error modal. This appears to be because the crawler refuses to make all the necessary network calls to load project assets, which causes an error. But because the google bot is identifying itself as chrome 43 instead of the chrome 70+ that it actually uses, we show our "you hit an error AND you are using an old browser, which is a full screen takeover. I believe the full screen takeover is why it thinks all project pages are the same, but that is really just speculation. I created scratchfoundation/scratch-gui#5333 to attempt a fix at this.

/cc @rschamp assigning to january for now, but can reprioritize if we want.

[Explosion-Scratch]

This may be because on /robots.txt is says not to index project pages because the projects could be unshared at the time, it also may not distinguish between shared and unshared projects.