update development environment guide #7612

zoran995 · 2025-04-26T15:13:47Z

What this PR does

Depends on TerriaJS/TerriaMap#746
TerriaMap has been configured with workspaces for some time, so we should update the docs to reflect this.

Test me

How should reviewers test this? (Hint: If you want to provide a test catalog item, create a Gist of its catalog JSON, add its url and your branch name to this url: http://ci.terria.io/<branch name>/#clean&<raw url of your gist> , and then paste that in the body of this PR)

Checklist

~~There are unit tests to verify my changes are correct or unit tests aren't applicable (if so, write quick reason why unit tests don't exist)~~
I've updated relevant documentation in doc/.
I've updated CHANGES.md with what I changed.
I've provided instructions in the PR description on how to test this PR.

niktayv

I hoped that it would work, but I was unlucky again.
Perhaps I missed something important.

niktayv · 2025-05-01T03:14:20Z

doc/contributing/development-environment.md

+To use `yarn` workspaces you need to have `yarn` installed globally, recommended version is 1.18.0, and you can install it by running

 ```
-npm install -g yarn
+npm install -g yarn@^1.19.0
 ```


AFAIK, the version of Yarn is locked in TerriaMap/.yarn/releases/yarn-1.18.0.js, so running just npm install -g yarn would suffice.

We should revisit the version of yarn we use as terriajs CI uses 1.19.0

niktayv · 2025-05-01T03:16:59Z

doc/contributing/development-environment.md

-
-Do _not_ commit this change.
-
 Now, you can clone any package (such as `terriajs` or `terriajs-cesium`) into the `packages` directory:


Would it be more helpful to keep a mention of the "workspaces" property here?

Not sure what to add here, we already mentioned yarn workspaces and the user should get familiar with what it is to be able to work with it effectively

niktayv · 2025-05-01T03:23:28Z

doc/contributing/using-a-custom-version-of-cesium.md

+Make sure that the `terriamap/package.json` workspaces list contains the following packages:
+
+- "packages/terriajs-cesium/packages/engine",
+- "packages/terriajs-cesium/packages/widgets"
+
+Check the versions of `terriajs-cesium` and `terriajs-cesium-widgets` in `terriamap/package.json` and `terriamap/packages/terriajs/package.json` and make sure they are the same so yarn can properly create symlinks.


This is very illuminating and I hoped that it would resolve my issues, but despite all my efforts, such as double-checking all the versions and package names alignment in the package.json files, running yarn install runs into an error:

$ yarn install yarn install v1.18.0 [1/5] 🔍 Validating package.json... [2/5] 🔍 Resolving packages... [3/5] 🚚 Fetching packages... [...] [5/5] 🔨 Building fresh packages... error /Users/yuri.vyatkin/Terria/TerriaMap/node_modules/terriajs: Command failed. Exit code: 1 Command: gulp post-npm-install Arguments: Directory: /Users/yuri.vyatkin/Terria/TerriaMap/node_modules/terriajs Output: [14:51:56] Using gulpfile ~/Terria/TerriaMap/packages/terriajs/gulpfile.js [14:51:56] Starting 'post-npm-install'... [14:51:56] Starting 'copy-cesium-assets'... [14:51:56] Starting 'copy-cesium-source-assets'... [14:51:56] Finished 'copy-cesium-source-assets' after 68 ms [14:51:56] Starting 'copy-cesium-workers'... [14:51:56] 'copy-cesium-workers' errored after 1.31 ms [14:51:56] Error: ENOENT: no such file or directory, scandir '/Users/yuri.vyatkin/Terria/TerriaMap/packages/terriajs-cesium/packages/engine/Build/Workers' [14:51:56] 'copy-cesium-assets' errored after 70 ms [14:51:56] 'post-npm-install' errored after 71 ms info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command. $

I tried to delete node_modules in all the repos, and tried again, ending up with the same error.
Am I missing something?

Nanda @na9da clarified that we should run npm run release within packages/terriajs-cesium before running yarn install in TerriaMap.
Now, both yarn install and yarn gulp build work fine in TerriaMap.

Given that I have struggled a lot to grasp that, and I am not alone, perhaps we should explain this in the documentation.

added a section above for building cesium packages

na9da · 2025-05-01T05:15:41Z

doc/contributing/using-a-custom-version-of-cesium.md

+Then run:

 ```
 yarn install
 ```


@zoran995 does running yarn install before building the cesium repo below give an error for you, because post-install terria tries to copy the cesium workers and other assets which it cannot find?

I think before yarn install from terriamap, we need to do:

cd packages/terriajs-cesium npm install npm run release

and then:

yarn install # from terriamap

Either that, or fix post-npm-install step in terriajs/gulpfile to not fail when it cannot find the cesium assets.

added a section for installing cesium dependencies and building the package

pjonsson · 2025-05-01T15:12:11Z

I have no idea about these things so I would like to try to follow the documentation and give some comments. I'm busy tomorrow and Monday, but I will try it on Tuesday.

Sorry for not noticing the PR sooner, too many things on my plate right now.

pjonsson · 2025-05-07T10:41:28Z

doc/contributing/development-environment.md

 In the process described in [Cloning and Building](../customizing/cloning-and-building.md), the [TerriaJS package](https://www.npmjs.com/package/terriajs) is installed to the `node_modules` directory by `yarn install`. Please do not edit TerriaJS directly in the `node_modules` directory, because changes will be clobbered the next time you run `yarn install`.

-Instead, we want to clone TerriaJS from its [GitHub repo](https://github.com/TerriaJS/terriajs) and use that in our TerriaMap build. Traditionally, `npm link` is the way to do this. However, we do not recommend use of `npm link` because it frequently leads to multiple copies of some libraries being installed, which in turn leads to all sorts of frustrating build problems. Instead, we recommend [yarn](https://yarnpkg.com) and its [workspaces](https://yarnpkg.com/lang/en/docs/workspaces/) feature. `yarn` workspaces let us safely clone a git repo into the `packages` directory and wire it into any other packages that use it.
+Instead, we want to clone TerriaJS from its [GitHub repo](https://github.com/TerriaJS/terriajs) and use that in our TerriaMap build. Traditionally, `npm link` is the way to do this. However, we do not recommend use of `npm link` because it frequently leads to multiple copies of some libraries being installed, which in turn leads to all sorts of frustrating build problems. Instead, we recommend and use [yarn](https://yarnpkg.com) and its [workspaces](https://yarnpkg.com/lang/en/docs/workspaces/) feature. `yarn` workspaces let us safely clone a git repo into the `packages` directory and wire it into any other packages that use it.


I'm guessing this paragraph reflects the state of things when it was written, but of the projects I've come across from looking at TerriaMap it's quite common of them to use yarn or pnpm so I would guess the average JavaScript developer are already somewhat aware of these tools existence. Can the (old?) npm link-stuff be removed, and the first sentence in this section start with:

To [something] we use yarn workspaces which lets us safely clone a git repository into the packages directory and wire it into any other packages that use it

?

pjonsson · 2025-05-07T10:47:20Z

doc/contributing/using-a-custom-version-of-cesium.md

 yarn install
 ```

+## Making the changes in cesium


Just grabbing a random point in the diff: Cesium is a name, so it should have capital C in text/headings. (They call it CesiumJS in their own documentation, so maybe that is the name that should be used here as well.)

pjonsson · 2025-05-07T10:52:06Z

doc/contributing/using-a-custom-version-of-cesium.md


-And then run:
+```sh
+npm run release


If you add a cd terriajs-cesium on the first line in this block, it's harder for a tunnel-visioning user to miss the instruction in the text.

pjonsson · 2025-05-07T10:53:28Z

doc/contributing/using-a-custom-version-of-cesium.md

+npm run build-ts
+```
+
+Make sure that the `terriamap/package.json` workspaces list contains the following packages:


Can this paragraph be removed after your TerriaMap-PR is merged?

pjonsson · 2025-05-07T10:57:29Z

doc/contributing/using-a-custom-version-of-cesium.md


+## Making the changes in cesium
+
+Cesium uses npm so you should use it instead of yarn when installing dependencies and running commands inside cesium repo. After making the changes you will need to run:


If the text doesn't mention yarn, it reduces the risk that a user might skim the text quickly and remember the wrong words:

Cesium uses npm, so use npm when installing dependencies and running commands inside the Cesium repository. After making changes you will need to run:

pjonsson · 2025-05-07T11:00:23Z

doc/contributing/using-a-custom-version-of-cesium.md

+npm run build-ts
+```
+
+to build the code and generate Typescript types definitions required for Terria to work correctly. For more details on building the cesium and coding guidelines consult the [cesium documentation](https://github.com/CesiumGS/cesium/tree/main/Documentation)


"Typescript types definitions" => "TypeScript type definitions"
"on building the cesium" => "on building Cesium"

doc/contributing/using-a-custom-version-of-cesium.md

pjonsson · 2025-05-07T11:07:30Z

doc/contributing/using-a-custom-version-of-cesium.md


 What if you need to make changes to [Cesium](https://github.com/AnalyticalGraphicsInc/cesium) while working on TerriaJS?

+TerriaJS is using a fork of Cesium monorepo located at https://github.com/terriajs/cesium, and similar to the upstream repo it consists of two packages:


"of Cesium monorepo" => "of the Cesium monorepo"

pjonsson · 2025-05-07T11:22:37Z

doc/contributing/development-environment.md

-
 Now, you can clone any package (such as `terriajs` or `terriajs-cesium`) into the `packages` directory:

 ```


Comment for next line:

For discoverability: can a packages directory containing a README.md be added to the TerriaMap repository? The directory is already in TerriaMap's .gitignore so people will not mistakenly git add more things there. The README can just contain:

This is the directory for yarn workspaces used during development. See [link to this documentation] for how to configure workspaces for TerriaMap.

pjonsson · 2025-05-07T12:10:43Z

I know nothing about JavaScript/TypeScript development, but I tested following the development-environment.md file in this PR when using Zoran's linked TerriaMap-PR rebased on top of latest main in TerriaMap. I have too much to do right now so so I didn't test the Cesium parts since I have never had to modify Cesium.

I managed to build TerriaJS/TerriaMap, and could run yarn gulp lint for both repositories. Here are some notes I took along the way about the lines below what I can comment on in the "Files" view:

Line 23: cd .. would be easier if it was cd - instead, because the next block of commands expect the current working directory to be terriamap/, not terriamap/packages.

Line 29: grep terriajs package.json gives quite a few lines of output that the user then has to read through, it would be nice with a more specific grep pattern. Not suggesting the output needs to be a single line, but less than 5 lines seems reasonable.

Line 51: Yarn v1 doesn't de-duplicate things completely, so unless Terria is about to upgrade to Yarn v2, it would be nice to weaken that sentence so the user doesn't believe they did something wrong when there are (some) duplicates in yarn.lock.

Line 55: discusses verifying that node_modules/terriajs is a symlink. I have a unix background so I know what a symlink is, but I suspect there are capable developers who don't know that, add a command (ls -ld node_modules/terriajs) that the user can run, and describe or give an example of what the output from that command should look like (lrwxrwxrwx ... node_modules/terriajs -> ../packages/terriajs) when it is a symlink.

pjonsson · 2025-05-07T12:17:10Z

doc/contributing/using-a-custom-version-of-cesium.md

@@ -1,23 +1,59 @@
 ## Working on TerriaJS and Cesium

 What if you need to make changes to [Cesium](https://github.com/AnalyticalGraphicsInc/cesium) while working on TerriaJS?


Should this link be updated to point to the Cesium in the CesiumGS project?

update development environment guide

804e90c

zoran995 requested review from ljowen and niktayv April 30, 2025 22:50

niktayv reviewed May 1, 2025

View reviewed changes

na9da reviewed May 1, 2025

View reviewed changes

add cesium build step to guide

0fe0adb

zoran995 force-pushed the dev-env-guide branch from 803e407 to 0fe0adb Compare May 2, 2025 11:30

zoran995 requested review from na9da and niktayv May 2, 2025 11:30

pjonsson reviewed May 7, 2025

View reviewed changes

doc/contributing/using-a-custom-version-of-cesium.md Show resolved Hide resolved

pjonsson reviewed May 7, 2025

View reviewed changes

niktayv self-assigned this Jul 24, 2025


		Do _not_ commit this change.

		Now, you can clone any package (such as `terriajs` or `terriajs-cesium`) into the `packages` directory:


		## Making the changes in cesium

		Cesium uses npm so you should use it instead of yarn when installing dependencies and running commands inside cesium repo. After making the changes you will need to run:


		What if you need to make changes to [Cesium](https://github.com/AnalyticalGraphicsInc/cesium) while working on TerriaJS?

		TerriaJS is using a fork of Cesium monorepo located at https://github.com/terriajs/cesium, and similar to the upstream repo it consists of two packages:

		@@ -1,23 +1,59 @@
		## Working on TerriaJS and Cesium

		What if you need to make changes to [Cesium](https://github.com/AnalyticalGraphicsInc/cesium) while working on TerriaJS?

update development environment guide #7612

Are you sure you want to change the base?

update development environment guide #7612

Uh oh!

Conversation

zoran995 commented Apr 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does

Test me

Checklist

Uh oh!

niktayv left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pjonsson commented May 1, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pjonsson commented May 7, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

zoran995 commented Apr 26, 2025 •

edited

Loading