Support custom `tileset.json` filenames for `3d-tiles-tools pipeline`

[jo-chemla]

Hi there,
When running the below pipeline, I receive error No tileset.json found in input. It seems that pipeline expects a tileset.json filename input and do not accept custom tileset filenames, probably from

3d-tiles-tools/specs/SpecHelpers.ts

Line 242 in 6560737

const tilesetJsonBuffer = await tilesetSource.getValue("tileset.json");

It would be great to support the same variations of absolute and relative naming conventions of #143
I'm running latest 3d-tiles-tools version.

npx 3d-tiles-tools pipeline -i pipeline-separateGltf.json

{
    "input": "./tileset_folder/tileset_filename_in.json",
    "output": "./tileset_folder_out/tileset_filename_out.json",
    "tilesetStages": [
        {
            "name": "ConvertB3dmToGltf",
            "contentStages": [
                {
                    "name": "b3dmToGlb",
                    "description": "Convert each B3DM content into GLB"
                },
                {
                    "name": "separateGltf",
                    "description": "Explode each GLB into GLTF + textures/geometry"
                }
            ]
        }
    ]
}

[jo-chemla]

Edit: I also tried with current main branch, and example pipeline in root Readme, it yields another error on windows,

> npx ts-node .\3d-tiles-tools\src\cli\main.ts pipeline -i pipeline-example.json


[13:37:43.327] ERROR (CLI): Error: Error: TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined
    err: {
      "type": "PipelineError",
      "message": "Error: Error: TypeError [ERR_INVALID_ARG_TYPE]: The \"path\" argument must be of type string. Received undefined",
      "stack":
          Error: Error: Error: TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined
              at TilesetStageExecutor.executeTilesetStage (\3d-tiles-tools\src\tools\pipelines\TilesetStageExecutor.ts:58:13)
              at processTicksAndRejections (node:internal/process/task_queues:105:5)
              at async PipelineExecutor.executePipeline (\3d-tiles-tools\src\tools\pipelines\PipelineExecutor.ts:87:7)
              at async ToolsMain.pipeline (\3d-tiles-tools\src\cli\ToolsMain.ts:592:5)
              at async runCommand (\3d-tiles-tools\src\cli\main.ts:560:5)
              at async run (\3d-tiles-tools\src\cli\main.ts:474:3)
              at async runChecked (\3d-tiles-tools\src\cli\main.ts:588:5)
    }

Edit: traced back this error, my tileset has not yet been upgraded so tile content has a property url instead of uri, which leads to sourceContentUri being undefined hence fail of

3d-tiles-tools/src/tools/tilesetProcessing/BasicTilesetProcessor.ts

Line 435 in 6560737

const fullSourceContentUri = Paths.join(basePath, sourceContentUri);

I'll try chaining a standard TilesetStages.TILESET_STAGE_UPGRADE with my ContentStages.CONTENT_STAGE_CONVERT_B3DM_TO_GLB and see if that works.

Note it might make sense to make some tilesetStage options configurable, somehting along these lines

3d-tiles-tools/src/tools/pipelines/TilesetStageExecutor.ts

Line 100 in 6560737

const targetVersion = "1.1";

 
- const targetVersion = "1.1";
+ const targetVersion = tilesetStage.targetVersion || "1.1"; // typescript might not like it
+ const targetVersion = tilesetStage.options.targetVersion || "1.1"; // a global options object with type any or more specific could be helpful

[jo-chemla]

Sorry, closing the issue, I was mixing things up.

The below pipeline do work:

{
    "description": "execute with npx 3d-tiles-tools pipeline -i ./pipeline.json",
    "input": "./folderpath/tileset_custom.json",
    "output": "./output/tileset_custom_out.json",
    "tilesetStages": [
        {
            "name": "upgrade",
            "description": "Upgrade tileset, also upgrade tile content from b3dm to glb",
            "options": {
                "targetVersion": 1.1 // Note this would be useful
            }
        },
        {
            "name": "_explodeGlbs",
            "description": "Convert all B3DM tiles to GLB",
            "contentStages": [
                {
                    "name": "separateGltf",
                    "description": "Explode each GLB into GLTF + textures/geometry"
                }
            ]
        }
    ]
}

[javagl]

I started and answer, and then tried to follow some of the edits, but may have to sort out some details.

First, the disclaimer: The "pipeline" functionality is experimental. I'd really like to extend that functionality, and have some ideas about what I'd like it to be. Among many, many other things, there's a reason for why I opened donmccurdy/glTF-Transform#1391 ...

And another disclaimer: The loose definition of what the "main" tileset file is has always been a source of headaches. While there is a convention to call it tileset.json, it is only a convention, and not required. (See CesiumGS/3d-tiles#184 ). While it appears to be technically "simple" to allow for other names, it often means that this name has to be passed along in call chains, and ambiguities (like the ones tested with https://github.com/CesiumGS/3d-tiles-tools/tree/main/specs/data/convert ) can arise at any point.

Back to the main points:

1. About the legacy URL field:

Some places in the code already use some utility function to graciously handle this case. But there has been a bit of back and forth: Which ~"errors" or ~"legacy elements" should be handled or ignored? One can try to be very forgiving and try to make sense of anything that is found. This may cause a false form of safety. People might start relying on tools being able to handle certain forms of invalid data. The other option is to be strict and explicit: If someone wants to process a tileset that uses the legacy URLs, then he has to do an explicit upgrade first.

But... in any case: The message "The "path" argument must be of type string. Received undefined" is all but helpful. It could/should check that case explicitly, and provide a more helpful message.

2. About the target version

Most of the "1.1 upgrade" functionality has been added after the original pipeline functionality had been "revived" from the (really old) original state of the 3D Tiles Tools. Integrating this (and other features that have been added since then) as explicit pipeline options is certainly on the radar, as part of the broader topic of "improving all that pipeline stuff".

3. About the tileset JSON file name

For the sake of conversion and trying to establish conventions, I might lean towards offering configurable filenames for inputs, but enforce tileset.json for the outputs. (The latter is already the case when the outputs are 3DTILES or 3TZ anyhow...). But I'd have to zoom into some details here, if I have/find/allocate the time for extending the pipeline functionality.

---

I think that some pof these points are worth keeping track of, so I hope that you don't mind when I repoen this.