[Node.js v20] Plugin Verification Meta Issue

[kavilla]

Background

To maintain optimal performance and security:
OpenSearch Dashboards is upgrading from Node.js v18 to v20

With plans to deprecate explicit support for older versions of Node.js. The current version OpenSearch Dashboards will soon EOL so therefore we must upgrade.

Node.js Version End-of-Life Dates

• Node.js 14 (LTS): Security support ended on April 30, 2023.
• Node.js 16 (LTS): Security support ended on September 11, 2023.
• Node.js 18 (LTS): Security support ends on April 30, 2025.
• Node.js 20 (LTS): Security support ends on April 30, 2026.
• Node.js 22 (LTS): Security support ends on April 30, 2027.

Recent plugin build and installation tests on the feature/node-2x branch have ✅PASSED✅.

OpenSearch Dashboards starts without issue under Node.js v20. However, plugin owners should verify their own plugins to ensure compatibility as soon as possible.

Timeline

We plan to merge this upgrade by the first week of March 2025. If no critical issues are reported by then, we will proceed with the upgrade. Please test and report any issues before this deadline.

Action Items

Recommended steps for plugin testing:

1. Checkout the feature branch with OpenSearch Dashboards root directory

git checkout feature/node-2x
# Install Node v20 (ensure you have NVM installed)
nvm install
# Install yarn
curl -o- -L https://yarnpkg.com/install.sh | bash
# Bootstrap OpenSearch Dashboards
yarn osd bootstrap --single-version=loose
# Start OpenSearch Dashboards
yarn start

2. Run automated tests within your plugin.
3. Perform manual functional testing to confirm everything works as expected.

Potential Issues

Node.js 20 Upgrade Issues and Solutions

Issue	Description	Solution
Deprecation Warnings	Node.js v20 has stricter URL parsing rules, causing warnings for invalid URL formats in package.json dependencies	Update URL format in package.json files (e.g., browserify-rsa#20)
Encoding Format	Node.js v20.5.0+ no longer accepts 'utf-8' (with hyphen) for encoding	Use 'utf8' (without hyphen) in all fs.readFileSync calls and similar functions
Stricter Type Checking	Node.js 20 enforces stricter type comparisons	Explicitly convert between types when making comparisons, especially in tests
Polyfill Conflicts	resize-observer-polyfill 1.5.1 conflicts with TypeScript 4.6+ built-in types	Use TypeScript 4.5.5 or update polyfill package
Type Definition Issues	@types/lodash v4.17.16+ syntax causes errors	Pin @types/lodash to version 4.14.165
Hapi Upgrade Challenges	Upgrading Hapi framework will be difficult	Consider removing Hapi dependency
Error Code Format Changes	Error codes now include UV prefix (UV_EACCES vs EACCES)	Update test expectations to handle new error formats
Future Upgrade Complexity	More significant challenges expected with Node.js 22 upgrade	Initial TypeScript upgrade efforts to 4.6+ had to be rolled back to 4.5.5; expect similar compatibility challenges with Node.js 22

Action Items

1. Update Tests: Modify test expectations to handle new error message formats
2. Verify Compression: Ensure brotli and gzip variants match raw data when testing file compression
3. Update Encoding References: Use 'utf8' instead of 'utf-8' in:
   • HTML Meta Charset tags
   • MIME Types in Blob objects
   • Node.js Buffer encoding specifications
4. Plan Incrementally: Recognize that Node.js 20 is just the first step, with Node.js 22 likely requiring more extensive refactoring
5. Node 20 promise handling that Node 18 was quietly tolerating. Example:

dataSources.map((ds) => getDirectQueryConnections(ds.id, http!).catch(() => []))

• 5.1. problems:
  • Silently swallowing errors with empty catch blocks
  • No timeout handling for network requests
  • Unbounded Promise.all operations assuming all promises resolve
  • Potential memory leaks from unmanaged promises
• 5.2. temp solution:
  • I implemented this temporary fix to unblock CI:

    Promise.race([
      getDirectQueryConnections(ds.id, http!),
      new Promise((_, reject) => setTimeout(() => reject('timeout'), 5000))
    ])

• 5.3: HOWEVER
  • this could be first of many similar issues.when Node 22 arrives, these patterns will become even more problematic

we can continue adding temporary solutions, but we might want to discuss a more sustainable approach before our technical debt becomes unmanageable.

Verification Checklist

Following the manifest order:

• Node.js v20 Upgrade Verification dashboards-reporting#540
• Node.js v20 Upgrade Verification opensearch-dashboards-functional-test#1749
• Node.js v20 Upgrade Verification security-dashboards-plugin#2193
• Node.js v20 Upgrade Verification dashboards-query-workbench#456
• Node.js v20 Upgrade Verification #9495
• Node.js v20 Upgrade Verification dashboards-observability#2366
• Node.js v20 Upgrade Verification dashboards-visualizations#445
• Node.js v20 Upgrade Verification dashboards-assistant#480
• Node.js v20 Upgrade Verification dashboards-flow-framework#652
• Node.js v20 Upgrade Verification dashboards-notifications#332
• Node.js v20 Upgrade Verification dashboards-maps#710
• Node.js v20 Upgrade Verification anomaly-detection-dashboards-plugin#989
• Node.js v20 Upgrade Verification ml-commons-dashboards#406
• Node.js v20 Upgrade Verification index-management-dashboards-plugin#1285
• Node.js v20 Upgrade Verification alerting-dashboards-plugin#1217
• Node.js v20 Upgrade Verification query-insights-dashboards#128

---

Thank you for your support and collaboration!

[kavilla]

Related issues:

• [RFC] Dashboards Modernization #4306
• [BUG] Compilation of 2.8.0 fails with node 18 due to lmdb-store #4260
• Remove isMaster usage once drop support for Node.js version 14 #4131
• Accommodate Different Node.js Versions (14, 16, 18) in Test Suite #4129
• set-value is not built in the artifact after node.js 18 and need to manually add it as a dependency #4064

[kavilla]

#9139

[kavilla]

• Deprecation warning interviewing with CI test result comparison.
  • the warning message you are seeing is related to Node.js v20's stricter URL parsing rules. It indicates that the URL format used for a dependency in your package.json is invalid and will cause an error in future Node.js versions.
  • browserify/browserify-rsa
    • had a incorrect URL. that was fixed in: Correct git URL browserify/browserify-rsa#20. v4?
    • potential fix: c64f73b
• fs.readFileSync issues (ENOENT issues, invalid character)
  • Both approaches are valid and will read the file as a UTF-8 encoded string. in Node.js versions prior to v20.5.0, both 'utf8' and 'utf-8' were accepted. But starting from Node.js v20.5.0, using 'utf-8' can lead to errors such as ENOENT: no such file or directory
  • therefore use 'utf8' (without a hyphen) when specifying the encoding in fs.readFileSync
  • In node 20 it's important to specify the correct encoding to ensure proper text interpretation:

const fs = require('fs');

// Using encoding as a string
const data1 = fs.readFileSync('example.txt', 'utf8');

// Using encoding within an options object
const data2 = fs.readFileSync('example.txt', { encoding: 'utf8' });

  // Verify the brotli variant matches
  expect(
    Zlib.brotliDecompressSync(
      Fs.readFileSync(Path.resolve(MOCK_REPO_DIR, `${filePath}.br`))
    ).toString('utf8')
  ).toEqual(raw);

  // Verify the gzip variant matches
  expect(
    Zlib.gunzipSync(Fs.readFileSync(Path.resolve(MOCK_REPO_DIR, `${filePath}.gz`))).toString('utf8')
  ).toEqual(raw);

title	example	should be	Action	Notes
HTML Meta Charset	<meta charset="utf-8" />	utf-8	No change	The HTML spec requires “utf-8” (with a hyphen).
MIME Type in Blob	text/csv;charset=utf-8	text/csv;charset=utf-8	No change	MIME types and HTML headers use “utf-8” (with a hyphen).
Node.js Buffer Encoding	'utf-8' (if used in Buffer.from)	'utf8'	Change to 'utf8'	Node’s Buffer APIs expect the canonical encoding name “utf8” (without the hyphen).
XML Declaration / xmlBuilder	{ encoding: 'utf-8' }	{ encoding: 'utf-8' }	No change	XML standards require “utf-8” (with a hyphen) for encoding declarations.

Node 20 now stricter about comparing types. Even in tests, in earlier versions or configurations, there might been implicit conversion or looser type checking but now we must explicitly convert one type to the other

#NOTES

resize‐observer‐polyfill 1.5.1. However, with TypeScript 4.6+ (and especially in environments where DOM types are included by default, as is the case with Node.js 20 setups), the polyfill’s own type definitions conflict with the built‐in ResizeObserver types.

new syntax in @types/lodash (v4.17.16) is causing errors even with newer TypeScript versions, the workaround is to downgrade both:

Use TypeScript 4.5.5 instead of 4.9.x or higher.
Pin @types/lodash to version 4.14.165.

if we bump opensaearch js
// Before
const response: ApiResponse = ...

// After
const response: ApiResponse = ...

This combination has been shown to avoid the TS1005 errors in similar environments.

we also need to map th apis that we are using from the client

const gzipCompressedData = Fs.readFileSync(Path.resolve(MOCK_REPO_DIR, ${filePath}.gz));
const gzipDecompressedData = Zlib.gunzipSync(new Uint8Array(gzipCompressedData)).toString('utf8');

hapi upgrade is going to be hard we should consider removing

In Node.js 20, error codes now include the UV prefix (like UV_EACCES instead of just EACCES)
The error ordering or additional error paths have changed

This is a common issue when upgrading Node.js versions - the tests are often checking for exact error message strings that change between versions.
You have a few options:

Update the test expectations: Change the tests to expect the new error message format. This is the "correct" solution but requires more work.