Scripting API Revamp

[AndrewBastin]

RFC: Scripting API revamp

This is the first time we are doing an RFC (Request for Comments) here at Hoppscotch but due to the need for us needing to get this right hopefully in a single iteration itself and there are a lot of opinions on what should be added to the Hoppscotch Scripting API, hence we are introducing this discussion (which hopefully will turn into a tradition).

Summary

Hoppscotch has a strict WASM-based JavaScript Sandbox in which all of the user-defined scripts (pre-request scripts and test scripts) are executed. The main reason why such a sandbox is enforced is to actually allow for a really strong security story and to reduce the possibilities of leaking access into sensitive information over the sandbox or even privilege escalation (in the context of evaluating page scripts).

The current implementation of the sandbox features a set of APIs under the pw namespace (where pw refers to Postwoman, the previous name of Hoppscotch). The pre-request scripts feature an env API for reading and writing environment variables and the test scripts feature the testing API (test and the fluent expect API for assertions) along with the aforementioned env API to facilitate making test scripts.

The above APIs haven’t changed much over time and have stayed pretty much the same, the only recently made change is to the env API where they can now do recursive environment resolution. But over time we have gotten several feature requests to improve the capabilities of these scripts. These are:

• [feature]: how to use sha256 in Pre-request Scripts Scripts #2015
• [feature]: Would like access to btoa and atob functions from the pre-request script environment. #2140
• [feature]: Function to send request from pre-request script #2205
• [feature]: Decrypt credentials and apply in pre-request script for single session #2001
• [feature]: Support "Doing something" with the responses from web requests in tests - artifacts #1745
• [feature]: console.log in the Pre-Script #2647

Along with similar discussions in pre-request and test scripts to support different use cases and APIs. This RFC plans to propose a solution for most of the issues defined here and some revamp to some of the existing APIs. Along with the introduction of a revamped hopp namespace with much cleaner and better APIs.

Goals of the RFC

With the revamp of the sandbox, we are tackling the following technical goals.

• The script execution environment will support asynchronous code and task queues (used by setTimeout and setInterval).
• Add support for ESM module loading and allow ESM modules to be imported from online sources (like esm.sh)
• Expose many more of the expected standard JS APIs like
  • console (see below point about Console implementation)
  • btoa and atob
  • crypto
  • fetch and XMLHTTPRequest
  • setTimeout and setInterval
• A console to which console.log (and friends’) calls are rendered while trying to preserve as much functionality as offered by DevTools. (mostly object logging and string logging, we are not planning for things like coloring and CSS support within logs).
• Updates to the env API.
• A new API artifact which allows processing and transformation to be applied to responses which can then be exported after script execution.
• Ability to share data between pre-request and test scripts ephemerally with the shared API.
• Revamped code completion and linting experience by leveraging a TypeScript language server.
• Marking the deprecation of the pw namespace to the hopp namespace.

Changes in detail

TypeScript based code completion and linting

Currently, we use Tern for code completion suggestions and linting support. Tern has worked well but compared to other code analysis tools available to us right now, it lacks the intuitiveness and capabilities. Hence, we will be using a TypeScript-based intelligence service running as a web worker to run the code completion engine. We are planning to leverage the work put forward by the Monaco Editor and its implementation of the TypeScript language services and try to recreate that in our code editor of choice, CodeMirror. This does not mean the scripts will be written in TypeScript. The scripts will continue as JS files (no type annotations and TypeScript exclusive stuff allowed), but the intelligence will be powered by the TypeScript language service. You can still improve type checking by using JSDoc Types similar to how it works for a JS file with the TypeScript compiler.

Proper handling of scheduled tasks and task queues

The sandbox will now properly handle scheduled JS tasks (created by things like the setInterval and setTimeout APIs) and wait for their completion before marking the script execution is complete.

Deprecation of the pw namespace

We are going to be marking the pw namespace as deprecated and will mark warnings when those APIs are used. The pw namespace will not receive any more future features that will be coming to the scripting APIs and will stay frozen in time the way it is in the present. Hence, none of the below-proposed changes introduced by this RFC will also make it to the pw namespace.

ESM Module Loading

We are planning to introduce ESM Module loading to the scripts. Since scripts do not deal with a file structure, this doesn’t make a lot of sense on its own. But we along with ESM Module import support we will be allowing remote URLs as module identifiers. This helps us deal with a long-standing issue with scripts where external libraries (like NPM libraries) can’t be used within the sandbox. With the revamp, users will be able to import modules from sources like Skypack, esm.sh, and other similar sources which provide ESM modules.

import lodash from "https://esm.sh/lodash"

It is up to the user to be aware of the limitations of the library (what APIs the library uses etc.) and whether it will work with the sandbox environment (if it just uses the standard JS objects and APIs, it should be fine) and also worry about versioning and reliability of the actual source of the module. The sandbox will deal with fetching and loading it into the environment and we are not planning any security measures for this right now (although I would love to discuss this over here).

More Compatibility with the Standard JS objects and APIs

Many more of the standard APIs that are not implemented by the sandbox will be implemented with this RFC implementation. Notably some of the requested APIs like crypto, console, atob/ btoa, and many more. This also helps with the ESM Module Loading story as more standard library integrations mean stronger compatibility with the library ecosystem.

Sandbox Console Implementation

Funnily enough, the sandbox currently doesn’t support the console API for logging things into a console for debugging purposes. This is a design oversight that should have been tackled early itself, the community has come up with really interesting solutions to this which are undoubtedly really hacky and not exactly recommended. Hence, this iteration will bring a subset of the console API to the sandbox environment. Currently in plans are console.log, console.warn, console.error and console.debug. One thing to note though is that we are not planning to support console coloring and styling which is present in the DevTools logging. We do want to implement the rest of the APIs as well and we want to allow for other remaining APIs in the console to be implemented as well, but we are not sure if they are worth the effort right now and can be added later on. (please do let me know in the discussion below if this is needed by you).

Artifact API

The Artifact API (hopp.artifact) is a solution to the problem given in issue #1745. This allows for the scripts (this will be available to both test and pre-request scripts) to process the response in a certain way and be able to export that processed value as a file that could then be downloaded. The implementation of the API will have 4 main functions.

• hopp.artifact.create(key: string, value: string) - Creates an artifact
• hopp.artifact.update(key: string, newValue: string) - Updates an artifact
• hopp.artifact.get(key: string) - Gets an artifact
• hopp.artifact.delete(key: string) - Deletes an artifact

The API will only take strings as of right now, but if wanted (please let me know below), we can allow for binary data as well. The reason why we are not considering binary right now is considering the marshaling cost of moving the data in and out of the sandbox especially if the data is really big.

Shared API

The Shared API (hopp.shared) is also a really simple API like the Artifact API, but it is more for storing ephemeral states that should be transferred between pre-request and test scripts, previously users resorted to using the env API to pass the data along which isn’t ideal. The API surface is basically the same as the Artifact API, but it can take any value that can be serialized to JSON (due to some marshaling restrictions).

Updates to the env API

We are working towards improving the env API to allow for more capabilities for managing environments.

• Ability to specify target environment for operations - The env API will now feature hopp.env.active and hopp.env.global which will have operations within it that will allow you to query specifically within the selected and the global environments (eg. hopp.env.active.get("username") will give you the value within the selected environment and ignore the global environment.). The previous way of doing hopp.env.get and so on will still maintain the current behavior and still look at both global and selected together.
• Ability to delete environment variables - hopp.env.delete operation can be used to delete an environment variable. If multiple declarations are present, the highest precedence version of the variable will be deleted.
• Recursive Resolution by default - A while back, we introduced the recursive resolution for environment variables (e.g., I have a variable “x” with the value “hello”, I can create a variable “y” with the value “<> world”, when Hoppscotch uses the environment variable “y”, it will resolve and substitute <<x>> with its value at that point). These are currently available to scripts with pw.env.getResolve and friends. We are going to now make hopp.env.get by default do recursive resolution, to allow to opt-out of the recursive resolution, we will be introducing hopp.env.getRaw which will give out the raw value of the environment variable.

Final Notes

This is what we have in mind as a roadmap for a better scripting experience for Hoppscotch, a lot of this is to improve the capabilities meanwhile still keeping the security and predictability of the feature. This implementation will not be sudden but a slow process that spans into the next couple of months and some of the above-defined improvements may be deferred or dropped if we encounter any technical limitations or other constraints.

As I have mentioned in the beginning, we want to open up the discourse to improving the scripting experience, if you have any suggestions or certain pain points we haven’t addressed above, please join in below on the discussion.

[elad-eyal]

@AndrewBastin there should be a way to manipulate the request before sending. For example postman has pm.request.headers.add('header', 'content') and this is very powerful. I don't see it mentioned anywhere.

[AndrewBastin]

Sorry for the delay. I think you are right. We need a way to manipulate headers (or even requests in general). I am on it.

[RusEu]

any news ?

[aitchnyu]

This file mentioned in readme is actually missing:

Try out the demo src/demo.ts using:

npm run demo

[aitchnyu]

Requesting comments on improving scripting DX.

First would be having a js context that has a lifetime from start of request to end of request. We could write one script instead of having separate scripts for prerequest and testing scripts. Also users wont have to learn a new thing called shared api since they can store stuff in regular variables. This has popular precedent in browser js scripting in attaching handlers to events.

We could try to do type definitions for onrequest( Fn(HoppRequest): null ) and onresponse( Fn(HoppResponse): null ) so editor can still do code completion.

pw.env.set("var", "foo")
console.log('before request')

let x = null

onRequest( request => {
    request.header = 'foo'
    x = 'foo' // No need of shared api
    request.get.date = Date.timestamp()
    console.log('request changed', request)
})

onResponse( response => {
    expect(response.code).toBe(200) // No test block
    test("expecting stuff", () => {
        expect(response.code).toBe(200)
        expect(response.body.place).toBe(x)
    })
    
    // mean is defined in another global script
    console.log('average temp', mean(response.json.temperatures))
})

Second would be cascading scripts. We can define a global script which is evaluated before each request script. This also has strong popular precedent in browser js scripting.

//global.js
function foo() {}

//perRequest.js
foo()

[DrunkenToast]

Any news on this? Seems to have gotten stale.

[jimifredjr]

Any update on this?

[Kevin-Molina]

Bumping this - Would love to be able to use fetch in a pre-request script to set headers before requests fire off.
Thanks

[alkanna]

Yep, this is sadly still a blocker for us, we would love to be able to migrate to hoppscotch but we rely on heavy use of scripting to automate rather annoying authentication flows.