Skip to content

multistatus.md

Latest commit

 

History

History
456 lines (331 loc) · 12.8 KB

multistatus.md

File metadata and controls

456 lines (331 loc) · 12.8 KB

MultiStatus Documentation

Table of Contents:

What is MultiStatus?

When delivering a batch of events to a destination from the performBatch block, Segment’s traditional behavior has been to treat the entire batch as either a success or a failure. This works well for some use cases, but it has limitations:

  • If one event in the batch fails (e.g., due to a transient network error or invalid data), none of the events are retried or marked as failed individually.
  • This can lead to incorrect observability metrics being generated.

MultiStatus support introduces the ability for destination handlers to return per-event statuses, enabling more granular control over delivery and retry behavior.

Advantages of MultiStatus response

  • Improved Reliability: Segment can now retry only the failed events instead of the entire batch.
  • Better Debugging: Helps identify which specific events failed and why.
  • Fine-grained Control: Enables partial success reporting in destination implementations that handle many events at once (e.g., bulk APIs).
  • Optimized Retries: Only the failed events with retryable error codes are re-tried.

Internal Types and Classes

ErrorCodes

A union type that represents the possible error codes that can be returned by the API. [Reference]


ActionDestinationSuccessResponseType

A type that represents the success response from the API.

type ActionDestinationSuccessResponseType = {
  status: number
  sent: JSONLikeObject | string
  body: JSONLikeObject | string
}

It contains the following properties:

  • status: The HTTP status code of the response.
  • sent: The payload that was sent to the API.
  • body: The response body from the API.

ActionDestinationErrorResponseType

A type that represents the error response from the API.

type ActionDestinationErrorResponseType = {
  status: number
  errortype?: keyof typeof ErrorCodes
  errormessage: string
  sent?: JSONLikeObject | string
  body?: JSONLikeObject | string
}

It contains the following properties:

  • status: The HTTP status code of the response.
  • errortype: The error type. This is optional and can be inferred from the status code.
  • errormessage: The error message.
  • sent: The payload that was sent to the API. This is optional.
  • body: The response body from the API. This is optional.

ActionDestinationSuccessResponse

A class that represents a success response from the API. Example usage:

 const actionDestinationSuccessResponse = new ActionDestinationSuccessResponse(data: ActionDestinationSuccessResponseType)

ActionDestinationErrorResponse

A class that represents an error response from the API.

Example usage:

 const actionDestinationErrorResponse = new ActionDestinationErrorResponse(data: ActionDestinationErrorResponseType)

Note: The errortype is optional and can be inferred from the status code.


The MultiStatusResponse Class

Importing and Initializing

To use the MultiStatusResponse class, you need to import it from the @segment/actions-core package and initialize it as follows:

import { MultiStatusResponse } from '@segment/actions-core'

const multiStatusResponse = new MultiStatusResponse()

pushResponseObject

Pushes a response object of type success or error at the end of the internal array.

pushResponseObject(response: ActionDestinationSuccessResponse | ActionDestinationErrorResponse): void

pushSuccessResponse

Appends a success response to the end of the list. If a plain object is passed, it will be wrapped in ActionDestinationSuccessResponse.

pushSuccessResponse(response: ActionDestinationSuccessResponse | ActionDestinationSuccessResponseType): void

pushErrorResponse

Appends an error response to the end of the list. If a plain object is passed, it will be wrapped in ActionDestinationErrorResponse.

pushErrorResponse(response: ActionDestinationErrorResponse | ActionDestinationErrorResponseType): void

pushResponseObjectAtIndex

Pushes a response object of type success or error at the specified index in the internal array.

pushResponseObjectAtIndex(index: number, response: ActionDestinationSuccessResponse | ActionDestinationErrorResponse): void

setSuccessResponseAtIndex

Sets a success response at the specified index in the internal array. Useful for mapping responses to their original batch index. If a plain object is passed, it will be wrapped in ActionDestinationSuccessResponse.

setSuccessResponseAtIndex(index: number, response: ActionDestinationSuccessResponse | ActionDestinationSuccessResponseType): void

setErrorResponseAtIndex

Sets an error response at the specified index in the internal array. Useful for mapping responses to their original batch index. If a plain object is passed, it will be wrapped in ActionDestinationErrorResponse.

setErrorResponseAtIndex(index: number, response: ActionDestinationErrorResponse | ActionDestinationErrorResponseType): void

unsetResponseAtIndex

Sets a response at the specified index in the internal array to undefined. This is useful for marking a response as unset.

unsetResponseAtIndex(index: number): void

isSuccessResponseAtIndex

Returns true if the response at the specified index in the internal array is a success.

isSuccessResponseAtIndex(index: number): boolean

isErrorResponseAtIndex

Returns true if the response at the specified index in the internal array is an error.

isErrorResponseAtIndex(index: number): boolean

getResponseAtIndex

Returns the response (success or error) at the specified index.

getResponseAtIndex(index: number): ActionDestinationSuccessResponse | ActionDestinationErrorResponse

getAllResponses

Returns all the responses (success or error) in the internal array.

getAllResponses(): (ActionDestinationSuccessResponse | ActionDestinationErrorResponse)[]

length

Returns the number of responses in the internal array. This will also include indexes that are set to undefined.

length(): number

Common usage patterns

1. MultiStatus without pre-validation

Sending the payload to the destination

const response = request<APIResponseType>(`https://example-api.com/api/v1/track`, {
  method: 'post',
  json: {
    events: payloads
  }
})

Handling the response

Assuming the API returns an HTTP 207 MultiStatus response as follows:

{
  "itemsProcessed": 5,
  "success": 3,
  "error": 2,
  "errorResponses": [
    {
      "status": 400,
      "message": "Invalid zip code",
      "index": 0
    },
    {
      "status": 400,
      "message": "Invalid zip code",
      "index": 1
    }
  ]
}

We can then handle the response as follows:

// Assuming all responses to success by default
for (let i = 0; i < payloads.length; i++) {
  multiStatusResponse.setSuccessResponseAtIndex(i, {
    status: 200,
    sent: payloads[i],
    // Since the API doesn't return the response body, we can set it manually
    body: 'Processed successfully'
  })
}

// Overwriting errored indexes with error responses from the API
if (response.body.errorResponses) {
  errorResponses.forEach((errorResponse) => {
    const { status, message, index } = errorResponse

    multiStatusResponse.setErrorResponseAtIndex(index, {
      status,
      // errortype is optional, if removed, it will be inferred from the status code
      errortype: ErrorCodes.BAD_REQUEST,
      errormessage: message,
      // Note: the index returned by the API is the index of the payload in the filtered list
      sent: payloads[index],
      // In our case, the error is returned with a message
      body: message
    })
  })
}

Returning the response

Finally, we can return the MultiStatusResponse object to Segment:

return multiStatusResponse

2. MultiStatus with pre-validation

Pre-validation

Depending on the use case, we can optionally pre-validate the events before sending them to the destination. Eg: A required combination of fields are missing or invalid.

const multiStatusResponse = new MultiStatusResponse()

const filteredPayloads: JSONLikeObject[] = []

// A bitmap data structure that stores arr[new_index] = original_batch_payload_index
const validPayloadIndicesBitmap: number[] = []

// Iterate over the payloads and validate them
payloads.forEach((payload, originalBatchIndex) => {
  const { email, phone } = payload

  // Either email or phone number is required
  if (!email && !phone) {
    multiStatusResponse.setErrorResponseAtIndex(originalBatchIndex, {
      status: 400,
      errortype: 'PAYLOAD_VALIDATION_FAILED',
      errormessage: 'Either "email" or "phone" is required.'
    })

    return
  }

  // Add the payload to the filtered list
  filteredPayloads.push(payload)
  // Add the original index to the bitmap
  validPayloadIndicesBitmap.push(originalBatchIndex)
})

Sending the payload to the destination

const response = request<APIResponseType>(`https://example-api.com/api/v1/track`, {
  method: 'post',
  json: {
    events: filteredPayloads
  }
})

Handling the response

Assuming the API returns an HTTP 207 MultiStatus response as follows:

{
  "itemsProcessed": 5,
  "success": 3,
  "error": 2,
  "errorResponses": [
    {
      "status": 400,
      "message": "Invalid zip code",
      "index": 0
    },
    {
      "status": 400,
      "message": "Invalid zip code",
      "index": 1
    }
  ]
}

We can then handle the response as follows:

// Assuming all responses to success by default
for (let i = 0; i < filteredPayloads; i++) {
  const originalBatchIndex = validPayloadIndicesBitmap[i]

  multiStatusResponse.setSuccessResponseAtIndex(originalBatchIndex, {
    status: 200,
    sent: filteredPayloads[i],
    // Since the API doesn't return the response body, we can set it manually
    body: 'Processed successfully'
  })
}

// Overwriting errored indexes with error responses from the API
if (response.body.errorResponses) {
  errorResponses.forEach((errorResponse) => {
    const { status, message, index } = errorResponse
    const originalBatchIndex = validPayloadIndicesBitmap[index]

    multiStatusResponse.setErrorResponseAtIndex(originalBatchIndex, {
      status,
      // errortype is optional, if removed, it will be inferred from the status code
      errortype: ErrorCodes.BAD_REQUEST,
      errormessage: message,
      // Note: the index returned by the API is the index of the payload in the filtered list
      sent: filteredPayloads[index],
      // In our case, the error is returned with a message
      body: message
    })
  })
}

Returning the response

Finally, we can return the MultiStatusResponse object to Segment:

return multiStatusResponse