Skip to content

Latest commit

 

History

History
472 lines (377 loc) · 15.2 KB

agency-api-spec.md

File metadata and controls

472 lines (377 loc) · 15.2 KB

FOIA.gov Draft RESTful HTTPS API Spec

This draft is transitioning to a new home. Please refer to beta.foia.gov/developer/agency-api/ for the most up to date version.

This is a draft spec for integrating the FOIA.gov portal with existing FOIA case management systems (e.g., FOIAonline) operated by individual agencies in the federal government. This work stems from the interviews and research that led to our FOIA Portal Discovery Recommendations.

Once an agency's case management system supports this specification, it can receive FOIA requests directly from the FOIA.gov portal, rather than having the request data sent to the agency via e-mail.

To minimize agency effort, we've designed this spec so that some of the tedious bits of implementing an API can be handled by a service like api.data.gov, which provides a free API management service for federal agencies.

Receive a FOIA Request

Notes

This draft does not address:

  • versioning
  • size or rate limits
  • error message/status code related to exceeded the rate limit
  • any subsequent calls to the internal FOIA.gov API (to capture info needed to subsequently retrieve status, for example)
  • detailed security controls

Security controls

Once we have confidence in our data models and the touch points for interoperability, we will be working with the security team at DOJ to ensure the Portal meets the federal requirements for security controls. For now, we include only some preliminary ideas based on the agency feedback we’ve received.

Each agency will be responsible for implementing security controls on their own end as per their agency regulations and authority to operate. This may include configuration of a web application firewall, anti-virus scanning of file attachments, and validation of HTTP headers.

HTTPS

Agency endpoints should be restricted to HTTPS only with valid TLS certificates. The Portal will validate your certificate as part of the HTTPS request.

Authentication

To ensure that your API and case management system aren't publicly writable, we recommend restricting your API access to the FOIA.gov Portal. This can be done via a shared secret HTTP header token. You will provide this secret token to the Portal though configuration. Every request from the Portal will include this token in the HTTP header FOIA-API-SECRET, and your API should validate that it is the correct token.

Services like api.data.gov provide this authentication for you and give you additional options.

URL

There are no required parameters or format for your API URL. You may choose any pathname you wish. If your system handles requests for multiple agency components (common for decentralized agencies), we recommend using a URL structure that explicitly identifies the agency component receiving the FOIA request. Your URL should not contain any query parameters.

Recommended URL format for decentralized agencies:

/components/:id/requests/

Where id is a unique identifier for a component within your agency.

For example:

  • /components/88/requests/

But not:

  • /requests?component=88

In addition, we recommend hosting the API on a dedicated sub-domain like foia-api.agency.gov. Using this kind of pathname hierarchy allows us to add additional API endpoints for future development and features.

Method:

POST

URL parameters

Required:

id=[integer], where id is the unique identifier of the agency component that should receive the request.

Data parameters

JSON payload that contains the form fields.

Content-Type: application/json

Request fields

Field: version
Type: string
Description: The version of the API used for determining compatibility. Reserved for future use.
Required: yes
Example: "1.0.0"
Field: request_id
Type: integer
Description: A unique identifier for the request within the Portal.
Required: yes
Example: 1543
Field: agency_name
Type: string
Description: Name of the tier 1 agency.
Required: yes
Example: "Department of Justice"
Field: agency_component_name
Type: string
Description: Name of the department, bureau, or office.
Required: yes
Example: "Office of Information Policy"
Field: name_first
Type: string
Description: First name of the requester.
Required: no
Example: "George"
Field: name_last
Type: string
Description: Last name of the requester.
Required: no
Example: "Washington"
Field: address_line1
Type: string
Description: Requester’s street mailing address.
Required: no
Example: "1800 F Street"
Field: address_line2
Type: string
Description: Apartment, suite, or additional information for requester’s mailing address.
Required: no
Example: "Suite 400"
Field: address_city
Type: string
Description: City for requester’s mailing address.
Required: no
Example: "Mount Vernon"
Field: address_country
Type: string
Description: Country for requester’s mailing address.
Required: no
Example: "Mount Vernon"
Field: address_state_province
Type: string
Description: State or province for requester’s mailing address.
Required: no
Example: "Virginia"
Field: address_zip_postal_code
Type: string
Description: Zip code or postal code for requester’s mailing address.
Required: no
Example: "98273"
Field: request_description
Type: string
Description: Description of the records the requester is seeking.
Required: yes
Example: "I am seeking records pertaining to ..."
Field: fee_amount_willing
Type: string
Description: The amount in USD that a requester is willing to pay in order to cover costs related to this request.
Required: no
Example: "25.00"
Field: fee_waiver
Type: string
Description: The requester would like to request that fees associated with the request be waived.
Required: no, defaults to "no"
Example: "no"
Field: fee_waiver_explanation
Type: string
Description: The justification for the fee waiver.
Required: no
Example: "As a journalist organization, I am requesting these records on behalf of the public and intend to make these records accesible to the public."
Field: request_category
Type: string
Description: The claimed category of the requester.
Required: no
Example: "individual"
Field: expedited_processing
Type: string
Description: The requester would like this request to be processed on an expedited basis.
Required: no, defaults to "no"
Example: "no"
Field: expedited_processing_explanation
Type: string
Description: The justification for why expedited processing should be granted.
Required: no
Example: "The request should be given expedited processing because…"
Field: company_organization
Type: string
Description: Name of the organization or company on which the requester is making a request on behalf of.
Required: no
Example: "Newspaper Inc"
Field: email
Type: string
Description: Email address of the requester.
Required: no
Example: "[email protected]"
Field: phone_number
Type: string
Description: Phone number of the requester.
Required: no
Example: "+15551234567"
Field: fax_number
Type: string
Description: Fax number of the requester.
Required: no
Example: "+15551234589"
Field: attachments_supporting_documentation
Type: array
Description: Documents or attachments supporting the request provided by the requester. The file data is base64 encoded. Most programming languages include in their standard library a method to decode base64 messages.
Required: no
Example: [{"filename": "letter.pdf", "content_type": "application/pdf", "filesize": 27556, "filedata": "YSBiYXNlNjQgZW5jb2RlZCBmaWxlCg=="}]
Field: *
Type: Determined by the agency metadata file. See agency component specific form fields below.
Description: Agency component specific request form field as specified in your agency's metadata file.
Required: if applicable
Example: See below
Sample payload
{
    "version": "1.0.0",
    "request_id": 1534,
    "agency": "Department of Justice",
    "agency_component_name": "Office of Information Policy",
    "attachments_supporting_documentation": [
        {
            "content_type": "application/pdf",
            "filedata": "YSBiYXNlNjQgZW5jb2RlZCBmaWxlCg==",
            "filename": "letter.pdf",
            "filesize": 27556
        }
    ],
    "request_description": "I am seeking records pertaining to ...",
    "request_category": "individual",
    "email": "[email protected]",
    "expedited_processing": "no",
    "expedited_processing_explanation": "",
    "fax_number": "+15551234589",
    "fee_waiver": "no",
    "fee_waiver_explanation": "",
    "fee_amount_willing": "25.0",
    "company_organization": "Newspaper Inc",
    "phone_number": "+15551234567",
    "address_line1": "1800 F Street",
    "address_line2": "Suite 400",
    "address_city": "Mount Vernon",
    "address_state_province": "Virginia",
    "address_country": "United States",
    "address_zip_postal_code": "98273",
    "name_first": "George",
    "name_last": "Washington"
}

Agency component specific form fields

Your agency component might have additional fields specified in your agency metadata file. These additional fields are unique to your agency and are also captured in this request payload.

These additional fields will be defined by your agency metadata file which includes both required and optional form fields. Any fields marked required will be considered required. The default is not required. The FOIA.gov portal will ensure that required fields are present before POSTing a request to your endpoint.

Example

Consider this sample agency metadata file. A truncated version is provided below.

{
    "abbreviation": "GSA",
    "components": [
        {
            // ...
            "form_fields": [
                {
                    "help_text": "If your request relates to a GSA contract, please provide the contract number (which starts with \"GS-\")",
                    "label": "GS- Contract number",
                    "name": "contract_number"
                },
                {
                    "help_text": "(i.e. New England Region (1A) - States Served: CT, MA, ME, NH, RI, VT",
                    "label": "GSA Region",
                    "name": "region"
                },
                {
                    "enum": [
                        "Company",
                        "Individual/Self",
                        "Organization"
                    ],
                    "help_text": "Company",
                    "label": "Request Origin",
                    "name": "request_origin",
                    "regs_url": null,
                    "required": true
                }
            ]
        }
    ]
}

Therefore, the payload has these additional fields.

  • (required) request_origin
  • contract_number
  • region

So in addition to the fields in the above request payload, these fields might appear for GSA.

{
    "name_first": "George",
    "name_last": "Washington"
    // ... standard request fields ...

    // agency component specific fields appear within payload
    "contract_number": "5547",
    "region": "9",
    "request_origin": "Individual/Self"
}

Responses

Responses should be in application/json format and include an appropriate HTTP status code.

Success Response

Code: 200 OK
Content: { "id" : 33, "status_tracking_number": "doj-1234" }
Meaning: Confirm that the request was created and return an id that can uniquely identify the request in the case management system. The status tracking number (required) will be sent to the requester and used to track a request in your case management system.

Error Response

Code: 404 NOT FOUND
Content: { "code" : "A234", "message" : "agency component not found", "description": "description of the error that is specific to the case management system"}
Meaning: The target agency component specified in URI was not found (error payload includes a place for a system-specific message, to make it easier to track down problems)
Code: 500 INTERNAL SERVER ERROR
Content: { "code" : "500", "message" : "internal error", "description": "description of the error that is specific to the case management system"}
Meaning: The case management system encountered an internal error when trying to create the FOIA request (error payload includes a place for a system-specific message, to make it easier to track down problems)

Sample request

$ curl -X POST -H "Content-Type: application/json" -d @- https://foia-api.agency.gov/components/234/requests <<EOF
{
    "version": "1.0.0",
    "request_id": 1534,
    "agency": "General Services Administration",
    "agency_component_name": "General Services Administration (General)",
    "attachments_supporting_documentation": [
        {
            "content_type": "application/pdf",
            "filedata": "YSBiYXNlNjQgZW5jb2RlZCBmaWxlCg==",
            "filename": "letter.pdf",
            "filesize": 27556
        }
    ],
    "request_description": "I am seeking records pertaining to ...",
    "request_category": "individual",
    "email": "[email protected]",
    "expedited_processing": "no",
    "expedited_processing_explanation": "",
    "fax_number": "+15551234589",
    "fee_waiver": "no",
    "fee_amount_willing": "25.0",
    "fee_waiver_explanation": "I request a fee waiver...",
    "company_organization": "Newspaper Inc",
    "phone_number": "+15551234567",
    "address_line1": "1800 F Street",
    "address_line2": "Suite 400",
    "address_city": "Mount Vernon",
    "address_country": "United States",
    "address_state_province": "Virginia",
    "address_zip_postal_code": "98273",
    "name_first": "George",
    "name_last": "Washington"
}
EOF