Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQL Proxy Docs #11397

Closed
wants to merge 16 commits into from
Closed

SQL Proxy Docs #11397

wants to merge 16 commits into from

Conversation

dannyroosevelt
Copy link
Collaborator

WHAT

copilot:summary

copilot:poem

WHY

HOW

copilot:walkthrough

Copy link

vercel bot commented Apr 10, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
docs-v2 ✅ Ready (Inspect) Visit Preview 💬 Add feedback May 28, 2024 2:52am
2 Ignored Deployments
Name Status Preview Comments Updated (UTC)
pipedream-docs ⬜️ Ignored (Inspect) May 28, 2024 2:52am
pipedream-docs-redirect-do-not-edit ⬜️ Ignored (Inspect) Visit Preview May 28, 2024 2:52am

Copy link
Contributor

@ctrlaltdylan ctrlaltdylan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just an interim review until the meat of the tutorial is fleshed out

docs-v2/pages/databases.mdx Outdated Show resolved Hide resolved
docs-v2/pages/databases.mdx Outdated Show resolved Hide resolved
docs-v2/pages/databases.mdx Outdated Show resolved Hide resolved
However, outbound traffic from your workflows uses the same network as other AWS services in the us-east-1 region. **This means your workflows share AWS's `us-east-1` network, sending requests from standard AWS IP ranges.**

## How to Connect to Restricted Databases
When you need to connect to a database that restricts access to a specific set of IP addresses, you have a couple options:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding an intro paragraph about database security for the unaware, then go into the specific options provided by Pipedream.

Suggested change
When you need to connect to a database that restricts access to a specific set of IP addresses, you have a couple options:
It's best practice to restrict which IP addresses can access your databases. This helps prevent unauthorized access by reducing the possible IP addresses that can make a connection.
Typically this is enforced by a firewall, or a configuration within the database to only accept connections from trusted IP addresses.
Pipedream offers two options to facilitate secure connections:

# Working with Databases
Connecting to databases is a fundamental aspect of creating powerful workflows for your applications. Whether you're storing application data, querying user information, or analyzing event logs, most workflows require querying data at some step.

Every Pipedream workflow is deployed to its own virtual machine in AWS. This means your workflow runs in its own execution environment with dedicated RAM and disk space, isolated from other users' workflows.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While this is definitely true, I don't think it's focused to how to connect to a Database in a workflow, let's keep the intro focused on the tutorial, then go into the weeds about security.

Suggested change
Every Pipedream workflow is deployed to its own virtual machine in AWS. This means your workflow runs in its own execution environment with dedicated RAM and disk space, isolated from other users' workflows.


Every Pipedream workflow is deployed to its own virtual machine in AWS. This means your workflow runs in its own execution environment with dedicated RAM and disk space, isolated from other users' workflows.

However, outbound traffic from your workflows uses the same network as other AWS services in the us-east-1 region. **This means your workflows share AWS's `us-east-1` network, sending requests from standard AWS IP ranges.**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same point as above, let's focus on:

  1. What the native database support in Pipedream offers
  2. The bare minimum you need to know to get started connecting to a database
  3. Then go into implementation details that you might get tripped up on (security)
Suggested change
However, outbound traffic from your workflows uses the same network as other AWS services in the us-east-1 region. **This means your workflows share AWS's `us-east-1` network, sending requests from standard AWS IP ranges.**

* Fixing typo in GH error message (#11816)

* Fixing typo in GH error message

* Incrementing versions

* Bump tqdm from 4.66.1 to 4.66.3 in /blog/pi (#11818)

Bumps [tqdm](https://github.com/tqdm/tqdm) from 4.66.1 to 4.66.3.
- [Release notes](https://github.com/tqdm/tqdm/releases)
- [Commits](tqdm/tqdm@v4.66.1...v4.66.3)

---
updated-dependencies:
- dependency-name: tqdm
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump tqdm from 4.66.1 to 4.66.3 in /packages/component_code_gen (#11819)

Bumps [tqdm](https://github.com/tqdm/tqdm) from 4.66.1 to 4.66.3.
- [Release notes](https://github.com/tqdm/tqdm/releases)
- [Commits](tqdm/tqdm@v4.66.1...v4.66.3)

---
updated-dependencies:
- dependency-name: tqdm
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Adding HIPAA docs, removing deprecated migration plan (#11729)

* New Components - vryno (#11628)

* vryno init

* [Components] vryno #11624
Actions
 - Create Unique Lead

* pnpm update

* remove console.log

* some adjusts

* New Components - orimon (#11802)

* orimon init

* [Components] orimon #11796
Actions
- Send Message

* pnpm update

* bug fix (#11812)

* New Components - relavate (#11803)

* relavate init

* new components

* pnpm-lock.yaml

* fix typo

* Telnyx - Ability to retrieve a message (#11710)

* Added the ability to list phone numbers

* Adjusted component/action versions

* eslint

* Updated constants file

* constants.mjs

* version

* Added Retrieve Message action

* Fixed error message details

* versions

---------

Co-authored-by: michelle0927 <[email protected]>
Co-authored-by: michelle0927 <[email protected]>

* Adding newest YT intro video

* Reusing Gitlab components for Developer App (#11603)

* New app/package

* Creating actions reusing Gitlab actions

* Package update

* pnpm

* Mapping propDefinitions to new app

* Adjusting actions' propDefinitions

* Adjusting sources to use mapped propDefinitions

* Removing unused file

* Fixing import error for sources

* Updating list projects

* Gitlab version bumps

* package newline

* Description update

* Making username required

* Fix labels and parentId optional

* Fixing 'new milstone'

* Adjusting 'new review request'

* Update issue: fixing labels on request

* Replacing node-fetch with axios

* Splittin group path and group id

* Adding error treatment for source

* Bump jinja2 from 3.1.3 to 3.1.4 in /blog/pi (#11825)

Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.3 to 3.1.4.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.3...3.1.4)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* WooCommerce App Marketplace page (#11826)

* Adding troubleshooting and getting started steps

* correcting hierarchy

* example use case docs

* Migrated to latest rest api version (#11707)

* Added actions (#11808)

* New Components - tripadvisor_content_api (#11403)

* tripadvisor_content_api init

* Added actons

* Update components/tripadvisor_content_api/actions/location-reviews/location-reviews.mjs

Co-authored-by: Luan Cazarine <[email protected]>

* Update components/tripadvisor_content_api/actions/location-search/location-search.mjs

Co-authored-by: Luan Cazarine <[email protected]>

* Update components/tripadvisor_content_api/tripadvisor_content_api.app.mjs

Co-authored-by: Luan Cazarine <[email protected]>

* Update components/tripadvisor_content_api/tripadvisor_content_api.app.mjs

Co-authored-by: Luan Cazarine <[email protected]>

* Update components/tripadvisor_content_api/tripadvisor_content_api.app.mjs

Co-authored-by: Luan Cazarine <[email protected]>

* Update components/tripadvisor_content_api/tripadvisor_content_api.app.mjs

Co-authored-by: Luan Cazarine <[email protected]>

* Bump package.json version

* Remove redundant prop

* Remove redundant prop

* Revert "Remove redundant prop"

This reverts commit 80b81c4.

* Revert "Remove redundant prop"

This reverts commit bca8cf1.

* Fixing file name

---------

Co-authored-by: Luan Cazarine <[email protected]>
Co-authored-by: Leo Vu <[email protected]>
Co-authored-by: Leo Vu <[email protected]>

* Adding app scaffolding for fly_io

* Adding mongodb connecting and troubleshooting (#11835)

* Tested component with queries (#11828)

* Firestore Firebase - parse boolean values in Update Document & Create Document (#11801)

* parse boolean values

* versions

* Remove 0 and 1 case for boolean

* Fix typos

---------

Co-authored-by: Leo Vu <[email protected]>
Co-authored-by: Leo Vu <[email protected]>

* Merge branch 'issue-11579' into 9395-action-zoom-app-zoom-phones-get-call-logs-action (#11781)

* AWS readme (#11841)

* Google Drive Triggers text/usability improvements (#11652)

* Description update

* Adding descriptions for update types

* description adjustment

* Adjustments

* Text improvements

* text update

* Reverting specific drive change

* Version bumps

* Creating common file interval deduping

* Applying dedupe changes for file modification sources

* Version bumps

* Version bumps

* Moving constants to common folder

* adjusting 'changes' schema

* 'Share File' improvements

* Sharing file adjustments

* Adding advanced role options

* Version adjustment

* Updating component code to use PD Proxy for requests. (#11842)

* Updating component code to use PD Proxy for requests.

* Version bump.

* Tested components (#11820)

* GDrive Component description updates (#11844)

* SendGrid readme (#11850)

* New Components - bilflo (#11840)

* bilflo init

* [Components] bilflo #11732
Actions
 - Create Client
 - Assign Contract Job To Invoice
 - Create Contract Job

* pnpm update

* add datetime format description

* New Components - cradl_ai (#11830)

* cradl_ai init

* new components

* pnpm-lock.yaml

* Klayvio Readme (#11852)

* Zendesk API (#11853)

* 9395 action zoom app zoom phones get call logs action (#11854)

* Merge branch 'issue-11579' into 9395-action-zoom-app-zoom-phones-get-call-logs-action

* bump version

* Auto-generated documentation for apps (#11800)

* Auto-generated documentation for apps

* Overledger (`overledger`)
* Chaser (`chaser`)
* SARE (`sare`)
* Encodian (`encodian`)
* ByteNite (`bytenite`)
* Sigma (`sigma`)
* Thoughtly (`thoughtly`)
* Plate Recognizer (`platerecognizer`)
* Flipando (`flipando`)
* Relavate (`relavate`)
* TimeTonic (`timetonic`)
* Cradl AI (`cradl_ai`)
* Bilflo (`bilflo`)
* Navigatr (`navigatr`)
* ChatFly (`chatfly`)
* Perplexity (`perplexity`)
* FracTEL (`fractel`)
* y.gy (`y_gy`)
* SkyCiv (`skyciv`)
* Zip Archive API (`zip_archive_api`)
* Connectwise PSA (`connectwise_psa`)
* Botpress (`botpress`)
* Orimon (`orimon`)
* Soax (`scrapein_`)
* Twenty (`twenty`)

* Delete components/bytenite/README.md

hallucination

* Delete components/flipando/README.md

* Delete components/navigatr/README.md

* Delete components/orimon/README.md

* Delete components/thoughtly/README.md

incorrect

* Delete components/twenty/README.md

---------

Co-authored-by: Pipedream Component Development <106282845+pipedream-component-development@users.noreply.github.com>
Co-authored-by: Michael Lim <[email protected]>

* ESLint PR check improvement (#11846)

* adding eslintignore extensions

* Adding quiet to suppress non-error warnings

* New Components - timetonic (#11813)

* timetonic init

* new components

* pnpm-lock.yaml

* handle link fields

* upload files

* parse boolean values

* versions & form-data dependency

* pnpm-lock.yaml

* updates

* Slack readme

* Component republishes (#11898)

* Google Sheets version bumps

* Google Drive & Gitlab Dev App bumps

* Zoom Admin version bumps

* Google sheets import fix

* Auto-generated documentation for apps (#11897)

* Auto-generated documentation for apps

* Fly.io (`fly_io`)
* Flipando (`flipando`)
* Orimon (`orimon`)

* Delete components/orimon/README.md

* Delete components/flipando/README.md

---------

Co-authored-by: Pipedream Component Development <106282845+pipedream-component-development@users.noreply.github.com>
Co-authored-by: Michael Lim <[email protected]>

* Updating Perplexity to latest models. (#11904)

* Updating Perplexity to latest models.

* Updating constants.

* Updating package.

* Adding app scaffolding for pro_ledger

* Adding app scaffolding for wiza

* Adding app scaffolding for summit

* Adding app scaffolding for leadiq

* docs: add tap info for brew installation (#11895)

Without tapping it first you get the following error:
```
Warning: 'pipedreamhq/pd-cli/pipedream' formula is unreadable: No available formula with the name "pipedreamhq/pd-cli/pipedream".
Please tap it and then try again: brew tap pipedreamhq/pd-cli
```

* Hubspot - New Event source (#11855)

* update description

* add configuration error

* Google Sheet Sources/Triggers Improvements (#11711)

* add test events

* move constants.mjs to common folder

* update deduping to support longer ids

* update versions

* combine shared & non-shared drive sources

* new-comment source

* add new-comment test-event

* versions

* versions

* Google Sheets Actions Improvements (#11782)

* add doc links

* replace sheetName w/ worksheetId, ConfigurationErrors, updates

* use alert prop for rows description

* combine find-row & find-row-vlookup

* combine delete-row & delete-rows

* versions

* fix version

* bump package version

* parse array inputs

* combine get-values & get-values-in-range

* update action names

* headersDisplay prop

* update cells description

* update clear-row to clear-rows

* versions

* versions

* [Components] Hotmart #8023 (#11626)

* Added actions

* Fixed requested changes

* Fixed requested changes

* Adding app scaffolding for pitchlane

* Adding app scaffolding for appwrite

* Bump actions/checkout from 4.1.4 to 4.1.5 (#11928)

Bumps [actions/checkout](https://github.com/actions/checkout) from 4.1.4 to 4.1.5.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v4.1.4...v4.1.5)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump pnpm/action-setup from 3.0.0 to 4.0.0 (#11929)

Bumps [pnpm/action-setup](https://github.com/pnpm/action-setup) from 3.0.0 to 4.0.0.
- [Release notes](https://github.com/pnpm/action-setup/releases)
- [Commits](pnpm/action-setup@v3.0.0...v4.0.0)

---
updated-dependencies:
- dependency-name: pnpm/action-setup
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* New Components - twenty (#11823)

* twenty init

* [Components] twenty #11736
Sources
 - New Record (Instant)

Actions
 - Create Update Delete Record

* pnpm update

* some adjusts

* some adjusts

* Adding app scaffolding for pidj

* Adding app scaffolding for keycloak

* Adding app scaffolding for insighto_ai

* Adding app scaffolding for kodagpt

* HIPAA on Enterprise (#11941)

* Update eslintignore to ignore *.mdx (#11948)

* [ACTION] Unsplash - Search Photos, Get Photo (#11939)

* Tested components

* pnpm-lock.yaml

* Added tested components (#11845)

* Tested components (#11893)

* Adding app scaffolding for boloforms

* New Components - chatfly (#11931)

* chatfly init

* [Components] chatfly #11730
Actions
 - SendMessage

* pnpm update

* Update components/chatfly/chatfly.app.mjs

* fix description

---------

Co-authored-by: Leo Vu <[email protected]>

* New Components - thoughtly (#11933)

* thoughtly init

* [Components] thoughtly #11728
Sources
 - New Response (Instant)

Actions
 - Trigger Call
 - Create Contact

* pnpm update

* fix import

* add doc links

---------

Co-authored-by: michelle0927 <[email protected]>

* Adding app partners page to docs and fixing some typos. (#11947)

* Adding app partners page to docs and fixing some typos.

* Update docs-v2/pages/apps/app-partners.mdx

Co-authored-by: Danny Roosevelt <[email protected]>

* Update docs-v2/pages/apps/app-partners.mdx

Co-authored-by: Danny Roosevelt <[email protected]>

---------

Co-authored-by: Danny Roosevelt <[email protected]>

* Adding app scaffolding for upstash_redis

* chore: replace unmaintained jitterbit/get-changed-files@v1 (#11949)

* [Components] leadiq (#11923)

* Tested component

* pnpm-lock.yaml

* fix(types): dedupe unique and greatest strategy needs id (#11945)

* Added actions (#11951)

* New Components - fractel (#11807)

* fractel init

* [Components] fractel #11791
Actions
- Call Phone
- Send SMS/MMS

* pnpm update

* some adjusts

* New Components - platerecognizer (#11944)

* platerecognizer init

* [Components] platerecognizer #11727
Actions
 - Run Recognition

* pnpm update

* [Docs] Data stores atomic operations (#11961)

* add section for data stores atomic operations

* link to integrated apps

* chore: remove temporary continue-on-error fix for changed files (#11959)

* Tested components (#11958)

* [BUG] Postmark - New Inbound Email Received #11950 (#11952)

* [BUG] Postmark - New Inbound Email Received #11950
Sources
 - New Inbound Email Received

* update package.json

* bump version

* fix prop

* New Components - boloforms (#11955)

* boloforms init

* [Components] boloforms #11953
Sources
 - New Form Response (Instant)
 - New Template Response (Instant)
 - New Signature Completed (Instant)

Ations
 - Send Form
 - Send Template For Signature

* pnpm update

* some adjusts

* Tested components (#11921)

* Bump actions/checkout from 4.1.5 to 4.1.6 (#12002)

Bumps [actions/checkout](https://github.com/actions/checkout) from 4.1.5 to 4.1.6.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v4.1.5...v4.1.6)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* ServiceNow docs (#11899)

* ServiceNow readme instructions

* Proofreading pass

* Notion readme (#11900)

* Update README.md (#12006)

* Update README.md (#12007)

* first commit for byoa docs

* more BYOA docs

* arcade embed

* Update oauth-clients.mdx

* Update oauth-clients.mdx

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dylan J. Sather <[email protected]>
Co-authored-by: Luan Cazarine <[email protected]>
Co-authored-by: michelle0927 <[email protected]>
Co-authored-by: Marcelo Jabali <[email protected]>
Co-authored-by: michelle0927 <[email protected]>
Co-authored-by: Dylan Pierce <[email protected]>
Co-authored-by: Guilherme Falcão <[email protected]>
Co-authored-by: Jorge Cortes <[email protected]>
Co-authored-by: Lucas Caresia <[email protected]>
Co-authored-by: Leo Vu <[email protected]>
Co-authored-by: Leo Vu <[email protected]>
Co-authored-by: danhsiung <[email protected]>
Co-authored-by: Michael Lim <[email protected]>
Co-authored-by: js07 <[email protected]>
Co-authored-by: Pipedream Component Development <106282845+pipedream-component-development@users.noreply.github.com>
Co-authored-by: Joscha Feth <[email protected]>
Co-authored-by: Andrew Chuang <[email protected]>
Co-authored-by: Tod Sacerdoti <[email protected]>
@dannyroosevelt dannyroosevelt mentioned this pull request May 26, 2024
@dannyroosevelt
Copy link
Collaborator Author

this PR got nasty with merging master in, so started a clean branch and cherry-picked my changes to #12162

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants