Skip to content

Conversation

chronark
Copy link
Collaborator

@chronark chronark commented Sep 5, 2025

What does this PR do?

Enhances the generate-sql Makefile target to create a more structured SQL schema file. The updated target now:

  1. Creates a clean schema.sql file with proper headers indicating it's generated code
  2. Specifies the source location in the header
  3. Uses more specific parameters for drizzle-kit generation, including schema path and output options
  4. Copies the generated SQL to the Go package directory
  5. Cleans up temporary output files after generation

Fixes # (issue)

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • Chore (refactoring code, technical debt, workflow improvements)
  • Enhancement (small improvements)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How should this be tested?

  • Run make generate-sql and verify that the schema.sql file is generated correctly
  • Verify that the generated SQL schema in go/pkg/db/schema.sql matches the expected output from drizzle-kit
  • Check that temporary files in internal/db/out are properly cleaned up after generation

Checklist

Required

  • Filled out the "How to test" section in this PR
  • Read Contributing Guide
  • Self-reviewed my own code
  • Commented on my code in hard-to-understand areas
  • Ran pnpm build
  • Ran pnpm fmt
  • Checked for warnings, there are none
  • Removed all console.logs
  • Merged the latest changes from main onto my branch with git pull origin main
  • My changes don't cause any responsiveness issues

Appreciated

  • If a UI change was made: Added a screen recording or screenshots to this PR
  • Updated the Unkey Docs if changes were necessary

Summary by CodeRabbit

  • Bug Fixes

    • More reliable duplicate detection when creating identities, keys, permissions, and roles, consistently returning clear conflict errors.
  • Chores

    • Upgraded database and migration tooling dependencies.
    • Streamlined SQL schema generation to produce a consolidated schema file with metadata.
    • Expanded Docker cleanup task to also prune system resources and volumes.
    • Updated API runtime compatibility date.

Copy link

changeset-bot bot commented Sep 5, 2025

⚠️ No Changeset found

Latest commit: c33451c

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link

vercel bot commented Sep 5, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments
Project Deployment Preview Comments Updated (UTC)
dashboard Ignored Ignored Preview Sep 8, 2025 4:56pm
engineering Ignored Ignored Preview Sep 8, 2025 4:56pm

Copy link
Contributor

coderabbitai bot commented Sep 5, 2025

📝 Walkthrough

Walkthrough

Updates Makefile SQL generation and Docker cleanup, bumps Cloudflare Worker compatibility dates, upgrades drizzle dependencies, adds a generated header to schema.sql, re-exports DrizzleQueryError from the DB package, and updates API route error handling to detect DrizzleQueryError-wrapped duplicate-entry errors.

Changes

Cohort / File(s) Summary
API routes: duplicate handling via DrizzleQueryError
apps/api/src/routes/v1_identities_createIdentity.ts, apps/api/src/routes/v1_migrations_createKey.ts, apps/api/src/routes/v1_permissions_createPermission.ts, apps/api/src/routes/v1_permissions_createRole.ts
Import DrizzleQueryError and update catch blocks to detect duplicate entries via e instanceof DrizzleQueryError && e.cause instanceof DatabaseError && e.cause.message.includes("Duplicate entry"); map to UnkeyApiError(CONFLICT). Minor tweak in keys route uses wrapper message.
Build/Tooling: Makefile
Makefile
Rewrites generate-sql to create consolidated go/pkg/db/schema.sql with header and drizzle output concatenation; extends nuke-docker to prune system and volumes.
Cloudflare Worker config
apps/api/wrangler.toml, apps/api/wrangler.custom.toml
Bumps compatibility_date to 2025-09-01.
Schema artifact
go/pkg/db/schema.sql
Adds generated file header metadata and trailing newline; no SQL semantics changed.
DB package API/export
internal/db/src/index.ts
Re-exports DrizzleQueryError from drizzle-orm/errors.
Dependency upgrades: Drizzle
internal/db/package.json, internal/encryption/package.json
Upgrades drizzle-orm to ^0.44.5; updates drizzle-kit devDep to 0.31.4 across packages.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant C as Client
  participant R as API Route (create*)
  participant DB as Database
  participant ORM as Drizzle ORM

  C->>R: POST create (identity/key/permission/role)
  R->>ORM: insert(...)
  ORM->>DB: SQL INSERT
  DB-->>ORM: Error (duplicate key)
  note over ORM: Drizzle wraps DB error<br/>as DrizzleQueryError(cause=DatabaseError)
  ORM-->>R: throw DrizzleQueryError

  alt Duplicate entry detected
    R->>R: if e instanceof DrizzleQueryError<br/>and e.cause instanceof DatabaseError<br/>and cause.message includes "Duplicate entry"
    R-->>C: 409 CONFLICT (UnkeyApiError)
  else Other errors
    R-->>C: Propagate/handle as before
  end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Suggested reviewers

  • imeyer
  • perkinsjr
  • mcstepp
  • Flo4604
  • ogzhanolguncu
✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch 09-05-feat_sync_drizzle_to_go_pkg_db_schema.sql

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Collaborator Author

chronark commented Sep 5, 2025

This stack of pull requests is managed by Graphite. Learn more about stacking.

Copy link
Contributor

github-actions bot commented Sep 5, 2025

Thank you for following the naming conventions for pull request titles! 🙏

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between b348aec and 6c20e6f.

📒 Files selected for processing (2)
  • Makefile (1 hunks)
  • go/pkg/db/schema.sql (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Test Go API Local / Test
  • GitHub Check: Test API / API Test Local
  • GitHub Check: Build / Build
🔇 Additional comments (2)
go/pkg/db/schema.sql (1)

414-415: No-op change: trailing newline — OK

The extra newline at EOF is fine and conventional. No action needed.

Makefile (1)

39-41: Use dynamic SQL filename resolution in Makefile
Hard-coded “0000_init.sql” can break if Drizzle’s naming changes. Replace the direct copy with runtime detection and fail fast if no file is found:

@@ Makefile:39-41
-	@cd internal/db && pnpm drizzle-kit generate --schema=src/schema/index.ts --dialect=mysql --out=out --name=init --breakpoints=false
-	@echo "\n" >> ./internal/db/out/0000_init.sql
-	@cp ./internal/db/out/0000_init.sql ./go/pkg/db/schema.sql
+	@cd internal/db && pnpm drizzle-kit generate --schema=src/schema/index.ts --dialect=mysql --out=out --name=init --breakpoints=false
+	@file="$$(cd internal/db/out && ls -1 *_init.sql 2>/dev/null | head -n1)"; \
+	  test -n "$$file" || { echo "ERROR: drizzle output not found"; exit 1; }; \
+	  echo "\n" >> "./internal/db/out/$$file"; \
+	  cat "./internal/db/out/$$file" >> ./go/pkg/db/schema.sql

Cannot autos-verify make generate-sql here (no make available). Manually confirm the target runs end-to-end, cleans internal/db/out, and appends the generated SQL (with header) correctly.

@Flo4604
Copy link
Contributor

Flo4604 commented Sep 8, 2025

I am getting the following error:

error: unknown option '--breakpoints=false'
make: *** [generate-sql] Error 1

Which drizzle kit version do I have to have installed on my system for it to work?
Maybe we can have the makefile ensure the correct one is installed and also the latest sqlc version?

Copy link
Collaborator Author

chronark commented Sep 8, 2025

drizzle kit is already defined in the packagejson in the internal/db package
so we should all have the same

Copy link
Contributor

Flo4604 commented Sep 8, 2025

❯ make generate-sql
error: unknown option '--breakpoints=false'
make: *** [generate-sql] Error 1
❯ cd internal/db && pnpm drizzle-kit -v
drizzle-kit: v0.22.7
drizzle-orm: v0.31.2

Mind checking what yours says?

Copy link
Collaborator Author

chronark commented Sep 8, 2025

looks like I had a newer version installed
will fix

@Flo4604 Flo4604 self-requested a review September 8, 2025 17:04
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 9

♻️ Duplicate comments (3)
Makefile (3)

33-33: Remove redundant || true with rm -f

rm -f already suppresses missing-file errors; || true can mask real failures.

-	@rm -f ./go/pkg/db/schema.sql || true
+	@rm -f ./go/pkg/db/schema.sql

37-37: Clarify Source path in header

Match the actual CLI input path (src/schema/index.ts).

-	@echo "-- Source: internal/db/src/schema" >> ./go/pkg/db/schema.sql
+	@echo "-- Source: internal/db/src/schema/index.ts" >> ./go/pkg/db/schema.sql

40-41: Avoid modifying generated artifact; append newline to final file with printf

Don’t mutate out/0000_init.sql; append a newline to the final schema.sql instead, and use printf for portability.

-	@echo "\n" >> ./internal/db/out/0000_init.sql
-	@cat ./internal/db/out/0000_init.sql >> ./go/pkg/db/schema.sql
+	@cat ./internal/db/out/0000_init.sql >> ./go/pkg/db/schema.sql
+	@printf "\n" >> ./go/pkg/db/schema.sql
📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 63845f6 and c33451c.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (11)
  • Makefile (1 hunks)
  • apps/api/src/routes/v1_identities_createIdentity.ts (2 hunks)
  • apps/api/src/routes/v1_migrations_createKey.ts (2 hunks)
  • apps/api/src/routes/v1_permissions_createPermission.ts (2 hunks)
  • apps/api/src/routes/v1_permissions_createRole.ts (2 hunks)
  • apps/api/wrangler.custom.toml (1 hunks)
  • apps/api/wrangler.toml (1 hunks)
  • go/pkg/db/schema.sql (2 hunks)
  • internal/db/package.json (1 hunks)
  • internal/db/src/index.ts (1 hunks)
  • internal/encryption/package.json (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (4)
apps/api/src/routes/v1_permissions_createPermission.ts (1)
internal/db/src/index.ts (1)
  • DrizzleQueryError (20-20)
apps/api/src/routes/v1_identities_createIdentity.ts (1)
internal/db/src/index.ts (1)
  • DrizzleQueryError (20-20)
apps/api/src/routes/v1_migrations_createKey.ts (2)
internal/db/src/index.ts (1)
  • DrizzleQueryError (20-20)
apps/api/src/pkg/errors/http.ts (1)
  • UnkeyApiError (117-124)
apps/api/src/routes/v1_permissions_createRole.ts (1)
internal/db/src/index.ts (1)
  • DrizzleQueryError (20-20)
🔇 Additional comments (7)
apps/api/wrangler.toml (1)

3-3: Validate behavior for new compatibility flags

Bumping compatibility_date to "2025-09-01" opts your Worker into a host of new runtime behaviors (e.g. global MessageChannel/MessagePort, uppercase HTTP methods, stricter fetch routing, TransformStream backpressure fix, disabled top-level await in require(), FinalizationRegistry/WeakRef, etc.). Smoke-test every environment before merging and, where needed, apply the corresponding no_* flags to retain legacy semantics.

apps/api/wrangler.custom.toml (1)

3-3: Confirm compatibility_date and nodejs_compat flags
compatibility_date = "2025-09-01" is valid; since this date is on or after 2024-09-23, v2 Node.js compatibility is auto-enabled. Ensure your compatibility_flags includes "nodejs_compat" or "nodejs_compat_v2". No additional gating required.

apps/api/src/routes/v1_permissions_createPermission.ts (1)

10-10: Import looks good

Re-export enables clean import from @unkey/db. No action needed.

apps/api/src/routes/v1_identities_createIdentity.ts (1)

8-8: Import OK

Using the re-export simplifies consumers. Keep it.

apps/api/src/routes/v1_migrations_createKey.ts (1)

10-17: Import addition is fine

No concerns on types or tree-shaking here.

apps/api/src/routes/v1_permissions_createRole.ts (1)

8-8: Good: surface DrizzleQueryError for nested DB errors

Import looks correct and aligns with the new re-export in @unkey/db.

go/pkg/db/schema.sql (1)

417-417: LGTM: trailing newline ensured

EOF newline present; avoids noisy diffs in future regenerations.

Comment on lines +558 to 566
if (
e instanceof DrizzleQueryError &&
e.cause instanceof DatabaseError &&
e.cause.message.includes("Duplicate entry")
) {
throw new UnkeyApiError({
code: "CONFLICT",
message: e.body.message,
message: e.message,
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

⚠️ Potential issue

Don’t surface raw driver message to clients; standardize error text

Leaking e.message may expose internal SQL/driver details and is inconsistent with other routes that return friendly messages. Use a stable message.

Apply:

-              throw new UnkeyApiError({
-                code: "CONFLICT",
-                message: e.message,
-              });
+              throw new UnkeyApiError({
+                code: "CONFLICT",
+                message: "Key already exists in this workspace",
+              });

Also adopt the shared isDuplicateEntry() helper to align detection across routes.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if (
e instanceof DrizzleQueryError &&
e.cause instanceof DatabaseError &&
e.cause.message.includes("Duplicate entry")
) {
throw new UnkeyApiError({
code: "CONFLICT",
message: e.body.message,
message: e.message,
});
if (
e instanceof DrizzleQueryError &&
e.cause instanceof DatabaseError &&
e.cause.message.includes("Duplicate entry")
) {
throw new UnkeyApiError({
code: "CONFLICT",
message: "Key already exists in this workspace",
});
}
🤖 Prompt for AI Agents
In apps/api/src/routes/v1_migrations_createKey.ts around lines 558-566, the
handler currently checks the raw driver error and re-throws UnkeyApiError using
e.message (which can leak SQL/driver details); replace this by importing and
using the shared isDuplicateEntry() helper to detect duplicate-key errors and
throw UnkeyApiError with a stable client-facing message (e.g. "Key already
exists") and the same "CONFLICT" code; remove reliance on e.message in the
response and keep the original error for internal logging if needed.

Comment on lines +91 to +95
if (
e instanceof DrizzleQueryError &&
e.cause instanceof DatabaseError &&
e.cause.message.includes("Duplicate entry")
) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Make duplicate-detection more robust (match message or SQL error code)

Matching only on "Duplicate entry" is brittle. Prefer checking DatabaseError code (1062 / ER_DUP_ENTRY) when available, and fall back to message includes. This keeps behavior stable across driver/version changes. (planetscale.com)

Apply:

-          if (
-            e instanceof DrizzleQueryError &&
-            e.cause instanceof DatabaseError &&
-            e.cause.message.includes("Duplicate entry")
-          ) {
+          if (
+            e instanceof DrizzleQueryError &&
+            e.cause instanceof DatabaseError &&
+            (
+              // Prefer structured code when present
+              // @ts-expect-error Planetscale DatabaseError may expose body with codes
+              e.cause?.body?.code === "1062" ||
+              // Fallback to message substring
+              e.cause.message?.includes("Duplicate entry")
+            )
+          ) {
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if (
e instanceof DrizzleQueryError &&
e.cause instanceof DatabaseError &&
e.cause.message.includes("Duplicate entry")
) {
if (
e instanceof DrizzleQueryError &&
e.cause instanceof DatabaseError &&
(
// Prefer structured code when present
// @ts-expect-error Planetscale DatabaseError may expose body with codes
e.cause?.body?.code === "1062" ||
// Fallback to message substring
e.cause.message?.includes("Duplicate entry")
)
) {
🤖 Prompt for AI Agents
In apps/api/src/routes/v1_permissions_createRole.ts around lines 91 to 95, the
duplicate-detection currently only checks e.cause.message.includes("Duplicate
entry"); update the conditional to first guard that e is a DrizzleQueryError and
e.cause is a DatabaseError, then detect duplicates by checking the error
code/errno (e.g. e.cause.code === 'ER_DUP_ENTRY' or e.cause.errno === 1062) and
fall back to message.includes("Duplicate entry") if those fields are
unavailable; ensure you access these fields safely (optional chaining) to avoid
runtime errors.

@chronark chronark merged commit 6de6149 into main Sep 9, 2025
17 of 18 checks passed
@chronark chronark deleted the 09-05-feat_sync_drizzle_to_go_pkg_db_schema.sql branch September 9, 2025 05:53
chronark added a commit that referenced this pull request Sep 11, 2025
* feat: local first ratelimits

* wip

* feat: sync drizzle to go/pkg/db/schema.sql (#3920)

* feat: sync drizzle to go/pkg/db/schema.sql

* fix: upgrade drizzle-kit to use required flag

* fix: autogeneration notice

---------

Co-authored-by: Flo <[email protected]>

* feat: script to notify users to migrate to v2 (#3924)

* feat: script to notify users to migrate to v2

* Apply suggestions from code review

* [autofix.ci] apply automated fixes

* fix: address james' feedback

* merge

* fix: exclude analytics

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* fix: prevent duplicates

* fix: make reset async

* fix: 404

* fix: rename variable for clarity

* fix: reset in parallel

* chore: revert zod version

* fix: I don't even know, this restacking is fucking with my breain

* chore: match zod schema

---------

Co-authored-by: Flo <[email protected]>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants