Lots of Bug Fixes #55

404Wolf · 2025-03-21T06:12:37Z

A bunch of bug fixes!

Empty directories get uploaded and downloaded correctly. Fixes vt pull does not create empty directories #46
- If foo/bar/buzz/iAmIgnored is ignored, then we do not create the parent directories
- But like, if foo/bar/buzz/iAmIgnored and foo/bar/iAmNotIgnored then foo/bar are created
Update val town sdk and stop using manual URI encoding!
Always add a deno.json and .vtignore.
Don't create an empty dir if vt create fails
Change ts compile options to force type in imports (deno check now fails)
Log API errors without the error code in the error message
Don't double copy when pulling (internal logic bug where we copied the same directory recursively twice)
Renaming vals inside dirs works now
Push and pull logic as per The meaning of vt pull and vt push #45
Fix Uncommitted changes lost when switching branches #49, switching branches should not touch local changes. If there aren't conflicts then local changes are carried over (aka new files).
Fix Renaming / removing a file hits a 400 error #48, no more uri encoding bugs! I think
I think fixes vt converted an http val, index.ts, to a script val #41. I'm like 95% sure. You do lose the type when you rename, but I'm not sure that's a bug. If you rename it automatically determines from filename.

Other changes:

Remove git dir, change to lib dir, since it was getting less gitty, and overall this feels simpler.
Redo all the tests for the internal lib dir. We need CLI tests! Soon!
Misc notes
I mentioned this with Sophie, but I think "" for root is unintuitive and even though it's less concise I think "/" makes more sense.
No async iterator for file listing?

maxmcd · 2025-03-22T16:03:13Z

I think the pull command still does not report whether or not it downloaded files:

$ deno run -A ../vt/vt.ts pull
√ Successfully pulled the latest changes

maxmcd · 2025-03-22T16:34:31Z

src/vt/lib/paths.ts

+    for (let i = 0; i < 5; i++) {
+      try {
+        // Try to get the file at the current version min, ensuring version
+        // doesn't go below 1
+        const versionToCheck = Math.max(1, version - i);
+        const result = await filePathToFile(
+          projectId,
+          branchId,
+          versionToCheck,
+          filePath,
+        )
+          .then((resp) => resp.type);
+        return result;
+      } catch (e) {
+        // If not found and we haven't tried all versions yet, continue to the
+        // next version
+        if (e instanceof Deno.errors.NotFound && i < 4) {
+          continue;
+        } else if (!(e instanceof Deno.errors.NotFound) || i >= 4) {
+          // If it's not a NotFound error or we've tried all versions, handle
+          // it in the outer catch
+          throw e;
+        }
+      }
+    }


what's up here? is there a situation where we're losing the file type information?

If the file was recently deleted and then recreated then it remembers its previous type when you create it again. I think that it's probably expected behavior for the type to get retained if you use local version control or your IDE's undo feature to add back the file shortly after deleting it. But also, this is because of a suspicion that in some contexts editors quickly delete and recreate the file. But I'm not entirely sure. This seems to in conjunction with other things fix the it resetting the file type issue. Right now in rare case's I've seen the file ID change while editing an HTTP val, this keeps it an HTTP val but the ID must change because of deletion/creation.

ok cool, makes sense

404Wolf · 2025-03-22T16:38:49Z

I think the pull command still does not report whether or not it downloaded files:
$ deno run -A ../vt/vt.ts pull
√ Successfully pulled the latest changes

Yeah this update is only logic fixes, I still need to add dry run and keep track of what's going on, add logging, etc

404Wolf · 2025-03-22T16:40:28Z

src/sdk.ts

+
+  while (true) {
+    const resp = await sdk.projects.files.retrieve(projectId, {
+      path,


Would love your eye on this. I think this is the weirdest (possible, not reproed besides here) API bug I've seen. Sometimes (seemingly arbitrarily) only certain batch sizes return results. The push_test fails for some batch sizes, so I got around it by trying these in order. Maybe you could try making this code more reasonable and seeing if it works for you.

ah yeah this is a tricky endpoint. I'll take a look

ok yeah, found the bug.

when using both recursive and path we're accidentally applying offset+limit to a query of all the projects files before filtering down to a subset of the files. so the offset/limit is kind of random depending on where your path is in the total structure of the project.

It is basically very difficult for us at the moment to efficiently query a subdirectory.

we'll fix this for sure, but for the time being you might want to either crawl the directory yourself (by not using recursive), or pull all the projects files and grabbing the subdir yourself.

cc: @sophiehouser

maxmcd

love the test additions. again, a pretty shallow review, but probably good to get this out to users

maxmcd · 2025-03-23T17:11:05Z

@404Wolf

You do lose the type when you rename, but I'm not sure that's a bug

Are you saying that if you rename through the api that the val type changes from (eg) http to script?

404Wolf · 2025-03-23T18:41:50Z

@404Wolf

You do lose the type when you rename, but I'm not sure that's a bug

Are you saying that if you rename through the api that the val type changes from (eg) http to script?

Yeah, if I rename foo_http.tsx to foo_email.tsx it changes the type. But if foo.tsx is a http val, and I rename foo.tsx to bar.tsx it becomes a script, since it reapplies the algorithm as a creation/deletion. I'm going to add rename detection this week, and once we are able to know something got renamed, we could preserve the type. What I think is:

Keep this PR as is
When I add rename detection, change the type detection algorithm:

Check if the file already exists in the project at the specified path.
If the file does not exist or was renamed determine its type based on its file extension:
- Files ending in .ts, .tsx, .js, or .jsx are considered "val" files.
- Check the filename for keywords like "cron", "http", or "email" to
  determine specific types:
  - If multiple keywords are found, default to previousType || "script".
  - Otherwise, return "interval" for "cron", "http" for "http", etc (so in this case renaming can change type)
  - Default to previousType || "script" if no keywords are found.
If the file does not match the val extension criteria (.ts + optional
identifier), return "file".

This means that:
"foo.tsx --> bar.tsx" preserves type
"foo.tsx --> bar.http.tsx" changes to http
"foo.http.tsx --> bar.tsx" preserves type
"foo.http.tsx --> foo.bar.email.http.tsx" preserves type

charmaine · 2025-03-23T21:14:56Z

Been using this branch for local development today and I haven't encountered these: #54, #48 again 🥳

404Wolf · 2025-03-23T21:21:03Z

Been using this branch for local development today and I haven't encountered these: #54, #48 again 🥳

Awesome! Waiting on https://github.com/val-town/val.town/pull/8503 to merge before I merge this so I can fix project listing pagination.

404Wolf added 20 commits March 20, 2025 14:22

Change internal git logic

bbcf784

Rename git to lib

7dca4d6

Don't create dir if create fails

10be639

Improve error messages

f439dbc

Make linter require 'import type'

3ae3332

Add deno.json and .vtignore automatically by default

94c815b

Remove useless error handling

479df57

Remove encodeURIComponents

147834c

Get rid of extra loadGigitnoreRules method

53216bd

Remove useless checks

0d3f2df

Repair git logic

9a90ee3

Update push test

81f7b8f

Improve test cases

4aea808

Fix jsdoc message

4eb4348

Add rename test

2c8db06

Update empty dir tests

a89786d

Fix push logic for files and dirs

049fdd6

Stronger dirty check for checkout

0939b6c

Add initial push

789f72b

Fix push test

8f9118c

404Wolf marked this pull request as ready for review March 21, 2025 07:29

404Wolf changed the title ~~Git Logic Fixes~~ Lots of Bug Fixes Mar 21, 2025

Tick version

f7705d5

404Wolf force-pushed the logic-fixes branch from be8a978 to f7705d5 Compare March 21, 2025 07:36

404Wolf requested a review from maxmcd March 21, 2025 07:37

404Wolf added 5 commits March 21, 2025 13:34

Clean up dirty check

791e85b

Improve type detection

06dcbfa

Update watch debounce

6e01f4f

Remove old install script

4ce4e68

Fix push for api bug

91ae666

maxmcd reviewed Mar 22, 2025

View reviewed changes

404Wolf commented Mar 22, 2025

View reviewed changes

Update deno lock

5199222

maxmcd approved these changes Mar 23, 2025

View reviewed changes

404Wolf added 4 commits March 24, 2025 15:52

Modify pagination

6450ddf

Tests passing

9e9f3a0

Format code

0b10165

Fix permissions

18b0a2c

404Wolf merged commit 9ab0813 into main Mar 24, 2025
1 check passed

404Wolf deleted the logic-fixes branch March 24, 2025 20:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Lots of Bug Fixes #55

Lots of Bug Fixes #55

404Wolf commented Mar 21, 2025 •

edited

Loading

maxmcd commented Mar 22, 2025

maxmcd Mar 22, 2025

404Wolf Mar 22, 2025 •

edited

Loading

maxmcd Mar 23, 2025

404Wolf commented Mar 22, 2025

404Wolf Mar 22, 2025 •

edited

Loading

maxmcd Mar 22, 2025

maxmcd Mar 22, 2025

maxmcd left a comment

maxmcd commented Mar 23, 2025 •

edited

Loading

404Wolf commented Mar 23, 2025 •

edited

Loading

charmaine commented Mar 23, 2025 •

edited

Loading

404Wolf commented Mar 23, 2025

Lots of Bug Fixes #55

Lots of Bug Fixes #55

Conversation

404Wolf commented Mar 21, 2025 • edited Loading

maxmcd commented Mar 22, 2025

maxmcd Mar 22, 2025

Choose a reason for hiding this comment

404Wolf Mar 22, 2025 • edited Loading

Choose a reason for hiding this comment

maxmcd Mar 23, 2025

Choose a reason for hiding this comment

404Wolf commented Mar 22, 2025

404Wolf Mar 22, 2025 • edited Loading

Choose a reason for hiding this comment

maxmcd Mar 22, 2025

Choose a reason for hiding this comment

maxmcd Mar 22, 2025

Choose a reason for hiding this comment

maxmcd left a comment

Choose a reason for hiding this comment

maxmcd commented Mar 23, 2025 • edited Loading

404Wolf commented Mar 23, 2025 • edited Loading

charmaine commented Mar 23, 2025 • edited Loading

404Wolf commented Mar 23, 2025

404Wolf commented Mar 21, 2025 •

edited

Loading

404Wolf Mar 22, 2025 •

edited

Loading

404Wolf Mar 22, 2025 •

edited

Loading

maxmcd commented Mar 23, 2025 •

edited

Loading

404Wolf commented Mar 23, 2025 •

edited

Loading

charmaine commented Mar 23, 2025 •

edited

Loading