Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(mojo): add mojo language #6441

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

austincummings
Copy link

Copy over Python queries and extend with Mojo specific syntax.

@clason clason added the new language PR adding a new (or switching to a different) parser label Apr 13, 2024
@ribru17
Copy link
Collaborator

ribru17 commented Apr 14, 2024

Instead of copying applicable queries, please use ; inherits: python at the top of the file (unless maybe they aren't perfectly compatible)

@clason
Copy link
Contributor

clason commented Apr 14, 2024

In particular, does the mojo grammar inherit from Python? If not, why not?

@lucario387
Copy link
Member

lucario387 commented Apr 14, 2024

currently mojo-python is like typescript-javascript. All valid python codes are valid mojo code, not the other way around.

Seeing how the parser is written, I have some feeling there will be some decent breakage from python :? So keeping it separated will mean less maintanence effort

@austincummings
Copy link
Author

austincummings commented Apr 16, 2024

Being new to tree-sitter I wasn't sure what the best way to approach the extend vs copy choice. I didn't find a good way to extend the existing Python parser though so I decided to just copy it and make some tweaks. If there's a better way I can go down that road.

I'm happy to make the change to use ; inherits: python in the query files if that's preferred, but I could see a situation where the Python queries change and no longer match the Mojo parser output, not sure if that's a valid concern or not though since I haven't worked much with this.

@clason
Copy link
Contributor

clason commented Apr 16, 2024

that's a valid concern or not though since I haven't worked much with this.

That is a valid concern, and something you'd have to keep an eye on as parser (and query) maintainer.

@austincummings
Copy link
Author

austincummings commented Apr 16, 2024

Yep understood, and in that case, and with the fact that Mojo's syntax is likely to change often in the near term, it seems to me the best choice is to keep the grammar and queries separate.

@clason
Copy link
Contributor

clason commented Apr 16, 2024

This risks the queries diverging over time, as changes (improvements) are made to python. Would people consider it a problem if the same Python file is highlighted differently depending on whether it's opened as a python or mojo file?

(Just exploring the landscape here, and giving ideas on what to consider when choosing one or the other.)

@clason
Copy link
Contributor

clason commented Apr 19, 2024

@austincummings please also test that TSInstallFromGrammar mojo works without errors. (tree-sitter generate --no-bindings src/grammar.json, basically)

Comment on lines +60 to +62
((decorator
"@" @attribute)
(#set! "priority" 101))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, why?
If anything, move this down so you don't need priorities for it

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is one of the parts brought over from the Python queries so I'm not exactly sure. I can see about cleaning this up though and maybe that can be added back upstream to the Python queries.

Comment on lines +424 to +427
; This one causes a crash for some reason
; (struct_definition
; superclasses: (argument_list
; (identifier) @type))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👀

You might want to run the workflows from https://github.com/tree-sitter/workflows against your scanner, in particular.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok, I'll give that a go and figure out what's going on there.

@austincummings
Copy link
Author

@clason Running TSInstallFromGrammar mojo seemed to work ok for me.

@austincummings
Copy link
Author

austincummings commented Apr 23, 2024

There are some updates I need to make on the grammar side, I'll update the lockfile when that's done. Should I make this a draft PR until I've ironed out all the issues on the grammar?

@lucario387
Copy link
Member

Whichever you think is more suitable for you and us.

@austincummings austincummings marked this pull request as draft April 23, 2024 03:54
Copy over Python queries and extend with Mojo specific syntax.
@lucario387
Copy link
Member

Is it ready for review now?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new language PR adding a new (or switching to a different) parser
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants