Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use SnakeYAML 2.3 for YAML parsing (instead of circe) #9113

Open
hubertp opened this issue Feb 20, 2024 · 5 comments
Open

Use SnakeYAML 2.3 for YAML parsing (instead of circe) #9113

hubertp opened this issue Feb 20, 2024 · 5 comments
Assignees
Labels
-compiler -language-server p-low Low priority p-medium Should be completed in the next few sprints

Comments

@hubertp
Copy link
Contributor

hubertp commented Feb 20, 2024

Follow up on #8692 and a related PR.

We use a lot of circe which has a high startup cost upfront due to class loading. Behind the scenes circe uses SnakeYAML engine for YAML parsing and we could potentially cut some startup time by using it directly with Java classes (won't work with plain Scala case classes).

@hubertp hubertp added p-medium Should be completed in the next few sprints -compiler -language-server labels Feb 20, 2024
@JaroslavTulach JaroslavTulach added the p-low Low priority label Feb 27, 2024
@JaroslavTulach
Copy link
Member

In addition to startup cost, we should also investigate independence on java.desktop JDK module. Alas SnakeYAML seems to require java.desktop:

enso$ rm -rf jdk; ~/bin/graalvm/bin/jlink --output jdk --add-modules java.net.http,jdk.unsupported,java.sql,jdk.management,jdk.jfr; ./jdk/bin/java --list-modules

# remove Python, JavaScript, regex:

enso$ rm ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/component/python-* ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/component/js-language-24.0.0.jar ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/component/icu4j-24.0.0.jar ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/component/regex-24.0.0.jar ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/component/bc*

enso$ ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --run test/Base_Tests/Exception in thread "main" java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:118)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at org.enso.runtime/org.enso.EngineRunnerBootLoader.main(EngineRunnerBootLoader.java:46)
Caused by: java.lang.NoClassDefFoundError: java/beans/IntrospectionException
        at org.yaml.snakeyaml.constructor.BaseConstructor.getPropertyUtils(BaseConstructor.java:645)
        at org.yaml.snakeyaml.constructor.Constructor.<init>(Constructor.java:60)
        at org.yaml.snakeyaml.Yaml.<init>(Yaml.java:82)
        at io.circe.yaml.Parser.$anonfun$parseSingle$1(Parser.scala:39)
        at io.circe.yaml.parser.package$.parse(package.scala:18)
        at org.enso.pkg.Config$.fromYaml(Config.scala:273)
        at scala.util.Using$.apply(Using.scala:113)
        at org.enso.pkg.PackageManager.readConfig$1(Package.scala:340)
        at org.enso.pkg.PackageManager.loadPackage(Package.scala:346)
        at org.enso.runner.Main.run(Main.java:694)

Isn't there some other, more modern library that would work with JavaBeans dependency?

@hubertp
Copy link
Contributor Author

hubertp commented May 7, 2024

Isn't there some other, more modern library that would work with JavaBeans dependency?

Looks like YamlBeans is the only alternative.

@JaroslavTulach
Copy link
Member

JaroslavTulach commented May 8, 2024

Looks like SnakeYML is ready to run on Android and avoid usage of java.beans package altogether. As such it should run without java.desktop too - it is just a matter of configuring it.

Note: Of course, such a configuration may require changes in upstream project to allow it, but maybe not. The construction stack is:

"main"
	at org.yaml.snakeyaml.introspector.PropertyUtils.<init>(PropertyUtils.java:45)
	at org.yaml.snakeyaml.constructor.BaseConstructor.getPropertyUtils(BaseConstructor.java:645)
	at org.yaml.snakeyaml.constructor.BaseConstructor.addTypeDescription(BaseConstructor.java:664)
	at org.yaml.snakeyaml.constructor.Constructor.<init>(Constructor.java:134)
	at org.yaml.snakeyaml.constructor.Constructor.<init>(Constructor.java:79)
	at org.yaml.snakeyaml.constructor.Constructor.<init>(Constructor.java:60)
	at org.yaml.snakeyaml.Yaml.<init>(Yaml.java:82)
	at io.circe.yaml.Parser.$anonfun$parseSingle$1(Parser.scala:39)
	at io.circe.yaml.Parser$$Lambda/0x00007fdb0c433680.apply
	at cats.syntax.EitherObjectOps$.catchNonFatal$extension(either.scala:391)
	at io.circe.yaml.Parser.parseSingle(Parser.scala:39)
	at io.circe.yaml.Parser.parse(Parser.scala:29)
	at io.circe.yaml.parser.package$.parse(package.scala:18)
	at org.enso.distribution.config.GlobalConfigurationManager$(GlobalConfigurationManager.scala:88)
	at org.enso.distribution.config.GlobalConfigurationManager.getConfig(GlobalConfigurationManager.scala:28)
	at org.enso.editions.updater.EditionManager$.makeEditionProvider(EditionManager.scala:55)

if we remove io.circe.yaml, then we need to invoke Yaml(BaseConstructor constructor) ourselves and provide instance of PropertyUtils that would have setBeanAccess to FIELD.

@JaroslavTulach
Copy link
Member

The pull request

comes with a workaround around current limitations of SnakeYaml. There is a PR updating the upstream project

once it is integrated we'll have to update to new version of SnakeYaml (probably 2.x). I am not sure whether io.circe can work with 2.x versions or if they require 1.x version. Should there be any incompatibilities, then the simplest solution to using latest SnakeYaml with the fix is to get rid of io.circe.

@JaroslavTulach
Copy link
Member

JaroslavTulach commented May 19, 2024

SnakeYaml fix got merged:

It will be delivered in version 2.3
https://bitbucket.org/snakeyaml/snakeyaml/wiki/Changes

Please:

  • update to SnakeYaml 2.3
  • and/or remove io.circe
  • remove the copied files from our repository

@JaroslavTulach JaroslavTulach changed the title Use SnakeYAML for YAML parsing (instead of circe) Use SnakeYAML 2.3 for YAML parsing (instead of circe) May 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
-compiler -language-server p-low Low priority p-medium Should be completed in the next few sprints
Projects
Status: 📤 Backlog
Development

No branches or pull requests

2 participants