Issue compiling Opal for a backend-less web app #82

andyhaskell · 2021-12-05T17:09:06Z

Hi, I recently found Opal and Clearwater, and wanted to try compiling your hello world example for use without a backend (the examples I've seen tend to all have servers). I am a newcomer to Opal and don't know Ruby's ecosystem in-depth (I'm doing Ruby just for fun and am a Gopher on the job), and came across an error when I tried to run a Clearwater app in my browser:

Uncaught constructor {name: 'append_path', message: "undefined method `append_path' for Opal", cause: constructor, args: Array(1), backtrace: Array(8), …}
$$raise @ kernel.rb:581
$$method_missing @ kernel.rb:5
method_missing_stub @ runtime.js:1469
Opal.modules.bowser @ bowser.rb:7
Opal.load @ runtime.js:2586
Opal.require @ runtime.js:2619
$$require @ kernel.rb:639
Opal.modules.clearwater @ clearwater.rb:2
Opal.load @ runtime.js:2586
Opal.require @ runtime.js:2619
$$require @ kernel.rb:639
(anonymous) @ application.rb:3
Opal.queue @ runtime.js:2693
(anonymous) @ application.rb:1

Here's the code I have.

My Gemfile:

# frozen_string_literal: true
source "https://rubygems.org"
git_source(:github) {|repo_name| "https://github.com/#{repo_name}" }
gem 'clearwater'

My index.html:

<!DOCTYPE html>
<html>
  <head>
    <title>Compiled Clearwater app</title>
    <script type="text/javascript" src="app.js"></script>
  </head>

  <body>
    <div id="app"></div>
  </body>
</html>

The compilation command I was running:

opal -g clearwater -c app/application.rb > app.js

My Ruby code is the hello world example in your repo.

Like I said, I am pretty new to Ruby, so maybe this was just a misunderstanding on my part about gems or running Opal, but I was wondering what I should be doing to compile Opal if a web app doesn't talk to any server.

The text was updated successfully, but these errors were encountered:

jgaskins · 2021-12-05T21:49:55Z

@andyhaskell Thanks for the report. I haven't looked much into Opal for a while because every release resulted in a much larger Opal JS runtime, so I've mostly kept Clearwater on Opal 0.x. With Opal 1.2, a baseline Clearwater app (specifically, the one generated with clearwater-roda out of the box) compiled to a JS artifact that was 145KB over the wire vs a 103KB one with Opal 0.10. To be clear, I don't think it produced a usable JS app, that's just the size of the gzipped JS file after compilation with a very manual process to see what the file sizes looked like.

When they released 1.0 I did make a branch that worked with it, but I believe it was still using Sprockets v3 to compile, so that undefined method "append_path" error may still be a part of it (Opal.append_path was just wrapping Sprockets.append_path under the hood, IIRC) and v4 apparently doesn't use the same API. I don't know what compilation pipeline they're using these days.

Clearwater's APIs are pretty stable (I've replicated features like React's hooks and Suspense using only things that Clearwater already provides), so a lot of its ecosystem wouldn't need to change much to support newer Opal versions, other than to relax the Opal version restriction and inject themselves into its asset load path with whatever the new API is. I'm happy to merge PRs to any gems that are needed to get Clearwater apps running on Opal 1.x:

bowser — Ruby bindings to browser APIs, Clearwater uses it to interact with the DOM
grand_central for state management
clearwater-hot_loader for hot module reload in development (does require a server, though)

And anything else one might want to use. For a minimal application, though, clearwater and bowser should be the only ones that need to be updated, but they're also the most complicated since they depend on Opal's JS APIs.

If you're new to Clearwater and just want to get started, the quickest way (other than the Clearwater Playground) is with clearwater-roda:

$ gem i clearwater-roda
$ clearwater-roda new my_cool_app
$ cd my_cool_app
$ ./dev

Then http://localhost:9292 will serve the Clearwater app. This does run a server (intended to let you build a Ruby API with Roda for the Clearwater app to consume, and in development also provides hot module reloading for the Clearwater app so you don't have to refresh the page to see your changes), but you can use rake assets:precompile to build the single static JS artifact (in public/assets) for deployment.

andyhaskell · 2021-12-06T01:17:24Z

Ah, that hypothesis makes sense. It looks like the version my Opal is on is 1.3.2, and my Ruby version is ruby 3.0.1p64.

I'll try clearwater-roda out. If you do end up making a new branch, happy to take it for a spin. I was trying to use this for a browser extension, which is why the app was intended to be backend-less.

By the way if you're ever in Boston, let me know and we can get some beers!

jgaskins · 2021-12-06T02:30:06Z

Ah, interesting! I looked into making browser extensions with Clearwater, too! Turned out to be too many yaks to shave all at once, though. I was trying to avoid JS and the DOM APIs when I created Clearwater — joke's on me, in order to abstract them away I had to learn how they worked.

I'll have a look and see if there's a good way to accommodate the compilation side of things here. That branch to support Opal 1.0 only applied to the JS runtime, so maybe we can support the compilation side with some metaprogramming.

By the way if you're ever in Boston, let me know and we can get some beers!

Definitely!

jgaskins · 2021-12-06T03:05:32Z

Oh, interesting. Looking into this and I totally misunderstood the problem you were reporting. It's compiling the server-side Ruby code to JS and the append_path was coming from the browser.

I was able to get this working, though! It did require adding a fix to my branch for Opal 1.x, but it was a tiny change.

For your Gemfile, can you update your clearwater dependency to this?

gem 'clearwater', github: 'clearwater-rb/clearwater', branch: 'allow-opal-1.0'

And then instead of -g to compile, try -q:

$ opal -v
Opal v1.3.2
$ opal -q clearwater -c app.rb > app.js
$ open index.html

And it showed my Clearwater app in the browser! This was the Ruby code:

require 'clearwater'
require 'bowser'

class App
  include Clearwater::Component

  def render
    div "hello world! The time now is #{Time.now}!"
  end
end

app = Clearwater::Application.new(component: App.new, element: Bowser.document['#app'])
app.call

If the self.prototype part of the diff works with Opal 0.x I'll go ahead and merge that branch.

andyhaskell · 2021-12-06T03:37:21Z

Great to hear! Regarding -q, it looks like I am getting this error:

andrewhaskell:~/go/src/github.com/andyhaskell/opal-tabs$ opal -q clearwater -c app/app.rb > app.js
/Users/andrewhaskell/.rvm/gems/ruby-3.0.1/gems/opal-1.3.2/lib/opal/cli.rb:98:in `require': cannot load such file -- clearwater (LoadError)
	from /Users/andrewhaskell/.rvm/gems/ruby-3.0.1/gems/opal-1.3.2/lib/opal/cli.rb:98:in `each'
	from /Users/andrewhaskell/.rvm/gems/ruby-3.0.1/gems/opal-1.3.2/lib/opal/cli.rb:98:in `create_builder'
	from /Users/andrewhaskell/.rvm/gems/ruby-3.0.1/gems/opal-1.3.2/lib/opal/cli.rb:94:in `builder'
	from /Users/andrewhaskell/.rvm/gems/ruby-3.0.1/gems/opal-1.3.2/lib/opal/cli.rb:75:in `run'
	from /Users/andrewhaskell/.rvm/gems/ruby-3.0.1/gems/opal-1.3.2/exe/opal:26:in `<top (required)>'
	from /Users/andrewhaskell/.rvm/gems/ruby-3.0.1/bin/opal:23:in `load'
	from /Users/andrewhaskell/.rvm/gems/ruby-3.0.1/bin/opal:23:in `<main>'

The -q option in Opal is documented as

-q, --rbrequire LIBRARY          Require the library in Ruby context before compiling

and here's my bundle install output

andrewhaskell:~/go/src/github.com/andyhaskell/opal-tabs$ bundle install
Using ast 2.4.2
Using parser 3.0.3.1
Using opal 1.3.2
Using bowser 1.1.0
Using bundler 2.2.15
Using clearwater 1.1.3 from https://github.com/clearwater-rb/clearwater (at allow-opal-1.0@bfad289)
Bundle complete! 1 Gemfile dependency, 6 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.

As I mentioned I'm pretty new to the Ruby ecosystem. Is there anything special I need to do after running bundle install in order for Opal to know where to look to get this branch of Clearwater?

andyhaskell · 2021-12-06T03:44:29Z

Also, I noticed -q was introduced in Opal 1.1.0 (below is my output for 0.11.0). Would there be something else I would need to do in order to try out your branch of Clearwater in 0.x? Sorry for the noob questions

andrewhaskell:~/go/src/github.com/andyhaskell/opal-tabs$ opal _0.11.0_ -h
Usage: opal [options] -- [programfile]

    -v                               print version number, then turn on verbose mode
        --verbose                    turn on verbose mode (set $VERBOSE to true)
    -d, --debug                      turn on debug mode (set $DEBUG to true)
        --version                    Print the version
    -h, --help                       Show this message

Basic Options:

    -I, --include DIR                Append a load path (may be used more than once)
    -e, --eval SOURCE                One line of script. Several -e's allowed. Omit [programfile]
    -r, --require LIBRARY            Require the library before executing your script
    -s, --stub FILE                  Stubbed files will be compiled as empty files
    -p, --preload FILE               Preloaded files will be prepared for dynamic requires
    -g, --gem GEM_NAME               Adds the specified GEM_NAME to Opal's load path.

jgaskins · 2021-12-06T04:49:59Z

I had a whole explanation written out and then the page reloaded and it lost the whole thing. The short version of what I wrote was to try prefixing the opal command with bundle exec:

bundle exec opal -g clearwater -c app/application.rb > app.js

I forgot that the Ruby version manager I use automatically does the equivalent of bundle exec for me. The reason behind bundle exec is long, but it boils down to Bundler supporting git-based dependencies but Rubygems does not. This shouldn’t be a problem when we get all this figured out and I can release another version of Clearwater on Rubygems.

Would there be something else I would need to do in order to try out your branch of Clearwater in 0.x? Sorry for the noob questions

No apologies necessary. You ran into some legacy Ruby ecosystem stuff that confused Ruby devs for a long, long time. People have given entire conference talks on the gaps between Bundler and Rubygems. The Clearwater docs could probably be updated for the Opal CLI. I’m really glad it’s gotten easier to use and that you can compile an entire Clearwater app with it now.

I tried it with Opal 0.x using clearwater-roda and it doesn’t work, but I think that since it seems the only difference is self.prototype = x vs Opal.defn(…, x) in the JS code, we should be able to work around it with a bit of metaprogramming.

jgaskins · 2021-12-06T05:12:21Z

Okay, I think I got it working on both Opal 1.3 CLI and Opal 0.10 via Sprockets. It looks like they changed the internal name of the property for the class prototype (if they now implement Ruby classes in terms of JS classes, that would explain this), so I've set it to check both.

zw963 · 2021-12-11T15:02:07Z

Okay, I think I got it working on both Opal 1.3 CLI and Opal 0.10 via Sprockets. It looks like they changed the internal name of the property for the class prototype (if they now implement Ruby classes in terms of JS classes, that would explain this), so I've set it to check both.

I am very glad to see, clearwater support the newest version opal + sprockets for now.

anyway, diverge is not good for the community, especially, for a so mini community, i works on ruby 8~9 years, but, never see anyone use Opal, we need help together, otherwize, no one use Opal, no one use clearwater too soon.

andyhaskell · 2022-10-06T15:12:46Z

Hi @jgaskins,

I don't really understand what happened (like, I think I now have a new version of Clearwater where the demo version of your app Works On My Machine, but RubyGems-wise, I am not sure how you had updated that gem).

As I play with Clearwater some more, would you be interested in more contributions on clearwater_docs?

Thanks again for the help last winter!

Sincerely,

&y

zw963 · 2022-10-06T15:20:05Z

the author mostly work with crystal-lang, this gem probably not so active maintain i guess.

jgaskins · 2022-10-06T16:36:29Z

@andyhaskell Sorry, I got your DM the other day and then pulled this move:

So thank you for following up! 💯 Docs contributions are always welcome! I believe they all live in /assets/js/components in the clearwater_docs repo. Admittedly, it's been a long time since I've updated the docs, mostly because things that made sense to me as a maintainer might not have made as much sense to someone who didn't have intimate knowledge of the internals.

@zw963 One of these days I should find the code where I got Clearwater frontends working nicely with Crystal backends. 😄 But the gem itself is relatively solid and I haven't needed to really maintain it much. I have proofs of concept for just about everything that came out of the React ecosystem (including hooks and suspense) without ever changing Clearwater internals because things like BlackBoxNode make it so you can render literally anything, so you can wrap any imperative rendering work inside a BlackBoxNode to make it declarative. For example, the Clearwater equivalent of React's time slicing is async_render (though it's opt-in and React's time slicing is automatic), which is implemented with BlackBoxNode, as are the placeholders for MemoizedComponent.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue compiling Opal for a backend-less web app #82

Issue compiling Opal for a backend-less web app #82

andyhaskell commented Dec 5, 2021

jgaskins commented Dec 5, 2021

andyhaskell commented Dec 6, 2021

jgaskins commented Dec 6, 2021

jgaskins commented Dec 6, 2021

andyhaskell commented Dec 6, 2021

andyhaskell commented Dec 6, 2021

jgaskins commented Dec 6, 2021

jgaskins commented Dec 6, 2021

zw963 commented Dec 11, 2021

andyhaskell commented Oct 6, 2022

zw963 commented Oct 6, 2022

jgaskins commented Oct 6, 2022

Issue compiling Opal for a backend-less web app #82

Issue compiling Opal for a backend-less web app #82

Comments

andyhaskell commented Dec 5, 2021

jgaskins commented Dec 5, 2021

andyhaskell commented Dec 6, 2021

jgaskins commented Dec 6, 2021

jgaskins commented Dec 6, 2021

andyhaskell commented Dec 6, 2021

andyhaskell commented Dec 6, 2021

jgaskins commented Dec 6, 2021

jgaskins commented Dec 6, 2021

zw963 commented Dec 11, 2021

andyhaskell commented Oct 6, 2022

zw963 commented Oct 6, 2022

jgaskins commented Oct 6, 2022