Add a JSON-based API

[matthewbennink]

I wanted to start some discussion on a JSON-based API that could be used alongside the existing Hotwire-based frontend. There are two pieces in particular that deserve some attention. The first is how the API is versioned and second is authentication.

I asked HostedGPT for some info on API versioning, and it gave me -

API versioning is an essential part of API design. It allows you to make changes to your APIs without disrupting the service for your existing users. Here are a few common patterns for API versioning:

URL Path Versioning: This is the most straightforward and commonly used method, where the version number is included in the URL of the API. For example, https://api.example.com/v1/users. This has the benefit of being simple and clear, but it does "pollute" the URL with extra information.

Query Parameter Versioning: In this approach, the version number is provided as a query parameter in the API call. For example, https://api.example.com/users?version=1. This method keeps the URL clean, but it can cause confusion as it's less visible.

Header Versioning: This approach uses custom headers to indicate the version number. The API URL stays the same, and the version information is included in the request header. For example, X-API-Version: v1.

Media Type Versioning (content negotiation): In this method, versioning is included in the Accept header of the HTTP request. This is a more advanced tactic and can be more difficult to use.

Choose the method that fits best with your specific API and its users. Backward compatibility is always essential as you don't want to disrupt your API's existing users during an update. Always give your users adequate notice of any changes, especially breaking changes, and try to support older versions as long as feasible.

The first way seems simplest, so we might end up with routes like /api/v1/messages, /api/v1/conversations, and /api/v1/assistants. Does that seem reasonable or would we prefer to do it some other way?

As for authentication, I'm thinking OAuth2 could be used to allow access to the various resources (users, conversations, messages, etc). Certain users may have more access than others, such as a superuser role of sorts that can create other user accounts. And so if another service wanted to access the HostedGPT API, it would first need to register with HostedGPT to get a client id and secret. From there, it could authenticate with the client_credentials grant type. And if individual users wanted to use the API access, the password grant type should work fine. (I'm getting this info from https://aaronparecki.com/oauth-2-simplified/#others.)

Thoughts?

[matthewbennink]

Regarding OAuth2, the doorkeeper gem seems robust enough and could handle the server-side logic. I see a recent issue regarding Rails 7.1, but the maintainer seems active and responsive with the project.

https://doorkeeper.gitbook.io/guides/ruby-on-rails/getting-started

[krschacht]

@matthewbennink I'm all in favor of making the major endpoints respond to API requests with JSON in addition to the current view templates. I think we should wait on trying to version the API. Let's get a first version working without namespacing things. I've maintained some APIs in the past and the biggest challenge with versioning isn't the routing or namespace pollution, it's the commitment to keeping old versions alive when you're doing major changes and refactoring. It's a form of technical debt that you're never allowed to pay down, until you remove the old version of the API.

Let's not go there yet. That matters the most in a situation where you have an ecosystem of clients you're trying to encourage development of and you want to make this commitment to them that you'll not break old API endpoints. But your situation is one where we/you control both the backend serving the API and the front-end consuming the API so in the future you can co-evolve those in tandem.

With regards to auth, how about we just do simple token key issuance which can be used as bearer tokens? Anticipating this, I already laid some foundation in another branch which will create multiple types of Credentials and Authentications. I can do the work to have a logged in user easily generate an API key (which is revokable if they ever need to) and add a before_filter that recognizes this in the Bearer field. But all the controller endpoints would need to be updated to reply appropriately. I don't have the bandwidth to take that one on.

[matthewbennink]

The work to generate API keys would be great to have - I've been hesitant to work on those more primitive pieces. I'm happy to work with the controller endpoints. What's in scope? For us, we really just need the following: messages#create, messages#update, and maybe messages#index. And so I could start with the MessagesController for this first round and then we could see what's helpful/required for some of the other major endpoints.

[matthewbennink]

With regard to handling CORS headers, are you OK with using the rack-cors gem? I've got the following working well enough in an initializer. I'm expecting the API to be available to web browsers, not just other backends.

if (allowed_request_origins = ENV['ALLOWED_REQUEST_ORIGINS'].to_s.split(',')).any?
  Rails.application.configure do
    config.action_cable.allowed_request_origins = allowed_request_origins

    config.middleware.insert_before 0, Rack::Cors do
      allow do
        origins allowed_request_origins

        resource "*",
        headers: :any,
        methods: [:get, :post, :put, :patch, :delete, :options, :head],
        credentials: true,
        max_age: 86400
      end
    end
  end
end

The ENV variable would contain a value like https://my.site.com,https://my.othersite.org, allowing those two origins to make requests to the API from a web browser that enforces CORS. If the ENV variable isn't set, then nothing is configured.

[krschacht]

Sorry for my slow reply! Yes, I’m okay with this gem. It looks pretty well supported.

I think that by the end of this week I will probably, maybe, hopefully have the API key authentication added. Also, I think I have a really easy way to convert all the controller endpoints to return JSON so don’t spend too much time on that. I’ll hopefully have an update for you by the end of the week, but no promises. :) I think though

[krschacht]

Hi Matthew, I think I’m on track to merge in the foundation of this stuff tomorrow. I got about 90% of the way done today. I don’t think it’ll be everything you need but it should be close. I’ll have a way to generate (and revoke/re-generate) an API key from your account page. I got the underlying authentication refactored to support this in a nice way (and multiple other authentication schemes). I also have a proof of concept for an application_controller level way of causing all the actions to return API responses. It’ll need testing and likely tweaking for some controller actions. And you may need better API key management within settings.

[matthewbennink]

Anything helps! I've been staying up-to-date with the auth improvements and thinking some on the automatic account creation, yet another piece of the puzzle since it's effectively one account creating other accounts, and not every account should have the privilege to do that. I'm of course curious to know if you have any thoughts around that? Given it'll just be our system creating the accounts, I'm using a shared secret to authenticate our system that can be rotated on a regular basis, and the assumption here is that if this shared secret is used, then the caller can essentially do anything one could do in a Rails console, as if the systems are fully integrated.

I'm also still thinking through how we can get a stream of JSON message updates rather than a (Turbo) stream of HTML updates. Do you have any thoughts, ideas, or concerns about that? The simplest might be to simply create a new ActionCable channel to subscribe to, so thinking about exploring that. For now we've gotten really far with just parsing the HTML returned in the conversation's Turbo stream.

[krschacht]

Current.client.authentication, that's an interesting point. I'll eventually want to have a team concept in the app and I could imagine people wanting to restrict new user creation to a team admin. I guess this is the start of permissions. What's a super simple v1 permissions?

Maybe add another json column to the User table (mirroring preferences) and this column is called abilities or permissions and for now there's only one supported key which is user.abilities[:create_users] which defaults to false.

That's pretty off the cuff. I'm open to suggestions. I'd like to shy away from trying to do anything too comprehensive right now so early in the project so that's an attempt to define a simple implementation which would still be pretty flexible.

[krschacht]

On the API streaming, man that really gives me pause... It really doesn't feel right to me. I know this has been the implementation plan and my initial reaction when you told me on the phone was "sounds good!" but now that I'm in the weeds on this, it really feels like the wrong tool for the job. I can unpack the reaction a bit and share a suggestion.

It makes sense for this app to support an API for all sorts of reasons. Take even the biggest and most complex rails apps like Basecamp, Github, even Shopify. They all have APIs, it's standard, but the idea behind the API is giving programmatic access to supplement the product's natural front-end. It's not typically the case for the API to support every possible thing necessary to support the full range of functionality a replacement front-end might want to provide.

APIs are typically stateless. At the point you're wanting a socket connection and trying to replicate the most advanced functionality of the client, that's when you start asking yourself: why are we trying to do this through an API? This is starting to smell off.

Which brings me to a suggestion... I've had multiple people ask if they can embed HostedGPT into their app. I've done a phone call with two other people who are investigating this. I really wish that the app had an embeddable widget, much like the Intercom customer support widget you see in the bottom-right of lots of websites. That would be a way to have a chat experience in a different app but without trying to re-create the whole front-end of HostedGPT.

The support an embeddable widget, it would be some kind of mini front end (maybe in an iframe) which is leveraging the full power of rails with it's deep views/controller/backend integration in order to do a high fidelity experience. I spoke to one person who wanted a mini widget like this: they're building a support agent and want to let their customers interact with the hostedgpt "Assistant" they've configured. I spoke to another person who wants to embed the full-screen chat experience on their site and just style it to match (I guess that's a big iframe?).

I told them both I didn't have a solid plan in place for how to do this so at least one of them is just forking the code base or copying & pasting huge chunks of code into their own app (which I'm fine with).

But long term, I think it really makes sense for me to support:

1. An embeddable widget in both fly-out mode and full-screen embed mode.
2. The whole app as a mountable engine so you can directly incorporate the rails controllers & views into your app's codebase (similar to how Devise does).

Could (1) be a better solution for you than re-building the whole streaming front-end experience on top of an API endpoint, which I fear is going to push in the wrong direction a bit.

[krschacht]

cc: @matthewbennink (no rush on this, i just realized it may not have pinged you)

[krschacht]

@matthewbennink I have an update on API response. It's taken longer than I expected — I probably shouldn't have shared with you the optimistic-programmer-estimate :) but I have a PR tee'd up now. Why don't you take a look at this and try it out:

#407

I'm really pleased with how that code turned out! In your dev environment, put a breakpoint right here:
https://github.com/AllYourBot/hostedgpt/pull/407/files#diff-ddebc9165ae068bdf9c4346dfe8d1c4f4100afce6045e5bb02838df7b826ecb3R8

and then run that test. You can inspect the response. What you'll see is that every @instance_var that gets set within a controller action automatically gets serialized in the JSON response. So we shouldn't have to touch any controller actions. I think I managed to make them all "smart" with this single application_controller logic.

The piece I didn't get done is the user-facing piece of this. I'd love help with this because I need to turn my attention to some other items this week. After you review this 407 and we get it merged in, what's left is:

• settings/people/edit should somewhere have a button that lets the user "Generate API token"
• Clicking this button should post to a clients_controller/create . It should do Client.api.create! and after creating the client it should immediately authenticate it: Authentication.create!(client: client, credentials: Current.client.authentication)
• And if the user already has one API token then the people/edit page should show the API key and include a "Revoke Key" button which does a destroy to authentications_controller/destroy (that destroy action will need to be smart and know if it's logging the user out or if it's destroying an API credential)

[matthewbennink]

I spent a little time with it. There are a couple of instance variables we can remove if we like, specifically rendered_format and marked_for_same_origin_verification. I was pleasantly surprised to find how few there were. Any update to Rails might introduce new ones, so the main concern there is whether any secrets become exposed unexpectedly. It may be worth a single test that looks for a specific set of known keys. The pain point is when adding new instance vars and the test breaks, but any gem upgrades that introduce new instance variables will fail the test as well.

The one issue I ran into was when a controller action doesn't reach default_render. This can happen if it redirects at the end of the action, such as when creating or updating a message. To handle that in a similar manner, it may work to introduce an after_action that can detect the request for JSON and replace the redirect (i.e., remove the Location header) with the instance variables hash (as the response body), but that feels clunky.

Another not-so-great approach would be to look for redirect_to ... in the controllers and replace with redirect_to ... unless api_request?, which allows it to fall through to the default render method. It may also be possible to override the redirect_to helper to no-op for API requests, though that doesn't feel great either.

---

It does help for endpoints that are simply returning data via templates. The authentication work is also helpful for us even if we need to maintain our own branch to add some of the JSON logic, so we could consider shipping this even if it's not fully complete. Or we could spin off the authentication from the JSON responses.

Shall I commit changes to #407 for the frontend work to generate/revoke API tokens? Dumb question lol - once we've got the work merged in, I'll open up a new branch for the work. Shouldn't take long.

[krschacht]

I thought I was just about to merge in but you raise some good points. I want to think more about redirect and about leaking ivars. This solution now feels incomplete.

I’m debating if I should merge in and revise in subsequent PRs or take another pass at this one.

I think probably I’ll merge in. No one will start using it yet. I’ll consider it an undocumented experimental feature. I’ll think more about those two Qs and it’ll make it easier for us both to keep working this feature path with additional PRs