Open letter for improving Home Assistant's Authentication system

[christiaangoossens]

TLDR: Moved to the forum, please vote there: https://community.home-assistant.io/t/open-letter-for-improving-home-assistants-authentication-system/494223

Original post:

TLDR: Sorry for the long post, it's a complicated discussion and I wanted to highlight all parts of the current discussions. Please take the time to read all of this, and any possible comments before commenting yourself.

TLDR2: It's time for Home Assistant to embrace SSO to allow for even more secure external authentication of users.

Context

Reason for this issue

This is a meta-analysis and open letter for authentication in Home Assistant. I have been working with the OAuth2 and OpenID Connect specifications, both integrating them on the web and in Android apps for a few years now and I would like to give my input on its implementation within the project. This repository, home-assistant/architecture, seems to be the best fit, as it involves many parts throughout the system.

This letter is not a meant as a critique of the current system, just as a suggestion for improvements that might benefit the security and usability of Home Assistant in some scenario's. I love the Home Assistant developers for their dedication to this great platform and the current apps that we already use daily. ❤️

Some discussion on words used in this issue and important distinctions

If you already know these, skip this ;)

Authentication = the process of making sure the user on the other end is the user that you have in your system (confirming identity), for instance, through username & password, an external party (such as Google login) and/or through 2FA or other additional authentication methods

Authorization = the process of granting an already identified user, access rights, for instance, internally within Home Assistant to access the APIs. It can also be used the other way around, where Home Assistant requests access to data for Google Assistant, through using Googles Authorization APIs.

OAuth2 = an Authorization specification which makes it possible to give external parties (other apps) access to data from your app (in this case Home Assistant), or is implemented by other parties (such as Google) to give Home Assistant access

OpenID Connect (OIDC) = an Authentication extension to the OAuth2 specification which also gives external parties access to the "authentication"/"identity" part of the login, such that they can use

Why did I include this?

Even the Home Assistant documentation on the access token API's, uses these terms interchangeably and sometimes in the wrong context. For instance, the first sentence "This page will describe the steps required for your application to authorize against and integrate with Home Assistant instances." is correct, but the title "Authentication API" is not. However the page on "Authentication Providers" is correctly named and provides the correct definition "Authentication providers confirm the identity of users.".

In short, and throughout the open-source space, these terms are not easy. Especially since this is not the part of apps that most developers deal with a lot. We cannot blame developers or documentation writers for not being experts in authentication/authorization.

Historical context

Back to Home Assistant: in release 0.77 the "new Authentication System" was activated. From this update onwards, both the web version and the mobile apps use OAuth2 to authorize the app or the web interface to access the Home Assistant API.

For authentication, a few providers were provided within Home Assistant:

• Home Assistant "Auth" Provider: username and password + TOTP
• Trusted Networks: authentication based on origin IP
• Command line: providing the entered username and password to a command line script (only possible in Home Assistant Core).
• Legacy API password: deprecated old way of authenticating

Identified current problems

While this new Authentication system is a big step forward and enables nice features, such as building third-party external apps against the Home Assistant API by use of the OAuth2 authorization method to obtain a properly-scoped access token, it has a few issues. I will deal with them in their own subsections.

Additional providers within Home Assistant

There have been some PR's to include useful additional authentication providers within Home Assistant, such as:

• LDAP/Active Directory authentication provider core#37645: Added LDAP as a provider within Home Assistant, without the use of external scripts, such that it would also work within Home Assistant OS. However it got declined: LDAP/Active Directory authentication provider core#37645 (comment)
• Support openid connect like authentication providers core#32926: Added OpenID Connect to authenticate the user right in Home Assistant. This is the correct way to do OpenID Connect for apps, which only uses it for authentication and keeps your internal authorization system. This got declined due to: Support openid connect like authentication providers core#32926 (comment)

I remember also seeing some comments that maintaining these providers within the Home Assistant code would lead to possible security issues as it put a large burden on the maintainers to know the specifications, all their flaws and keep this secure.

Additional providers as plugins

With the new authentication system, it should have been easy to integrate new providers, such as LDAP and OIDC as plugins, possibly through HACS. I only know of one successful authentication plugin: https://github.com/BeryJu/hass-auth-header by @BeryJu, the creator of Authentik, a project I have also followed from its Passbook early days. While the plugin itself works fine, if a header is passed, it updates the authentication screen to include a "Header auth" option which just has a button to detect the header and continue, the setup around it has problems.

Many users running Homelabs would like to tie Home Assistant into their existing SSO system, but as authentication proxies (using https://github.com/BeryJu/hass-auth-header) are the only option, proxy auth for apps using OAuth2 and/or using service workers is not the right idea.

Currently, you can find many issues and Reddit threads, such as home-assistant/android#1438, authelia/authelia#1842 (comment) and goauthentik/authentik#884 (comment), which include ideas to disable the service worker, exclude certain paths from the proxy auth or only enable proxy auth for /auth/*.

For apps implementing OAuth2 as their authorization layer, such as Home Assistant, one should have access to the login page and the static assets, as those are vital to allowing third-parties access to the API. Other applications should always be able to make an OAuth2 authorization request, without being stopped by the proxy auth first. They should however be stopped at the authentication phase right after, but as there are no OpenID Connect or LDAP plugins that integrate to that stage (like the PR's above), it is not possible to go that route right now for integrating Home Assistant with your current SSO, be it at home, or for a company.

Android Mobile app uses Webview instead of Custom Tabs

Currently, the Android Companion app uses Webview (https://github.com/home-assistant/android/blob/fa001ffc29fc1ff8bfd2f47f0628666edbe3f386/app/src/main/java/io/homeassistant/companion/android/onboarding/authentication/AuthenticationFragment.kt) for its Authentication Fragment.

The Android Webview implementation lags behind in its support of modern web standards, does not share a context with the main browser and is therefore not recommended for use with OAuth2 in the authorization-community. This also lead Google to disallow embedded webviews for OAuth2 (read this article to find out more why it's a bad idea).

Instead, you should use Custom Tabs. These tabs are part of the main browser (often Chrome) and share a session with this browser. If you are already logged in with the main browser, you will be logged into the Custom Tab as well. Additionally, as the tabs use modern Chrome, they have access to all modern web standards.

Doing this would also prevent issues like: goauthentik/authentik#3304 and BeryJu/hass-auth-header#124, where the webview does not render Authentik correctly and does not support FIDO2 Security Keys, which totally disables Authentik for some users.

Proposal

Having looked at some current identified issues, I would like to propose the following:

Home Assistant should welcome external authentication

The Home Assistant project is about connecting with many home devices and integrating as many different devices to one platform. It is not about creating the strongest authentication method. We should leave strong auth, or the option to use stronger auth, to other platforms, such as Authentik and Authelia at home, or Keycloak, Azure AD and Okta in the enterprise setting.

External SSO authentication providers allow for many benefits that will likely not be included within Home Assistant's native authentication provider, such as:

• Support for WebAuthN, FIDO2 security keys and other extra 2FA options, such as push-based mobile authentication
• Detection of strange logins, such as based on changing locations of user logins (for instance: you always login from Amsterdam, but suddenly you are in the Maldives) and blocking of such logins dynamically
• Requiring different or additional 2nd factors within or outside of a network.
• Native SSO, where you never enter any password within a trusted context

In conclusion, external SSO providers will always be better at authentication than your own solution. Embrace them, not extinguish.

Mobile Apps

The mobile apps should not assume anything about the authentication/authorization process, except that it is OAuth2, happens within a webbrowser and uses the Authorization Grant flow with PKCE (RFC 7636). This is the recommended OAuth2 flow for native clients as per RFC8252.

They should open this authorization endpoint URL (https://YOUR_INSTALL_URL:8123/auth/authorize?......) within a Android Custom Tab on Android, the native browser or the SFSafariViewController on iOS and should expose a return URL (homeassistant://auth-callback) to call back with the authorization code within the OAuth2 process. You should then use PKCE to ensure that the app returned to did start the request and check the state.

This:

• Follows all security recommendations within the authorization community, as well as RFC8252
• Does not assume anything about the actual authentication provider used, which might include Google Social Login, Authentik, Authelia, Keycloak etc, and because it uses a modern browser prevents rendering conflicts and enables FIDO2 keys (such as Yubikeys) to be used as well.
• Shows the user at all times which path they are entering their credentials into, while webviews hide this, such that they are less likely to be phished.
• Shows users that their credentials are entered using HTTPS (and allows for checking the certificate).

Authentication Providers

Home Assistant should encourage the creation of LDAP and OIDC plugins that add authentication providers that nicely integrate within the current flow, instead of the "brute force proxy" approach which destroys many of the benefits of the OAuth2 and service worker systems within Home Assistant.

home-assistant/core#37645 and home-assistant/core#32926 should either be revived as core parts of Home Assistant, or - and I think most likely - as well-maintained separate plugins that are easy to install through HACS.

Again, embrace external authentication, not extinguish to improve security for all users.

Consequences

I want to get back to the specific reason why the PRs got declined by @balloob:

However, there is one big question mark… how do we want to deal with users that are no longer allowed to log in?

Credentials are only validated during login, which results in the creation of a refresh/access token pair. Once granted, all interactions are done with these tokens and no one will check back with the OpenID provider if the user is still valid.

Example how other platforms do this: if you log in to GitHub using Okta, you will need to re-login every 24ish hours (I think). That way they verify that you are still allowed to log in. If I remember correctly, Okta is actually smart enough in that if you are already logged in to it, it's just a few redirects and you're back at GitHub, no need to actually enter user/pass again.

When we added the command_line auth provider, we decided that it would be up to the implementer to disable users (custom integration) given that it was already a pretty manual process to wire it all together. With OpenID it is getting very easy to link external authentication. People will expect that users are blocked from logging in by changing the auth provider configuration or the user in the source system. However, they won't realize that it won't log out users that are already logged in.

You are right and it's very good that you realized this shortfall! OpenID Connect implementations in fact often have two common shortfalls (I will give the solutions later on):

1. Suppose that you add OIDC to an existing Home Assistant install, where all your users have 2FA, users with the same username or email from all providers are considered the same and you add a OIDC authentication provider that does not require 2FA (possibly bad config), you can circumvent the original 2FA! Well, this actually happend to CloudFlare, when they added "Signin with Apple". If you had an insecure Apple account matching the same email of a 2FA secured CloudFlare user, you could just login to their account using Apple, no questions asked.

2. User logout at the IdP (OIDC provider) are not propagated to the consuming app, especially if you are using refresh_tokens or long-lived access tokens. As @balloob correctly identifies, the check (often) only happens on the authentication stage!

Solutions:

1st issue

When implementing OIDC/SAML in your app you do the following:

• If a user logs in through OIDC with a new username that you do not have yet, you give them no rights (or make new user rights a configurable field) and create a new account for them.
• If a user logs in through OIDC with a username that is also a local account, you should require them to login with the local account (including 2FA) once to "link their accounts".

Alternatively, you could always prefix usernames with the OIDC provider name (or similar) to prevent username "collisions" entirely and always create a unique account. This however disallows for migrating from local accounts to OIDC accounts, so I would recommend the approach above.

2nd issue

First off, sensitive apps should always use short expiry access tokens, for instance, 1 hour. Native/mobile apps can then use the refresh token to get a new token, and web apps can use the prompt=none silent refresh flow to try and get a token within an iframe.

Additionally, when changing a password or disabling a user, you should revoke all of their JWT's by keeping a list of revoked JTI's (RFC 7519) and checking these on every API request. For distributed apps, this could mean keeping this list in a shared cache, such as Redis, and checking it with all API microservices.

However, we have not solved the issue of users getting disabled at the SSO provider, just for local disabling.

We actually have three solutions here:

• There are specifications for logout with OpenID Connect: https://openid.net/specs/openid-connect-frontchannel-1_0.html and https://openid.net/specs/openid-connect-backchannel-1_0.html. The OpenID Connect Back-Channel logout specification solves this issue perfectly. It adds a sid field within the original id_token that can be saved by the RP (Home Assistant). Then, Home Assistant exposes an API route that allows for the OIDC provider to call it whenever the token should be revoked, including the sid within a signed JWT. After this, the exact same happens as with local disabling, we add the JTI of all issued tokens for that user onto the blacklist. Sadly, many providers do not implement this, as indicated by backchannel_logout_supported within their OpenID well-known discovery document.

• Alternatively, we could show external SSO users on the user list for admins and allow disabling the accounts from there, including revoking all tokens. While this does not disable them automatically from the OIDC provider and should be used in tandem with any of the two other solutions, this does make sure that they cannot login immediately, like with local accounts.

• Finally, we can use the UserInfo endpoint to do a basic check if the authentication is revoked, as detailed below:

Implementing OIDC

If you implement OIDC for authentication, you only get proof of user identity at that moment from the OIDC provider. Therefore, you can use the issued id_token to obtain identity data, such as name, email etc. You can also use the UserInfo Endpoint, which includes the same information.

You will also get an access_token and possibly a refresh_token. These should only be used for the UserInfo Endpoint.

Using the UserInfo Endpoint to check if a user is still valid

We can store these access_tokens and refresh_tokens on the backend (remember that they only have access to the openid, profile and email scopes and can thus do nothing else than UserInfo checks and are thus limited in security scope) to do a new user info request on every token refresh (refresh token / silent flow).

If a user is returned and it matches the saved OIDC user, we allow a new refresh or access token for our app to be issued and revoke the old refresh token or access tokens (as stated in the OAuth2 spec).

If the access token is expired and the refresh token cannot be used to obtain a new UserInfo Endpoint access token from the OIDC Provider, we MUST have the user re-authenticate.

Summary

This mostly solves the issue by:

• Allowing for long-lived sessions for local use, where you can revoke long lived access/refresh tokens within the app (as is currently possible on your User Profile screen within Home Assistant) because we check against the JTI on all API routes
• Allowing for long-lived sessions for OIDC use, for as long as the refresh/access token from the OIDC provider allows access to the UserInfo endpoint.
• Allowing for revocation of access rights at the OIDC Provider for Home Assistant as they will invalidate the saved access token and the next UserInfo endpoint check will fail (after at most the Home Assistant token lifetime, see above).
• Working with most OIDC implementations

However, it requires storing this IdP access/refresh token in the database.

Conclusion

While not all questions are answered with a perfect answer, as many parties do not fully implement all OIDC specs, we should not let perfect be the enemy of good. Yes, we should have warnings for users enabling SSO for Home Assistant, possibly even requiring them to install HACS, a custom OIDC plugin and then enable it within YAML. This will at least keep very novice users from doing something dangerous with badly configured OIDC providers.

However, the benefits of having external providers, with much stronger authentication than Home Assistant can ever implement within the bounds of the app (see above at Proposal), massively out way the downsides. Even SSO without proper logout is safer than users with bad passwords, leaky proxy auth configs and too extensive "trusted network" configs. With the solutions proposed and proper documentation, we can even eliminate most scenarios.

Home Assistant, a platform I trust daily to keep my home automated and controllable, deserves the option for strong SSO to be configured, such that I can rest assured that logging in is not possible without using my physical security key, while still allowing for me to quickly switch to other apps linked to my SSO config.

Final notes

Thank you @balloob for starting this project, and thanks for all maintainers and contributors for creating something that I interact with every day. This was just on my mind after trying to integrate Home Assistant with Authentik, after Immich succesfully integrated OpenID Connect as well. I was frustrated with the current tickets and the lost users, most of who do not know enough about the specification to write a full analysis and were just configuring proxy auth in many dangerous ways. Therefore, these are my 2 cents.

I would like to hear from everyone interested in this issue and I hope that we can all make Home Assistant better! I am also open to call about this or answering any comments you might have. Thank you for reading!

@BeryJu, @elupus, @bramkragten, @bdraco, @frenck, @pvizeli

[frenck]

Please do not ping everybody to get attention, that is generally not appreciated (surely not by me). It makes me less interested as you demanded my attention.

Additionally, we do not use issues for architectural proposal discussions anymore. If you want to propose an architectural change, please use discussion (as highlighted on multiple places). Be sure to include the technical change you want to make and detail the implementation you want to make. As this repo is not for raising feature requests but for getting an architectural change, you are implementing, approved.

../Frenck

[christiaangoossens]

Please do not ping everybody to get attention, that is generally not appreciated (surely not by me). It makes me less interested as you demanded my attention.

Additionally, we do not use issues for architectural proposal discussions anymore. If you want to propose an architectural change, please use discussion (as highlighted on multiple places). Be sure to include the technical change you want to make and detail the implementation you want to make. As this repo is not for raising feature requests but for getting an architectural change, you are implementing, approved.

../Frenck

@frenck Thank you for your response. I pinged you, and some others, because you have all directly contributed to authentication in Home Assistant before, either through a PR, plugin or as a core maintainer from the first hour, and therefore I would greatly appreciate your input specifically.

With a busy day job, studying and a lot of volunteering, I sadly have no time to write a solution myself in this case. I have moved this discussion to the forum: https://community.home-assistant.io/t/open-letter-for-improving-home-assistants-authentication-system/494223, but it does not allow me to add the links. I also value this Github issue as it allows me to link it to other PR's, people and general internet links such that others can find it, who had the same issues I did. I have also started sponsoring some maintainers, and I would be open to discussing sponsering this implementation as well.

It was, and never will be my intention to annoy open-source contributors by not following procedure necessary to keep this maintainable, but after having spent more than 1,5 hours creating a comprehensive overview of the issue with a lot of pointers, I am quite sad to have to take it off Github (where it is easily findable) and post it without links. It feels a bit like you are taking a beating for trying to help.

Nevertheless, keep up the good work on Home Assistant. I will close this issue for now. ❤️

[frenck]

Reading through this all, I think it is not worth spending time on. While cool to be able to use external auth, it works a little against the core values (local vs remote in auth), but even disregarding that, I genuinely believe the use case and, eventually, the user base using it would be negligible.

As a matter of fact, I think it would be so small, it won’t be worth the effort of reviewing it, and maintaining it IMHO. Which probably also was the part of the driver in the listed PRs.

Webauthn was mentioned above too, which is an absolutely lovely example.
I would love to have it; it was even attempted to be added, reviews weren’t followed up, and it died stale. It hasn’t been tried again later. Just to point out, the interest is even low from a contributor perspective. This is not just that example; we have seen very little auth-related PRs.

Home Assistant is aimed at a Home user, the home environment. IMHO this proposal/open letter is for feeding the enterprise smart home syndrome. I am pretty sure my dad (or any other average user of Home Assistant) isn’t using SSO to log in to his home devices.

../Frenck

[frenck]

It feels a bit like you are taking a beating for trying to help.

Sorry, I was just trying to help by getting things in the right place and not getting across as demanding, as that doesn’t help, IMHO. We have 13.5k contributors yearly; if we all start pinging, it becomes a mess. Worse, some contributors started turning off notifications because of such behavior. We should be sparse with demanding attention by pinging. Don’t take it too personal 😉

I appreciate your write-up and did formulate a response to the contents above (wish was crossed during your closing this).

If you are not pursuing implementing a change, the forums might indeed be a better place 👍 In general, the forums can help getting the “community voice” involved too 👍

../Frenck

[elupus]

For what its worth i would support openid or other authentication providers. Ive already done such an implementation once. I dont think mom/dad is the target for esphome custom devices either, yet we support those. So i dont feel that is a good argument. But I wont fight for it either.

[frenck]

I've locked this discussion, as per above (first line), the discussion has moved, as this is not the right place for these things.

../Frenck