Support MySQL 8 Authentication

[nwoltman]

This is a rebase + update of @ruiquelhas's PR #1962.

One thing that has been changed from the original PR is that only MySQL 8.0.x has been added to integration tests rather than multiple RC versions of MySQL 8. It's been long enough that people are probably only using GA versions of MySQL 8 now.

Fixes #2002

---

Original PR Description

This patch accommodates some breaking changes introduced with MySQL 8.

Closes #1959

In a nutshell, the caching_sha2_password plugin (which is used by default since MySQL 8.0.4) hashes the password using SHA-256 and, after a first successful authentication attempt, saves it in a cache. That first attempt needs to be done under one of two conditions. The client either uses SSL and sends the password as clear text or it encrypts the password using the RSA public key generated by the server. On any subsequent attempt, until the password is somehow removed from the cache or the server shuts down, these rules no longer apply.

The handshake process remains unchanged when connecting to any server with version lower or equal to 8.0.3. Whereas for 8.0.4 or above, the process is now the following:

• the client sends a ClientAuthenticationPacket with a scramble computed using a SHA-256 hash
• if the password is not cached, the server sends back a PerformFullAuthenticationPacket
• if the client uses SSL, the password is sent to the server (as clear text) via a ClearTextPasswordPacket to which the server replies with a OkPacket
• otherwise it uses the server authentication public key compute the scramble, sending a AuthSwitchResponsePacket to which the server replies with a OkPacket
• if the client does not know the server public key (is not provided by the user), it requests it from the server, which sends it back using a AuthMoreDataPacket
• after a first successful authentication attempt, and until the password is cached, the server will reply to the initial ClientAuthenticationPacket with a FastAuthSuccessPacket (which basically just signals that an OkPacket will follow)

If the account is created using the mysql_native_password authentication plugin, the client will just fall back to the "traditional" process during the handshake, keeping compatibility, by default for any previously supported server version.

MySQL 8.0.2 disables the local_infile server variable by default, which breaks a couple of integration tests. The tests were updated to enable the feature by themselves (something that does not have any effect on older server versions and allows the tests to pass with newer versions).

Additionally, one of the integration tests was updated to avoid failing after the first run (using any server version) since it tried to create a table that already existed from the previous runs.

[dougwilson]

This PR came in a bit before I took some time off. I am spending this weekend to get items on this module in place, though. I hope that helps. Now that the original PR has been rebased and CI is all passing, just need to do a quick review and then we can have a new version out here soon.

[dougwilson]

An update to what I said above (since there are some thumbs ups): I'm currently dealing with some fallout where GitHub is flagging a module as a security issue when it is not, which is causing many folks to open support requests I need to respond to. I have been trying to get GitHub to correct the detection, but have been unsuccessful in getting any response from them. I may end up having no choice but to spend the time to rework a module to get a dependency updated just to get the support requests to stop if GitHub does not correct their detection.

This may end up using all my time this weekend, but I will keep posted here. I wanted to post an update so folks don't think I'm ignoring this pr/repo if I don't get to it, just there is a higher priority issue at hand.

[focux]

@dougwilson Thank for your work. I really appreciate it.

[ErisDS]

@dougwilson this comes from a place of absolute understanding of how much awesome work you do - would it be helpful if other people tested this PR?

[dougwilson]

Yes, that would be awesome! I was also thinking this weekend to spin out an actual beta package to npm with this as well to help even myself get it testing in qa.

[SuperManifolds]

For what it’s worth we have been running a build using this PR fork in our beta branch for a few months now and so far not encountered an issue

[johannes]

If there's anything we (Oracle/MySQL) can do to help please let us know.

[ErisDS]

I was also thinking this weekend to spin out an actual beta package to npm with this as well to help even myself get it testing in qa.

Oh that would be fab - we (Ghost) would definitely be able to dedicate a bit of time to testing that next week 🙌

[sidorares]

this needs to be changed to rolling xor, otherwise it'll fail for password longer than 19 characters

See discussion at sidorares/node-mysql2#1044

[dougwilson]

Yea, I've been working to fix this PR up over the past week and I did notice that long passwords didn't work on the new auth. There are a bunch of other little minor issues I've been finding as well. I didn't intend to list them out and was just intending to push up all the changes here then merge, but let me know if you think I should list them all out in addition to pushing up the fixes.

[sidorares]

Note that mysql2 is using slightly different api for this. No config required, and if you need to use non-defaults you pass custom configured plugin as authPlugins: { caching_sha2_password: XXXX }. Should not be a big problems for user migrating both ways mysql<->mysql2

[ahrib]

Hi any update to this?

[dougwilson]

Hi @ahrib thanks for asking! Just earlier this year I released a minor update with a few things. It may very will be the last 2.x release. This is because I am organizing work for a 3.x line which will include this new auth methods from this pr. Look forward to an alpha or beta releasing soon! I have been getting the things resolved this week. Sorry for the extended "lul" in work here, but I'm committed to full stream to get many of these prs merged and released into a 3.0 early this year!

[drwharris]

Where is this at? Ive just started to work on an upgrade from 8.0.16 to 8.0.21 and cannot connect with

ER_NOT_SUPPORTED_AUTH_MODE: Client does not support authentication protocol requested by server; consider upgrading MySQL client.

I am at a dead standstill and if I cant get a resolution here very quickly I need to look at moving to the MySQL supplied client which is not my preference.

[nwoltman]

@drwharris From my end, I don't plan on updating this PR. For my own work, I'm going to be switching to mysql2, which supports MySQL 8's new default authentication protocol (caching_sha2_password).

You can keep using mysql though without getting that error if you switch the default authentication method for your MySQL instance to mysql_native_password.

[dr-js]

Some notes on debug connecting to MySQL8 with mysql@2: (using Mysql.createPool())

The mysql@2 package only support mysql_native_password auth plugin,
to corretly connecting to MySQL8 we will need to make sure:

• Change default_authentication_plugin = 'caching_sha2_password':
  This will cause mysql@2 connection to fail,
  as current code just responds with data for mysql_native_password and fail the auth
  (check with debug enabled).
  To change this, a start option like --default-authentication-plugin=mysql_native_password or edit in /etc/mysql/my.cnf is required (add [mysqld] \n default-authentication-plugin = mysql_native_password),
  as there is no runtime change for this option, (related doc),
  but we can check current value with SHOW VARIABLES LIKE 'default_authentication_plugin';.
• A user with auth plugin set to mysql_native_password:
  Use SQL command like to create/alter user:

  # create new & give access
  CREATE USER IF NOT EXISTS 'root'@'127.0.0.1' IDENTIFIED WITH mysql_native_password BY 'SOME-PASSWORD';
  GRANT ALL PRIVILEGES ON *.* TO 'root'@'127.0.0.1' WITH GRANT OPTION;
  # or reset existing
  ALTER USER 'root'@'127.0.0.1' IDENTIFIED WITH mysql_native_password BY 'SOME-PASSWORD';
  # log user status
  SELECT Host,User,plugin,authentication_string FROM mysql.user WHERE User='root';

  And when using the root user, better understand the difference between:
  • 'root'@'%' for any host
  • 'root'@'127.0.0.1' for local TCP
  • 'root'@'localhost' for local socket, and will magically be used for 'root'@'127.0.0.1' with sys-variable skip_name_resolve = OFF REF

[vlasky]

@1chisensei and others - if you are in a hurry, you can try my fork. It's available on NPM as @vlasky/mysql.

I have not only merged @nwoltman's fantastic PR, but I have also implemented the rolling XOR fix suggested by @sidorares that ensures passwords longer than 19 characters will work with the caching_sha2_password plugin.

I hope @dougwilson eventually finds the time to continue his excellent work on this package.

I forked it only because I urgently needed to add MySQL 8 support and (partial) protocol compression support to my other fork @vlasky/zongji which is a MySQL binlog listener that relies on functionality that currently only exists in this mysql package and not in node-mysql2.