[Fix] Netatmo refresh_token breaking change

Author: philippelt

README.md

@@ -7,16 +7,11 @@ For more detailed information see http://dev.netatmo.com
 I have no relation with the netatmo company, I wrote this because I needed it myself,
 and published it to save time to anyone who would have same needs.
 
-Following the implementation of "Home" everywhere in the Netatmo API with various behavior, the library has been adjusted to include the home parameters in most calls.
-If you are using a single account with a single home and single weather station, the library has been implemented so that your code should continue to run without change.
+I am trying to make this library survive to continuous Netatmo changes but their habbit to introduce breaking changes anytime without notice make this target hard to reach.
 
-If you have multiple homes or were supplying a station name in some method calls, you will have to adapt your code :
- - to supply a home name when looking for data for most class initializers
- - to use the new station name set by Netatmo (which is not your previously set value)
-  
->BREAKING CHANGE: Netatmo seems no longer (july 2023) to allow grant_type "password", even for an app credentials that belong to the same account than the home. They have added the capability of creating access_token/refresh_token couple from the dev page (the location where app are created). As a consequence, the username/password credentials can no longer be used and you must replace them with a new parameter REFRESH_TOKEN that you will get from the web interface. To get this token, you are required to specify the scope you want to allow to this token. Select all that apply for your library use.
+>BREAKING CHANGEi (july 2023): Netatmo seems no longer (july 2023) to allow grant_type "password", even for an app credentials that belong to the same account than the home. They have added the capability of creating access_token/refresh_token couple from the dev page (the location where app are created). As a consequence, the username/password credentials can no longer be used and you must replace them with a new parameter REFRESH_TOKEN that you will get from the web interface. To get this token, you are required to specify the scope you want to allow to this token. Select all that apply for your library use.
 
->SHORT VERSION TO UPGRADE: If you where using a netatmo_credentials file, juste remove USERNAME and PASSWORD fields and add a REFRESH_TOKEN field which value is the one you will obtain from the https://dev.netatmo.com in MyApps selecting you app and using "Token Generator" after selecting required scopes.
+>NEW BREAKING CHANGE (december 2023): Web generated refresh_tokens are no more long lived tokens, they will be automatically refreshed. Consequences : No more static authentication in the library source and ~/.netatmo-credentials file will be updated to reflect change in the refresh token. This file MUST be writable and if you run Netatmo tools in container, remember to persist this file between container run.
 
 ### Install ###
 

lnetatmo.py

@@ -42,34 +42,22 @@
 
 # To ease Docker packaging of your application, you can setup your authentication parameters through env variables
 
-# Authentication use :
-#  1 - Values hard coded in the library
-#  2 - The .netatmo.credentials file in JSON format in your home directory
-#  3 - Values defined in environment variables : CLIENT_ID, CLIENT_SECRET, REFRESH_TOKEN
-
-# Each level override values defined in the previous level. You could define CLIENT_ID and CLIENT_SECRET hard coded in the library
-# and REFRESH_TOKEN in .netatmo.credentials or environment variables
-
-# 1 : Embedded credentials
-cred = {                           # You can hard code authentication information in the following lines
-        "CLIENT_ID" :  "",         #   Your client ID from Netatmo app registration at http://dev.netatmo.com
-        "CLIENT_SECRET" : "",      #   Your client app secret   '     '
-        "REFRESH_TOKEN" : ""       #   Your scoped refresh token (generated with app credentials)
-        }
+# Authentication:
+#  1 - The .netatmo.credentials file in JSON format in your home directory (now mandatory for regular use)
+#  2 - Values defined in environment variables : CLIENT_ID, CLIENT_SECRET, REFRESH_TOKEN
 
-# Other authentication setup management (optionals)
+# Note that the refresh token being short lived, using envvar will be restricted to speific testing use case
 
+# Note: this file will be rewritten by the library to record refresh_token change
+# If you run your application in container, remember to persist this file
 CREDENTIALS = expanduser("~/.netatmo.credentials")
+with open(CREDENTIALS, "r") as f:
+    cred = {k.upper():v for k,v in json.loads(f.read()).items()}
 
 def getParameter(key, default):
-    return getenv(key, default[key])
-
-# 2 : Override hard coded values with credentials file if any
-if exists(CREDENTIALS) :
-    with open(CREDENTIALS, "r") as f:
-        cred.update({k.upper():v for k,v in json.loads(f.read()).items()})
+    return getenv(key, default.get(key, None))
 
-# 3 : Override final value with content of env variables if defined
+# Override values with content of env variables if defined
 _CLIENT_ID     = getParameter("CLIENT_ID", cred)
 _CLIENT_SECRET = getParameter("CLIENT_SECRET", cred)
 _REFRESH_TOKEN = getParameter("REFRESH_TOKEN", cred)
@@ -261,9 +249,11 @@ def renew_token(self):
                 }
         resp = postRequest("authentication", _AUTH_REQ, postParams)
         if self.refreshToken != resp['refresh_token']:
-            print("New refresh token:", resp['refresh_token'])
+            self.refreshToken = resp['refresh_token']
+            cred["REFRESH_TOKEN"] = self.refreshToken
+            with open(CREDENTIALS, "w") as f:
+                f.write(json.dumps(cred, indent=True))
         self._accessToken = resp['access_token']
-        self.refreshToken = resp['refresh_token']
         self.expiration = int(resp['expire_in'] + time.time())
 
 

setup.py

@@ -4,7 +4,7 @@
 
 setup(
     name='lnetatmo',
-    version='3.3.0',
+    version='4.0.0',
     classifiers=[
         'Development Status :: 5 - Production/Stable',
         'Intended Audience :: Developers',
@@ -17,7 +17,7 @@
     scripts=[],
     data_files=[],
     url='https://github.com/philippelt/netatmo-api-python',
-    download_url='https://github.com/philippelt/netatmo-api-python/archive/v3.3.0.tar.gz',
+    download_url='https://github.com/philippelt/netatmo-api-python/archive/v4.0.0.tar.gz',
     license='GPL V3',
     description='Simple API to access Netatmo weather station data from any python script.'
 )

usage.md

@@ -17,6 +17,8 @@ Python Netatmo API programmers guide
 
 >2023-07-24, Adding rawAPI call to direct access the netatmo API when no additional support is provided by the library
 
+>2023-12-04, New update to Netatmo authentication rules, no longer long lived refresh token -> credentials MUST be writable, Hard coding credentials in the library no longer possible (bad luck for small home automation device)
+
 No additional library other than standard Python library is required.
 
 Both Python V2.7x and V3.x.x are supported without change.
@@ -38,8 +40,6 @@ Before being able to use the module you will need :
   * An application registered from the user account (see http://dev.netatmo.com/dev/createapp) to obtain application credentials.
   * Create a couple access_token/refresh_token at the same time with your required scope (depending of your intents on library use)
 
-In the netatmo philosophy, both the application itself and the user have to be registered thus have authentication credentials to be able to access any station. Registration is free for both.
-
 
 
 ### 2 Setup your authentication information ###
@@ -48,10 +48,9 @@ In the netatmo philosophy, both the application itself and the user have to be r
 
 Copy the lnetatmo.py file in your work directory (or use pip install lnetatmo).
 
-Authentication data can be supplied with 4 different methods (each method override any settings of previous methods) : 
+Authentication data can be supplied with 3 different methods (each method override any settings of previous methods) : 
 
- 1. Some or all values can be hard coded in the library source (and default to empty strings). If you use this method, you are back to the initial suggested method. It would be nice to switch to other methods to isolate credentials and ease library upgrades.
- 2. Some or all values can be overriden in a ~/.netatmo.credentials (in your platform home directory) file containing the keys in JSON format  
+ 1. Some or all values can stored in ~/.netatmo.credentials (in your platform home directory) file containing the keys in JSON format  
 
         $ cat .netatmo.credentials   # Here all values are defined but it is not mandatory
         {
@@ -61,19 +60,21 @@ Authentication data can be supplied with 4 different methods (each method overri
         }
         $
 
- 3. Some or all values can be overriden by environment variables. This is the easiest method if your are packaging your application with Docker. It also allow you to do some testing with other accounts without touching your current ~/.netatmo.credentials file 
+> Due to Netatmo continuous changes, this method is the only one available for production use as the refresh token will be frequently refreshed and this file MUST be writable by the library to keep a usable refresh token.
+
+ 2. Some or all values can be overriden by environment variables. This is the easiest method if your are packaging your application with Docker. It also allow you to do some testing with other accounts without touching your current ~/.netatmo.credentials file 
 
         $ export REFRESH_TOKEN="yyy"
         $ python3 MyCodeUsingLnetatmo.py
         ...
         
- 4. Some or all values can be overriden by explicit call to initializer of ClientAuth class  
+ 3. Some or all values can be overriden by explicit call to initializer of ClientAuth class  
 
         # Example: REFRESH_TOKEN supposed to be defined by one of the previous methods
         authData = lnetatmo.ClientAuth( clientId="netatmo-client-id",
                                         clientSecret="secret" )
 
-If you provide all the values, using any method or mix except 4, you can test that everything is working properly by simply running the package as a standalone program.
+If you provide all the values, using any method or mix except 3, you can test that everything is working properly by simply running the package as a standalone program.
 
 This will run a full access test to the account and stations and return 0 as return code if everything works well. If run interactively, it will also display an OK message.