Always on display #1869

KaffeinatedKat · 2023-09-30T03:14:44Z

I have added an option in the "Display timeout" for an always on display. Not sure how useful this is, or how good the battery life is (will be testing this and will post a comment with how long the battery lasted).

Any suggestions on how I could possibly increase the battery with the display always on would be great.

github-actions · 2023-09-30T03:25:04Z

Build size and comparison to main:

Section	Size	Difference
text	378724B	1516B
data	948B	8B
bss	63504B	-52B

JF002 · 2023-09-30T18:02:44Z

This will probably reduce the battery life to just a few hours. The LCD uses between 10 and 30mA when it's ON, which is quite high compared to the current power usage of InfiniTime (less than 1mA).

I can't think of any way to reduce power usage of the display when it's on... Lowering the brightness will effectively reduce the power usage, but probably not enough to maintain a reasonable battery life. This is caused by the LCD technology of the display.

KaffeinatedKat · 2023-09-30T18:12:56Z

is there anyway to reduce the brightness lower than the lowest setting right now? Currently on the lowest brightness setting it's gone ~15 hours and it's at ~35%, which was better than I was expecting

JF002 · 2023-09-30T18:17:58Z

You could drive the brightness using PWM. There might already be a PR for that.

mark9064 · 2023-10-01T11:22:02Z

You could drive the brightness using PWM. There might already be a PR for that.

Indeed there is, see #575

KaffeinatedKat · 2023-10-01T16:48:23Z

I've changed it so that the "always on" setting prevents the display from turning off, but the watch still goes into the sleep mode, and just sets it to a low brightness with pwm. Now the display will light up when any of the wakeup actions happen, then go dim again after the display timeout.

My only issue is the watch is in sleep mode and the watchface does not update while sleeping. Because the display is still on the watchface is just frozen till woken up.
I've been poking around the code and I cannot find where I can make the display update while sleeping

A video of the watchface not updating while asleep:
https://drive.google.com/file/d/1pNt-cbVS1s_8tmAK6B8ZmDR_WU164OI8/view

mark9064 · 2023-10-01T23:20:03Z

A common pattern you'll spot all across InfiniTime is using queue read timeout to implement periodic tasks. Many tasks have an event queue that they read from forever, and when fetching an event from a queue you can specify a timeout, which is how long to wait if nothing arrives. After this timeout, the queue fetch function returns that no items were fetched from the queue as nothing was available in the time specified. So if you have a loop which does:

get event with timeout 1 minute
process event if one exists
execute x task

then x task runs at least every minute.

This is exactly how it's implemented for the display, with x task in this case being refreshing display contents. So change

    case States::Idle:
      queueTimeout = portMAX_DELAY;
      break;

to

    case States::Idle:
      if (settingsController.GetAlwaysOnDisplay()) {
        queueTimeout = lv_task_handler(); // returns time until LVGL tasks need running
      } else {
        queueTimeout = portMAX_DELAY;
      }
      break;

and you're good :)

mark9064 · 2023-10-01T23:39:30Z

Seems to work well with the above change. I haven't been running it long enough to test battery yet though. One suggestion would be to disable always on if InfiniTime is set to sleep mode

KaffeinatedKat · 2023-10-02T00:06:43Z

Seems to work well with the above change. I haven't been running it long enough to test battery yet though. One suggestion would be to disable always on if InfiniTime is set to sleep mode

This is something I plan to implement, this feature would pair really nicely with #1461

KaffeinatedKat · 2023-10-02T00:30:30Z

A common pattern you'll spot all across InfiniTime is using queue read timeout to implement periodic tasks. Many tasks have an event queue that they read from forever, and when fetching an event from a queue you can specify a timeout, which is how long to wait if nothing arrives. After this timeout, the queue fetch function returns that no items were fetched from the queue as nothing was available in the time specified. So if you have a loop which does:
1. get event with timeout 1 minute

2. process event if one exists

3. execute x task
then x task runs at least every minute.

This is exactly how it's implemented for the display, with x task in this case being refreshing display contents. So change
    case States::Idle:
      queueTimeout = portMAX_DELAY;
      break;
to
    case States::Idle:
      if (settingsController.GetAlwaysOnDisplay()) {
        queueTimeout = lv_task_handler(); // returns time until LVGL tasks need running
      } else {
        queueTimeout = portMAX_DELAY;
      }
      break;
and you're good :)

I have applied this, and it doesn't work. Screen still doesn't update while sleeping and when I wake it up the colors invert, which is extremely weird. Any idea why?

KaffeinatedKat · 2023-10-03T16:46:30Z

I have applied this, and it doesn't work. Screen still doesn't update while sleeping and when I wake it up the colors invert, which is extremely weird. Any idea why?

I have solved this issue, this change worked fine in the simulator but did not work on the watch. Figured out that by not putting the SPI to sleep, these changes work as intended

mark9064 · 2023-10-03T22:02:18Z

Nice job, I'm running a patchset that disables SPI sleep and I suspected that might be difference - unfortunately I didn't have time to test and report back so sorry about that

One change I'd suggest: running the LVGL tasks full speed (ie max display refresh) is pretty power hungry, and I've found throttling it to 250ms saves power while keeping watchfaces with seconds up to date (as the display effectively refreshes at 4Hz). I'm getting ~30h battery life running the screen on constantly right now (This is a calculated number as I turn it off at night)
I also thought for correctness it might be good to pull up the screen running check. It also might be good not to reinitialise the SPI inside GoToRunning if it wasn't actually suspended

So I've been running this - feel free to integrate changes if you fancy

      if (settingsController.GetAlwaysOnDisplay()) {
        if (!currentScreen->IsRunning()) {
          LoadPreviousScreen();
        }
        int lvglWaitTime = lv_task_handler();
        // while in always on mode, throttle LVGL events to 4Hz
        queueTimeout = std::max(lvglWaitTime, 250);
      } else {
        queueTimeout = portMAX_DELAY;
      }
      break;

I've also noticed an issue where sometimes the brightness is on Low rather than Lowest while the watch is asleep - I haven't had time to debug that either but see if you spot it. For reference I have my normal brightness set to Low

KaffeinatedKat · 2023-10-04T02:54:13Z

src/components/settings/Settings.h

+	  } else {
+            settings.alwaysOnDisplay.state = true;
+	  }
+	}


i'm unsure if this code belongs here, or if we should check the notification status before sleeping instead

escoand · 2023-10-04T06:07:13Z

Wouldn't be an update every minute enough for an always on display? It should increase the battery runtime further.

Off course with hiding the seconds hand...

mark9064 · 2023-10-04T08:27:43Z

Wouldn't be an update every minute enough for an always on display? It should increase the battery runtime further.

Off course with hiding the seconds hand...

Yeah it's definitely something worth thinking about. On one hand it saves power, but on the other, it allows incorrect information to be onscreen for more than a minute like seconds on watchfaces with them, seconds on the timer app, bluetooth status, music status, steps, heart rate (if continuous/background measuring implemented) etc
Whether these are important enough for the power saving, I'm not sure. Maybe it's worth testing to see how big the power difference is?

KaffeinatedKat · 2023-10-04T17:25:40Z

Wouldn't be an update every minute enough for an always on display? It should increase the battery runtime further.
Off course with hiding the seconds hand...

Yeah it's definitely something worth thinking about. On one hand it saves power, but on the other, it allows incorrect information to be onscreen for more than a minute like seconds on watchfaces with them, seconds on the timer app, bluetooth status, music status, steps, heart rate (if continuous/background measuring implemented) etc Whether these are important enough for the power saving, I'm not sure. Maybe it's worth testing to see how big the power difference is?

This might be worth it, we can set the update timer to 4Hz when apps like the timer and music are running, but set it back when not running these apps; perhaps an enum with apps that need a higher refresh timer. We can also disable the seconds on every watchface that has them when always on mode. We could also add a setting to disable this if someone wants seconds on. I think with the display the pinetime has, any power savings we can get is worth it

hexisXz · 2023-10-12T19:58:56Z

This will probably reduce the battery life to just a few hours. The LCD uses between 10 and 30mA when it's ON, which is quite high compared to the current power usage of InfiniTime (less than 1mA).

I can't think of any way to reduce power usage of the display when it's on... Lowering the brightness will effectively reduce the power usage, but probably not enough to maintain a reasonable battery life. This is caused by the LCD technology of the display.

I honestly think this would be pretty useful. You could just have a low quality version of the digital clock display on the screen with like medium brightness and just have everything else suspended in the background.

KaffeinatedKat · 2023-10-14T04:40:02Z

This pr, for the most part, is complete. Only remaining issue is sometimes the display does not go into the lowest setting when going to sleep, and stays at the brightness setting. I have not been able to figure this out because it happens so infrequently.

@JF002, is there any way this feature would be considered for merging, or is the battery life reduction while enabled to high? Maybe there could be some kind of popup warning when enabled, to notify the user of the significant battery drain while enabled.

It lasts about a full day when enabled, and most smartwatches only last that long anyways. I feel like the people who genuinely find this useful would be willing to charge it everyday

mark9064 · 2023-10-14T13:59:48Z

The issues with the screen getting stuck in Low are due to the restorebrightness messages coming from the system task. (Edit: I can post some actual patches soon)

I think this PR (with the above issue fixed) is usable. But we should also test what the majority of the power consumption is when always on (I suspect it's the LCD) and if we can reduce it further (the LCD offers a power mode between sleep (display blank) and fully on with reduced colours etc.). Also might be possible to redrive the LCD at a lower refresh rate when in always on, but the datasheet for it on the Pine wiki is an absolute monster and I haven't had the time to fully explore it. As an aside it looks like the solution to the display invert issue might be in there too

KaffeinatedKat · 2023-10-14T16:41:11Z

I took a peek at the datasheet, and implementing switching to reduced colors while always on. The datasheet says this uses less power, but by how much I am unsure. I don't have a devkit, and can't measure power consumption.

This is also essentially free power reduction. Turns out, InfiniTime doesn't utility many colors at all, and while in reduced colors, you can hardly tell. The reduced brightness also helps hide it. The one app where you can really tell is 2048, and I doubt anybody will be letting it fall asleep on 2048 anyways.

I'm gunna keep poking around the datasheet to see if I can figure out the refresh rate thing, if that is possible it would probably help the power consumption by a sizeable margin

mark9064 · 2023-10-14T21:23:32Z

Oh yeah also forgot to mention, the settings version number should be bumped as a new member has been added

mark9064 · 2023-10-14T21:49:10Z

#1869 (comment)

The patch in question: mark9064@d43759f
Feel free to pull the commit into your branch if you're happy with it :)

Locally working well - to reproduce the issue on the original try disconnecting and reconnecting BT (the display should get stuck on after). With the patch it should stay in always on as expected

Reduced colours mode seems to be working, only just flashed it so haven't had time to see if there are power changes (I only have sealed). Some of the watchfaces suffer though

KaffeinatedKat · 2023-10-14T22:07:15Z

Applied your patch, and it has fixed the issue on my device as well. Bumped the settings version, while I was at it

Also figured out how to modify the refresh rate, it now goes to ~4.8hz while always on, which is the lowest setting available. Hopefully this helps increase the battery life

If anyone with a devkit wants to test if the reduced colors actually saves a realistic amount of power that would be great. I feel like it would be worth reverting if it doesn't save enough power due to making some watchfaces look worse

mark9064 · 2023-10-14T22:29:56Z

+1 on the reduced colours thought

Can confirm refresh rate changes have applied properly, well done on figuring that out! I think it should actually be possible to inform LVGL of the exact refresh rate too and that way we shouldn't have to do the timer throttling hack? I'm not totally sure on this, I'd need to dig through the LVGL internals but it might be worth considering

mark9064 · 2023-10-14T22:44:33Z

I had a quick look: it seems that LV_DISP_DEF_REFR_PERIOD is used as a define to set the refresh period on all the LVGL tasks to 20ms. So I think what we'd have to do is go through all of these tasks (there are many inside InfiniTime and a couple in LVGL itself), and then use lv_task_set_period to set them to a longer execution period. Quite a bit of work, but looks possible

Edit: Damn this is looking pretty complicated. Some of the tasks should be suspended altogether (like the input reading task) while some should just be slowed. Not sure how feasible doing this is

Edit 2: The more I think about it, the less this seems like a good idea. Trying to bring all of the LGVL tasks together to change the intervals sounds like a bit of a nightmare organisation wise. Perhaps it would be better to properly understand exactly what lv_task_handler does and see if we should implement our own task handler that runs tasks with efficiency in mind rather than latency (ie batching task execution). What are your thoughts?

KaffeinatedKat · 2023-10-15T00:39:21Z

Perhaps it would be better to properly understand exactly what lv_task_handler does and see if we should implement our own task handler that runs tasks with efficiency in mind rather than latency (ie batching task execution). What are your thoughts?

I think that this is a good idea, would this new task handler replace lv_task_handler all together, or only be used for the always on display?

mark9064 · 2023-10-15T23:17:50Z

Probably just for always on as I suppose we still want responsive inputs etc while on. But I guess it depends on how it's implemented? It might even not be needed at all - I'm not sure how the LVGL scheduler works

A quick list of things I've noticed so far:

Screen tearing is quite visible. Try with the analogue face (or timer app), notice how the second hand gets chopped in half occasionally. Also, the lcd pulses with the refresh rate (pretty tricky to spot but try looking at the top right). Maybe the LCD is being driven too slowly? I'm honestly no expert on what refresh rates the panel supports. I think I might try keeping 60Hz but using /8 divider on that and seeing if it does better - will report back. There also might be some VSYNC stuff we can do with the panel to fix this
Frame pacing is noticeably poor (visible with seconds) but honestly I think that's fine as a sacrifice if it gets battery life back
There are still some wake/sleep state bugs I need to investigate (display gets stuck on in some scenarios)
A background transition to the Sleep mode (eg with the auto sleep PR) causes everything to break

Battery life is looking promising though! Not sure if it's the reduced refresh or reduced colours but I'm seeing an improvement for sure

KaffeinatedKat · 2023-10-16T00:50:38Z

Screen tearing is quite visible

I've noticed the screen tearing, it's really bad if you run it at that refresh rate while awake. I don't personally run a watchface with seconds and so I haven't noticed it much, but it's for sure worth investigating for those who do. I'll do some tinkering with the refresh rate and see about VSYNC

A background transition to the Sleep mode (eg with the auto sleep PR) causes everything to break

I have encountered this, stopping the spi from sleeping altogether fixes the crashing, and then I just call "GoToRunning" to fix the screen state being wrong afterwards (see e6d60f1). We should find a way to fix this bug without the autosleep pr however, i'll keep you updated on that

There are still some wake/sleep state bugs I need to investigate (display gets stuck on in some scenarios)

I have not encountered any of these, if you could provide some scenarios that cause this, I can help debug it

mark9064 · 2023-10-16T22:28:53Z

Coming in on 48h runtime now with 20% still in the tank, very impressive! I'd say it's a lot more usable with those changes, awesome job

Another thought I had for power usage is driving the LCD at a slightly lower voltage. From what I can see the LCD driver allows choosing what drive voltage is used for the LCD, so maybe a slightly lower voltage could save some power? It also might cause the LCD to not turn pixels on/off properly so not sure if this is a viable route (and without a power profiler I'm not even sure if the panel draws much power or if it's mostly the driver chip). Also there appears to be about 10 driver voltages 😬 so yeah sounds fun

One other thing: some of the comments from the original PWM PR should probably be addressed. I think the points about not running the PWM engine when the screen is off or at one of the fixed brightness modes has merit, with PWM only being used between brightness settings. I think it also makes sense to PWM just one pin at a time, though I can see the formulas for calculating brightness being annoying as the brightness increase per duty increase is different for each pin
Ideally I think this should take the form of a unified brightness interface where it takes some target brightness value between 0-10000 (or whatever) and the brightness code calculates what pin configuration is needed along with whether PWM is required? I could have a crack at implementing this, what do you reckon?

KaffeinatedKat · 2023-10-16T22:38:11Z

I can go ahead and mess with the voltage stuff, still messing around with the refresh stuff to reduce screen tearing, might as well mess with the voltage while i'm at it. Really glad to see how much potential there is for greater battery life with the display being on all the time, considering it's just a regular LCD screen

About the PWM stuff, I honestly have no idea how it works, but if you wanna take a crack at implementing the unified brightness interface, I would happy merge that into the branch. Sounds like it would be a really nice addition to improving the PWM implementation.

mark9064 · 2024-06-19T16:14:56Z

I've just rebased to migrate this PR to batched display arguments as well, and to accept the RTC based implementation.

Thanks for the power testing again, you've been brilliant running so many tests over the course of this PR!
300µA takes AOD backlight PWM from 50% to 100% efficiency :)
I think that's as good we'll get

All required PRs here are merged

mark9064 · 2024-06-19T20:37:16Z

Small rebase mistake (used old version of a comment)

There are no more technical enhancements I can think of - if people want to try this PR out and feedback now is the time :)

The commits should tell a reasonably coherent story of development. We might want to consider breaking out the brightness message refactor (fix brightness getting stuck high / flashlight brightness restore) but I think it's a simple enough change?

Either way, everything here is ready for review

KaffeinatedKat · 2024-06-19T23:08:28Z

awesome to see that this pr is finally ready for review

thanks so much to @mark9064, without your help with all the lower level issues this would not have been possible

JF002 · 2024-06-23T16:11:06Z

Thank you very much for you work @mark9064 !

I flashed this branch on my PineTime so I can test it, and I'm pleasantly surprised by the result : the time is well readable in most situation : indoor and even outside if you live in Belgium right now (it's been raining for numerous weeks...).

The battery lasted for 2 days on a single charge (BLE enabled, wake up on double tap on the display), From June 19th to June 22nd :

While it's expected that the always-on display would reduce battery life, 2 full days is a reasonable trade-off

I'll do my best to review this PR ASAP !

mark9064 · 2024-06-23T23:00:17Z

And in real use the screen is off ~8h per night, so 3 days is possible (total runtime 48h as tested above - 16h on per day in real usage - 48/16 = 3)

One thing that's come to mind - should the brightness PWM bits belong in a driver rather than the controller?

JF002

Thanks you for this very high quality work (in this PR and all previous PR that were needed for the always on mode)!

I have the feeling that all the low-level code added in BrightnessController should fit better in a driver class rather than in a Controller class (drivers encapsulates all the low-level interactions with the hardware, controllers add higher-level - applicative logic to those driver).
But since there's no brightness driver in InfiniTime right now, I guess we can do this in a future PR :)

I added a few non-critical comments. Would you like to have a look at them? The most important one consists in clarifying the always-on setting (state, enabled).

src/components/settings/Settings.h

src/components/brightness/BrightnessController.h

mark9064 · 2024-07-21T15:25:21Z

Cleaned up code style in a couple other places. I think all that's left to resolve now is the private class constants discussion

JF002 · 2024-08-05T18:34:11Z

Thank you so much to everyone who contributed to this PR, and especially to @mark9064 who did a wonderful job and who was very patient during this very long review process (sorry about that!) 🥇

mark9064 · 2024-08-05T21:05:51Z

A massive thanks to you too for taking the time to test configurations and review/discuss even the most minor details (including for all the supporting PRs!). Also cheers to @KaffeinatedKat for graciously trusting me with contributing directly to this feature :)

KaffeinatedKat commented Oct 4, 2023

View reviewed changes

mark9064 force-pushed the aod branch from e3afe07 to c5aac9a Compare June 19, 2024 16:13

mark9064 force-pushed the aod branch from c5aac9a to ae0e893 Compare June 19, 2024 20:30

JF002 requested changes Jul 13, 2024

View reviewed changes

src/components/settings/Settings.h Outdated Show resolved Hide resolved

src/components/settings/Settings.h Outdated Show resolved Hide resolved

src/components/brightness/BrightnessController.h Show resolved Hide resolved

KaffeinatedKat and others added 13 commits July 21, 2024 16:19

aod: integrate with display timeout

301e68b

aod: disable while in notification sleep

765f037

aod: switch to 8 colors when always on

d99651d

aod: lower refresh rate when always on

1876249

aod: fix brightness getting stuck high

4da3153

aod: fix flashlight brightness restore

6c2fcea

aod: lower voltage going to the display

259c171

aod: lower lcd voltage

40f98d5

aod: porch control: 2Hz idle + 75Hz on

8051ae8

aod: constant frequency idle frames

504fd05

aod: run LVGL task handler until all work finished

2710237

aod: avoid spinning DisplayApp under high LVGL load

2b1f194

aod: simplify AOD disablement based on notification status

cc1d4fe

mark9064 force-pushed the aod branch from 7f3720e to cc1d4fe Compare July 21, 2024 15:24

JF002 merged commit 53dc9da into InfiniTimeOrg:main Aug 5, 2024
6 of 7 checks passed

This was referenced Aug 18, 2024

Restore brightness level when closing the Flashlight application #2038

Closed

Fix brightness icon when swiping to close flashlight #1907

Closed

Brightness icon changes to flashlight's brightness when flashlight is closed by swipe #1751

Closed

FintasticMan added this to the 1.15.0 milestone Nov 21, 2024

Always on display #1869

Always on display #1869

Conversation

KaffeinatedKat commented Sep 30, 2023

github-actions bot commented Sep 30, 2023 • edited Loading

JF002 commented Sep 30, 2023

KaffeinatedKat commented Sep 30, 2023

JF002 commented Sep 30, 2023

mark9064 commented Oct 1, 2023

KaffeinatedKat commented Oct 1, 2023

mark9064 commented Oct 1, 2023 • edited Loading

mark9064 commented Oct 1, 2023

KaffeinatedKat commented Oct 2, 2023

KaffeinatedKat commented Oct 2, 2023

KaffeinatedKat commented Oct 3, 2023

mark9064 commented Oct 3, 2023 • edited Loading

KaffeinatedKat Oct 4, 2023

Choose a reason for hiding this comment

escoand commented Oct 4, 2023 • edited Loading

mark9064 commented Oct 4, 2023

KaffeinatedKat commented Oct 4, 2023

hexisXz commented Oct 12, 2023

KaffeinatedKat commented Oct 14, 2023

mark9064 commented Oct 14, 2023 • edited Loading

KaffeinatedKat commented Oct 14, 2023

mark9064 commented Oct 14, 2023

mark9064 commented Oct 14, 2023 • edited Loading

KaffeinatedKat commented Oct 14, 2023 • edited Loading

mark9064 commented Oct 14, 2023

mark9064 commented Oct 14, 2023 • edited Loading

KaffeinatedKat commented Oct 15, 2023 • edited Loading

mark9064 commented Oct 15, 2023 • edited Loading

KaffeinatedKat commented Oct 16, 2023

mark9064 commented Oct 16, 2023

KaffeinatedKat commented Oct 16, 2023

mark9064 commented Jun 19, 2024 • edited Loading

mark9064 commented Jun 19, 2024

KaffeinatedKat commented Jun 19, 2024

JF002 commented Jun 23, 2024

mark9064 commented Jun 23, 2024

JF002 left a comment

Choose a reason for hiding this comment

mark9064 commented Jul 21, 2024 • edited Loading

JF002 commented Aug 5, 2024

mark9064 commented Aug 5, 2024

github-actions bot commented Sep 30, 2023 •

edited

Loading

mark9064 commented Oct 1, 2023 •

edited

Loading

mark9064 commented Oct 3, 2023 •

edited

Loading

escoand commented Oct 4, 2023 •

edited

Loading

mark9064 commented Oct 14, 2023 •

edited

Loading

mark9064 commented Oct 14, 2023 •

edited

Loading

KaffeinatedKat commented Oct 14, 2023 •

edited

Loading

mark9064 commented Oct 14, 2023 •

edited

Loading

KaffeinatedKat commented Oct 15, 2023 •

edited

Loading

mark9064 commented Oct 15, 2023 •

edited

Loading

mark9064 commented Jun 19, 2024 •

edited

Loading

mark9064 commented Jul 21, 2024 •

edited

Loading