Dynamic apps and custom watchfaces

[JF002]

Verification

• I searched for similar feature request and found none was relevant.

Pitch us your idea!

I want to install new watchfaces and apps without flashing the whole firmware

Description

InfiniTime is currently a monolithic firmware : everything is built into a single binary file : OS, drivers, applications & watchfaces, fonts, pictures,...

This feature request is about adding the possibility to create and install new watchfaces and app without having to rebuild and flash the whole firmware.

InfiniTime is not quite ready yet to support these features, but we are slowly but surely moving forward to that:

• The PineTime is equipped with a huge 4MB SPI flash memory
• We've already implemented a filesystem (based on LittleFS on that SPI flash memory
• We've also implemented a BLE FS API to allow companion apps to access to this FS via BLE
• We are working on [moving assets fonts and picture to that external memory

There's of course still a lot of researches, experiments and work to do before we can enable dynamic app and custom watchfaces.

Over the time, a few feature requests on this topic were opened, and I create this one to group them all.
Those existing feature requests are :

• API for plugins? #113
• Adding the ability to change the watch face without reflashing InfiniTime #421
• Add a way to make customized watch faces #755
• Support more than four watch faces #1055

[Elara6331]

I had an idea for this a while ago. It may be possible to use WebAssembly for this. Searching online, I see that there are a few projects that have successfully run WASM on microcontrollers. It will be slower than native, but if it can be done, should be an improvement over something like MicroPython. My concern would be memory usage, but I don't know just how much memory it uses.

There seems to be an example for ESP-IDF here, which uses FreeRTOS: https://github.com/bytecodealliance/wasm-micro-runtime/tree/main/product-mini/platforms/esp-idf. I'm not sure how applicable it would be to InfiniTime, but it might be useful.

Obviously, native code would be preferable, but I'm just not sure how to accomplish that without having to recompile every time you add something.

[Tiggilyboo]

If a binary can be transferred to the SPI memory via the BLE service and then executed directly on the watch, this would be ideal for performance without the VM / dynamic code execution baggage, nor additional compilation or interpreter overhead.

The question though still remains on how to "execute directly" from the FS?

Since the platform is FreeRTOS, the only way I can think of is through static linking (I don't think FreeRTOS can do dynamic linking). So how might this be approached? One such design could implement a lookup table of addresses (slots) which point to a point of flash memory to load. Each slot could follow a structure / API which must be followed (via some header for this purpose) in order to load / unload, access lvgl and underlying services.

Looking forward to seeing what people come up with on this, exciting times ahead!

[Elara6331]

Of course, as I said, native code is preferable. However, I think its implementation, if possible, would be A LOT more complex and while the performance of WASM would be lower, I think that it would be a lot easier to implement, which would increase maintainability in the future and decrease the amount of bugs both because of the easier to understand implementation, and because WASM is already a relatively mature standard.

Another issue is that I would prefer not having to load a header or anything else that is language-dependent at all as that would make it more difficult if not impossible to use languages like Rust and TinyGo to make apps for InfiniTime.

[lvlgl]

Should the dynamic apps be as capable as standalone apps are? It could be done with an external Android app which has a UI builder for PineTime and a Scratch-like interface to program actions/transitions between screens

[toastom]

I'm currently working on a tool that does involve the recompilation of InfiniTime, although it more easily allows the user to choose which apps and watchfaces they want to include in the build. The result will hopefully allow people to create a less bloated system to save memory for other larger or less memory-efficient apps and resources (tons of pictures and fonts?). It's still in the ridiculously early stages right now (not at all functional) and I'm no expert in bash or CMake, but the idea is to simply run a bash script and have it simply not include some user-defined apps in the build. It's called "infinitweaks" right now. Again, I just started this so it doesn't function but that's the idea.

[NeroBurner]

I think infinitweaks could be implemented purely in CMake configured by cmake-options. We can add compile defines and add the source files to the main target with target_sources()

[hrmckay]

With the Pebble watch you could (still can) download watch faces and apps. In my opinion this ability was what made the Pebble so popular, a lot of amazing faces and apps were created. There were four major ways to create a watch face or app.

1. Download the SDK, compiler, libraries and follow the documentation to build an app or watch face.
2. Use the Watch Face Generator, a web site which is now gone, to build a watch face by moving elements like the time, day of the week, etc into place on the watch face, add some personal elements like background pictures, and let the web site build the watch face and download it for you.
3. Use an Android app called Canvas to build a watch face similar to the way the Watch Face Generator did, or like the current Android apps like Facer or Watchmaker.
4. Use a web site called CloudPebble, which is now gone, to create a watch face or an app by writing C code in a web based editor (or you could upload the code) and then letting the web site compile it with the proper libraries and download it. Much easier then option 1.

Excellent documentation was supplied online.

[Avamander]

Side-note: This has been discussed a few times in the InfiniTime chat rooms. I do recommend searching for relevant keywords to find valuable info about. One potential approach not listed here are PIC functions copied dynamically into internal flash and then executed. Something akin to the PIC approach described in this blog post: https://mcuoneclipse.com/2021/06/05/position-independent-code-with-gcc-for-arm-cortex-m/

[Dheenhasty]

Hello,

For the WatchFace, a simple way to overcome the build is to have some map reference file in external storage.
It will limit the customisation possibility, but we can compile for example a watchfacedynamci who will place element based on a reference file located on a specifici directory on external storage (the pixel location, the font used and capapbilities can be extracted at the fly)

And to add some point to @hrmckay Did someone already check the way Peeble manage that in detail (i saw that ReebleOS is based on Freertos too .....), App and Watch seems to be based on a JS and a Prebuilt C binary maybe we don't need to reinvent the wheel ?

i don't know the impact it can have to performance and if it fits to the vision.

[JF002]

Taking inspiration from other successful project and products is probably a good idea, indeed!

However... according to Wikipedia, the Pebble watch had 128KB of RAM memory and allocated 24KB to the applications. It's twice as much as in the PineTime (it has only 64KB). And there's certainly far less than 24KB available right now in InfiniTime.
I think that the RAM scarcity will probably be one of the major difficulties for this feature.

[Dheenhasty]

I was more talking about the mecanism behind, seems like for example they consider watchface like an app.

But i clearly agree that memory could be quite troublesome here. Saw some post regarding trick to swap binary from rom to ram on freertos but it means that you need to reserve it cause it don't support dynamic linking.

I think we need an attack plan here or some guidance :)

[JF002]

I think we need an attack plan here or some guidance :)

I don't have any right now, unfortunately. I have never done such kind of development and I don't have a clear view on how to achieve it. If I was to start working on it, I guess I would read the blog post mentioned by Avamander and do some experiment from there.

[Dheenhasty]

I don't have any right now, unfortunately.

No problem, i just want to help here don't want to push you.
i wanted to see if we have already consider some option.

The fPIC (@Avamander Blog) way is a cool and smart way but very advanced implementation ..... sadly i can't help too much on that (I'm not a senior Dev). as i understand it's the same stuff i read on other blog and freertos doc, that by activate it you can reserve some memory at kernel time and put some code in it later by reusing the address.
I suppose that's want Peeble do with their 24k mem for app but they counter some memory usage by using framework already loaded in memory.

On my humble side, i will try to do WatchFace with Semi Dynamic content (like background/font and placement in )
Here's my reflexion (tell me if it's clearly dumb )
so one watchface compiled who will rely on set of external resources (in distinct directory)
you can switch from different faces on setting menu of the watchfaces itself (like the color change on others)
i was thinking about this kind of tree on external storage :
- watchface /
settings.txt <= current selected setting like watchfaces background and fonts/placement
myfirstwatchface.conf <= conf file who i can rely to load placement, fonts placement for myfirstwatchface
myfirstwatchface_bg.bin/png <= background file for myfirstwatchface (if needed)
myfirstwatchface_fonts.bin <= fonts file for myfirstwatchface (if needed)
mysecondwatchface.conf <= conf file who i can rely to load placement, fonts placement for myfirstwatchface
mysecondwatchface_bg.bin/png <= background file for myfirstwatchface (if needed)
mysecondwatchface_fonts.bin <= fonts file for myfirstwatchface (if needed)

for the settings/conf do we have prefered format or raw text is better to limit memory ?

and last question is it realistic to do that ? (maybe to simple and not enough optimized for infinitime)

[Avamander]

I suppose that's want Peeble do with their 24k mem for app but they counter some memory usage by using framework already loaded in memory.

Theoretically the PIC part could link against certain parts of the main firmware and call functions from there to reduce duplication. But that's way-way more advanced.

As discussed in the chatroom, the first few steps for a Proof of Concept would be:

1. Create a watchface that gets constructed or modified by a single function
2. Modify the linker script to place that function in a specific location
3. Call that function based on the (raw) offset
4. Follow the blogpost to turn that function PIC
5. Build and flash the function separate from the firmware
6. ???
7. Profit

This would get us very compact and very dynamic watchfaces, for starters.

The gotcha here is that it would take days of manhours from core maintainers... which are already few and rare and could be spent more efficiently.

[squidink7]

while the performance of WASM would be lower, I think that it would be a lot easier to implement

WASM would also make it easier to use languages other than C++ (Rust, V, TinyGo, etc.) to write InfiniTime apps, and also mean apps written for the PineTime could be used on potential future watches (maybe even using a risc-v chip like the BL60X) without recompilation. The largest difficulty would be fitting an entire WASM runtime on such a small chip, but from what I've seen it can (and has) been done before with projects like the aforementioned WAMR.

[everypizza1]

I want this for a wallpaper on my watch, without the hassle of making a fork, adding it to the code, compiling, etc.

[pipe01]

I've made a PR that implements a small stack-based bytecode language specially designed to make watchfaces.

[coffandro]

I may be naive here but couldnt a possible solution for the app development side also be to implement something like Lua scripting and then run the files from memory?
Or if thats takes too much storage implementing a "library" or function calls for something like C++(or Lua with some diy compiler to shorten it) to be able to make an app and run functions from the system to draw stuff and such.
(No idea if thats makes sense, its 1 am but i needed it out my head lol)

[jspkay]

Hello everyone!
Although I am not really an active member of this community, I love this project so much! Thank you for making such a useful device!

I come back in this thread because I believe having dynamically loaded apps on the OS is the key for having an explosive community!
Specifically, I am working with tensorflow and I stumbled upon a very interesting function: load_delegate.
Reading the documentation, you can see that its argument is the name of a shared object, which is loaded on the environment.

I thought it might be helpful! I'll try to take a look on it myself when I'll have more time!