GIGA Display Shield: How to capture touch events?

[KurtE]

I am trying to understand how to have an Arduino sketch be able to use the touch events generated
by the GIGA Display shield.

If I look at the overlay file, that is defined for the giga in zephyr:
D:\github\zephyr\boards\shields\giga_display_shield\boards\arduino_giga_r1_m7.overlay

It has:

&i2c4 {
	pinctrl-0 = <&i2c4_scl_pb6 &i2c4_sda_ph12>;
	pinctrl-names = "default";
	clock-frequency = <I2C_BITRATE_FAST>;
	status = "okay";

	gt911: gt911@5d {
		status = "okay";
		compatible = "goodix,gt911";
		reg = <0x5d>;
		alt-addr = <0x14>;
		reset-gpios = <&gpioi 2 GPIO_ACTIVE_LOW>;
		irq-gpios = <&gpioi 1 GPIO_ACTIVE_HIGH>;
	};
};

So it is using the gt911 input ... It has status of okay so it starts up. with IRQ on GPIO I1...
And it is probably using the code in zephyr\drivers\input\input_gt911.c

I found the zephyr example: samples\subsys\input\draw_touch_events, which I was curious if it would
build and run... So far did not build.

D:/zephyrproject/zephyr/drivers/display/display_stm32_ltdc.c:422:25: error: 'CONFIG_VIDEO_BUFFER_SMH_ATTRIBUTE' undeclared (first use in this function)
  422 |                         CONFIG_VIDEO_BUFFER_SMH_ATTRIBUTE,
      |                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
D:/zephyrproject/zephyr/drivers/display/display_stm32_ltdc.c:422:25: note: each undeclared identifier is reported only once for each function it appears in
[41/48] Linking C static library zephyr\kernel\libkernel.a

Thought I would try doing the same thing in a sketch, however I don't think this will work.

static void touch_event_callback(struct input_event *evt, void *user_data)
{
	if (evt->code == INPUT_ABS_X) {
		touch_point.x = evt->value;
	}
	if (evt->code == INPUT_ABS_Y) {
		touch_point.y = evt->value;
	}
	if (evt->code == INPUT_BTN_TOUCH) {
		touch_point.pressed = evt->value;
	}
	if (evt->sync) {
		k_sem_give(&sync);
	}
}
INPUT_CALLBACK_DEFINE(touch_dev, touch_event_callback, NULL);

As I don't believe the INPUT_CALLBACK_DEFINE will work in a sketch?

Wondering best way to handle this?

Potentially, maybe define some callback within the ArduinoCore-zephyr code space.
Like maybe in loader\fixups.c ?

Turn off the interrupt? And try to poll it?

Suggestions?

[KurtE]

Quick note:
The example sketch: ~/zephyrproject/zephyr/samples/subsys/input/input_dump
does build and run.
From the monitor window:

I: input event: dev=gt911@5d             type= 3 code=  0 value=373
I: input event: dev=gt911@5d             type= 3 code=  1 value=444
I: input event: dev=gt911@5d         SYN type= 1 code=330 value=1
I: input event: dev=gt911@5d             type= 3 code=  0 value=373
I: input event: dev=gt911@5d             type= 3 code=  1 value=444
I: input event: dev=gt911@5d         SYN type= 1 code=330 value=1
I: input event: dev=gt911@5d             type= 3 code=  0 value=373
I: input event: dev=gt911@5d             type= 3 code=  1 value=444
I: input event: dev=gt911@5d         SYN type= 1 code=330 value=1
I: input event: dev=gt911@5d             type= 3 code=  0 value=373
I: input event: dev=gt911@5d             type= 3 code=  1 value=444

Note: if you hold your finger down at all, you will see a lot of messages like:

I: input event: dev=gt911@5d             type= 3 code=  0 value=368
I: input event: dev=gt911@5d             type= 3 code=  1 value=424
I: input event: dev=gt911@5d         SYN type= 1 code=330 value=1
I: input event: dev=gt911@5d             type= 3 coW: Event dropped, queue full, not blocking in syswq.
de=  0 value=368
I: input event: dev=gt911@5d             type= 3 codW: Event dropped, queue full, not blocking in syswq.
W: Event dropped, queue full, not blocking in syswq.
e=  1 value=424W: Event dropped, queue full, not blocking in syswq.
W: Event dropped, queue full, not blocking in syswq.
W: Event dropped, queue full, not blocking in syswq.

[KurtE]

Edit: meant to add it to this issue instead of display library issue.

Quick notes: It was unclear to me how I might adapt the Arduino_GigaDisplayTouch library to zephyr. There is a PT911 object
defined in the overlay for the board GIGA under the Shield (GIGA Display Shield), that I know is defined and running.

As as soon as I touch the display I get lots of these messages in the monitor window:

[00:00:25.648,000] <wrn> input: Event dropped, queue full, not blocking in syswq.
[00:00:25.658,000] <wrn> input: Event dropped, queue full, not blocking in syswq.
[00:00:25.669,000] <wrn> input: Event dropped, queue full, not blocking in syswq.
[00:00:25.679,000] <wrn> input: Event dropped, queue full, not blocking in syswq.

I know that the PT911 class is looking for me to hook up a callback function. like some of the input samples show:
INPUT_CALLBACK_DEFINE(touch_dev, touch_event_callback, NULL);
But I know that adding this define to a library that gets loaded in our LLTEXT code won't work. Was not sure if there are any
Dynamic ways to setup the callback. So I asked up on Zephyr Discussion:
zephyrproject-rtos/zephyr#89415

The response I received:
zephyrproject-rtos/zephyr#89415 (comment)
From fabiobaltieri,

Can't be done at the moment, but it'd be very easy to a static callback that goes through a dynamic list. Or just implement the >whole thing in the subsystem and make it Kconfig optional, don't see a reason not to accept such a contribution.

My read of this is, that I would need to add the callback code into some place that is built into the loader, like maybe fixups.c
and maybe that code would receive the callbacks and save the touch information and maybe set a semaphore or the like
and then the library code could detect the semaphore and retrieve the last callback information.
Probably beyond my pay grade 😆, especially if your plans are to always use LVGL.

[KurtE]

Experiments update:

Trying directly on zephyr:
I generated a slightly modified version of one of the zephyr samples, sort of a cross between the two in
samples/subsys/input

I put a copy of it up at: https://github.com/KurtE/zephyr_test_sketches/tree/master/print_touch_events
Nothing much to it:

/*
 * Copyright (c) 2024 Antmicro <www.antmicro.com>
 *
 * SPDX-License-Identifier: Apache-2.0
 */

#include <zephyr/logging/log.h>
#include <zephyr/kernel.h>
#include <zephyr/device.h>
#include <zephyr/input/input.h>
#include <zephyr/drivers/display.h>
#include <zephyr/sys/util.h>

static const struct device *const touch_dev = DEVICE_DT_GET(DT_CHOSEN(zephyr_touch));

static struct k_sem sync;

static struct {
	size_t x;
	size_t y;
	bool pressed;
} touch_point;

static void touch_event_callback(struct input_event *evt, void *user_data)
{
	if (evt->code == INPUT_ABS_X) {
		touch_point.x = evt->value;
	}
	if (evt->code == INPUT_ABS_Y) {
		touch_point.y = evt->value;
	}
	if (evt->code == INPUT_BTN_TOUCH) {
		touch_point.pressed = evt->value;
	}
	if (evt->sync) {
		k_sem_give(&sync);
	}
}
INPUT_CALLBACK_DEFINE(touch_dev, touch_event_callback, NULL);

int main(void)
{

	if (!device_is_ready(touch_dev)) {
		//LOG_ERR("Device %s not found. Aborting sample.", touch_dev->name);
		return 0;
	}

	k_sem_init(&sync, 0, 1);

	while (1) {
		//k_msleep(REFRESH_RATE);
		k_sem_take(&sync, K_FOREVER);
		printk("TOUCH %s X, Y: (%d, %d)\n", touch_point.pressed ? "PRESS" : "RELEASE",
			touch_point.x, touch_point.y);
	}
	return 0;
}

Which when I touch the display I get messages like:

TOUCH PRESS X, Y: (466, 281)
TOUCH PRESS X, Y: (466, 281)
TOUCH PRESS X, Y: (466, 281)
TOUCH PRESS X, Y: (466, 282)
TOUCH RELEASE X, Y: (466, 282)

With a reasonable response, although there is a delay for the release...

Built on Ubuntu and Windows using west build...
Note: on windows west flash fails on WIndows: I created an issue on this:
zephyrproject-rtos/zephyr#89434
Can flash on windows using arduino dfu command:
"C:\Users\kurte\AppData\Local\Arduino15\packages\arduino\tools\dfu-util\0.10.0-arduino1/dfu-util" --device 0x2341:0x0366 -D "build\zephyr\zephyr.bin" -a0 --dfuse-address=0x08040000:leave

ArduinoCore-zephyr:

I added similar code to the above into fixups.c and added entry to llext_exports.c
In fixups, I extended the code under GIGA/VIDEO #if

Note: currently just an experiment and if actually works out, maybe should be conditional on if LVGL is defined for the build.

Currently:

#if defined(CONFIG_BOARD_ARDUINO_GIGA_R1) && defined(CONFIG_VIDEO)
#include <zephyr/kernel.h>
#include <zephyr/device.h>
#include <zephyr/drivers/clock_control.h>
#include <zephyr/logging/log.h>

#include <zephyr/input/input.h>

// experiment to try to capture touch screen events
typedef struct {
	size_t x;
	size_t y;
	bool pressed;
} touch_point_t;

touch_point_t last_touch_point;

static struct k_sem touch_event_sync;

bool getVideoTouchEvent(touch_point_t *tp, k_timeout_t timeout) {
	if (k_sem_take(&touch_event_sync, timeout) != 0) return false;
	// BUGBUG: should probably put stuff in to return only
	// data from whole event, but first see if anything works
	memcpy(tp, &last_touch_point, sizeof(touch_point_t));
	return true;
}


static void touch_event_callback(struct input_event *evt, void *user_data)
{
    printk("touch_event_callback(%p %p): %p %u %u %u %d\n", evt, user_data,
            evt->dev, evt->sync, evt->type, evt->code, evt->value);
	if (evt->code == INPUT_ABS_X) {
		last_touch_point.x = evt->value;
	}
	if (evt->code == INPUT_ABS_Y) {
		last_touch_point.y = evt->value;
	}
	if (evt->code == INPUT_BTN_TOUCH) {
		last_touch_point.pressed = evt->value;
	}
	if (evt->sync) {
		k_sem_give(&touch_event_sync);
	}
}
static const struct device *const touch_dev = DEVICE_DT_GET(DT_CHOSEN(zephyr_touch));
INPUT_CALLBACK_DEFINE(touch_dev, touch_event_callback, NULL);


int camera_ext_clock_enable(void)
{
	int ret;
	uint32_t rate;

	// Hack in init semaphore for touch events
	k_sem_init(&touch_event_sync, 0, 1);


	const struct device *cam_ext_clk_dev = DEVICE_DT_GET(DT_NODELABEL(pwmclock));

	if (!device_is_ready(cam_ext_clk_dev)) {
		return -ENODEV;
	}

	ret = clock_control_on(cam_ext_clk_dev, (clock_control_subsys_t)0);
	if (ret < 0) {
		return ret;
	}

	ret = clock_control_get_rate(cam_ext_clk_dev, (clock_control_subsys_t)0, &rate);
	if (ret < 0) {
		return ret;
	}

	return 0;
}

SYS_INIT(camera_ext_clock_enable, POST_KERNEL, CONFIG_CLOCK_CONTROL_PWM_INIT_PRIORITY);

#endif

Added the callback, added semaphore, and had it init in the camera_ext_clock_enable.

In the exports I added the callback in the section:

#if defined(CONFIG_VIDEO)
FORCE_EXPORT_SYM(video_buffer_aligned_alloc);
FORCE_EXPORT_SYM(video_buffer_alloc);
FORCE_EXPORT_SYM(video_buffer_release);
FORCE_EXPORT_SYM(getVideoTouchEvent)
#endif

So far really really bad performance/latency!!!
The first touch sort of goes through then get lots of syswq full messages, then delay then
some events processed, then full then messages... And these happen long after my
finger was removed...

[00:00:01.605,000] <inf> usb_cdc_acm: Device configured
touch_event_callback(0x2400e1f4 0): 0x8062e9c 0 3 0 303
touch_event_callback(0x2400e1f4 0): 0x8062e9c 0 3 1 203
touch_event_callback(0x2400e1f4 0): 0x8062e9c 1 1 330 1
touch_event_callback(0x2400e1f4 0): 0x8062e9c 0 3 0 303
X = 303 Y= 203
[00:00:09.887,000] <wrn> input: Event dropped, queue full, not blocking in syswq.
[00:00:09.899,000] <wrn> input: Event dropped, queue full, not blocking in syswq.
[00:00:09.909,000] <wrn> input: Event dropped, queue full, not blocking in syswq.
[00:00:09.919,000] <wrn> input: Event dropped, queue full, not blocking in syswq.
[00:00:09.930,000] <wrn> input: Event dropped, queue full, not blocking in syswq.
[00:00:09.940,000] <wrn> input: Event dropped, queue full, not blocking in syswq.
[00:00:09.950,000] <wrn> input: Event dropped, queue full, not blocking in syswq.
[00:00:09.961,000] <wrn> input: Event dropped, queue full, not blocking in syswq.
[00:00:09.971,000] <wrn> input: Event dropped, queue full, not blocking in syswq.
[00:00:09.982,000] <wrn> input: Event dropped, queue full, not blocking in syswq.
[00:00:09.992,000] <wrn> input: Event dropped, queue full, not blocking in syswq.
[00:00:10.002,000] <wrn> input: Event dropped, queue full, not blocking in syswq.
[00:00:10.013,000] <wrn> input: Event dropped, queue full, not blocking in syswq.
touch_event_callback(0x2400e1f4 0): 0x8062e9c 0 3 1 203
touch_event_callback(0x2400e1f4 0): 0x8062e9c 1 1 330 1
touch_event_callback(0x2400e1f4 0): 0x8062e9c 0 3 0 303
X = 303 Y= 203
touch_event_callback(0x2400e1f4 0): 0x8062e9c 0 3 1 203
touch_event_callback(0x2400e1f4 0): 0x8062e9c 1 1 330 1
touch_event_callback(0x2400e1f4 0): 0x8062e9c 0 3 0 303
X = 303 Y= 203
touch_event_callback(0x2400e1f4 0): 0x8062e9c 0 3 1 203
touch_event_callback(0x2400e1f4 0): 0x8062e9c 1 1 330 1
touch_event_callback(0x2400e1f4 0): 0x8062e9c 0 3 0 303
X = 303 Y= 203
[00:00:21.087,000] <wrn> input: Event dropped, queue full, not blocking in syswq.
[00:00:21.097,000] <wrn> input: Event dropped, queue full, not blocking in syswq.
[00:00:21.107,000] <wrn> input: Event dropped, queue full, not blocking in syswq.
...

Edit: test sketch with the touch code in it:
https://github.com/KurtE/Arduino_GIGA-stuff/tree/main/sketches/zephyr/zephyr_GIGA_shield_touchpaint

Not sure if something blocking? threads? priorities? ...

[KurtE]

@iabdalkader @pillo79 @facchinm @mjs513 (and all)

Quick update:
Current potential theory: The Display output code interferes with the input thread: zephyr/subsys/input/input.c
We get a lot of Event dropped message from the same file (input_report).

One issue I had with my code was to have printk calls in the callback function mentioned in previous mentioned so I removed it.

It still was still not working right.

So I created another test sketch: up on my github.... zephyr_GIGA_shield_touch_only, where I started from the
touchpaint sketch, but removed everything associated with the display (did not start it, ...)
It appears like it is properly getting the callbacks from the PT911 input object....

So wondering if the display update code is running long periods of time and that thread is being starved?

Wondered if maybe increasing the size of the QUEUE might help, so tried:

CONFIG_INPUT_QUEUE_MAX_MSGS=10
Did not help:
Tried a few different priorities (2 4 -1) for this thread:
CONFIG_INPUT_THREAD_PRIORITY=-1
Did not help.

Wondering what else to try? Like maybe have PT911 not be interrupt? in the config for GIGA under the shield it has
CONFIG_INPUT_GT911_INTERRUPT=y
Maybe try NO, I think it then uses Timer interrupt?

Side note: I was curious about threads in sketches. So I hacked up our test sketch: GIGA_Display_first:

I added a thread to the sketch, that blinks a pin and every n
times blinks LED_BUILTIN... The interesting parts:

K_THREAD_STACK_DEFINE(blink_pins_thread_stack, STACKSIZE);
static struct k_thread blink_pins_thread_data;

void blink_pins_thread(void *dummy1, void *dummy2, void *dummy3)
{
	ARG_UNUSED(dummy1);
	ARG_UNUSED(dummy2);
	ARG_UNUSED(dummy3);

	Serial.print("thread_a: thread started \n");

	while (1)
	{
    static uint8_t pin_state = 0;
    static uint8_t loop_count = 0;
    pin_state ^=1;
    digitalWrite(BLINK_PIN, pin_state);
    loop_count++;
    if ((loop_count & (LED_BLINK_COUNT - 1)) == 0) digitalWrite(LED_BUILTIN, (loop_count  & LED_BLINK_COUNT)? HIGH : LOW);
  	k_msleep(SLEEPTIME);
  }
}
...
void setup() {
  while (!Serial && millis() < 5000)
    ;
  Serial.begin(115200);

  if (!display.begin()) {
    Serial.println("Failed to start display");
  };

  Serial.println("Display configured!!");
  pinMode(BLINK_PIN, OUTPUT);
  pinMode(LED_BUILTIN, OUTPUT);

  // Start a thread:
    k_thread_create(&blink_pins_thread_data, blink_pins_thread_stack,
      K_THREAD_STACK_SIZEOF(blink_pins_thread_stack),
      blink_pins_thread, NULL, NULL, NULL,
      PRIORITY, 0, K_FOREVER);
  k_thread_name_set(&blink_pins_thread_data, "blink pins");

	k_thread_start(&blink_pins_thread_data);



  void* FB =  display.getFrameBuffer();
  if (FB == nullptr){
...
  delay(2000);
  display.startFrameBuffering();
  fillScreen(RGB565_BLACK);
  display.endFrameBuffering();
}

void loop() {
}

And the sketch runs and when it reaches the end of main, the thread never runs again.

However if I change loop to:

void loop() {
  // put your main code here, to run repeatedly:
  delay(500);
}

The issue is in cores\arduino\main.cpp:

int main(void) {
#if (DT_NODE_HAS_PROP(DT_PATH(zephyr_user), cdc_acm) && CONFIG_USB_CDC_ACM)
  Serial.begin(115200);
#endif

  initVariant();

#ifdef CONFIG_MULTITHREADING
  start_static_threads();
#endif

  setup();

  for (;;) {
    loop();
  #if 0 //(DT_NODE_HAS_PROP(DT_PATH(zephyr_user), cdc_acm) && CONFIG_USB_CDC_ACM) || DT_NODE_HAS_PROP(DT_PATH(zephyr_user), serials)
    if (arduino::serialEventRun) arduino::serialEventRun();
  #endif
  }

  return 0;
}

There is nothing in the for loop calling loop() that yields or delays... So this logical thread never
yields back to the task manager.

[KurtE]

@ALL....

I am maybe having some better luck right now with touch.
I pushed my current WIP branch up:
https://github.com/KurtE/ArduinoCore-zephyr/commits/pr130/

It is based on Pull request #130
And I had edited in the changes in #117
As the zephyr changes for it were merged and the zephyr fork/branch refered to by 130 has it in it.

I added some comments on 117 as some things were not working, like nothing showed up on the screen.
Found that the backlight was not turned on... more on that PR...

I added a callback and the like for the touch controller into loader/fixups.c

#include <zephyr/input/input.h>

// experiment to try to capture touch screen events
typedef struct {
	int32_t x;
	int32_t y;
	int32_t pressed;
} touch_point_t;

touch_point_t last_touch_point;

static struct k_sem touch_event_sync;

bool getVideoTouchEvent(touch_point_t *tp, k_timeout_t timeout) {
	if (k_sem_take(&touch_event_sync, timeout) != 0) return false;
	// BUGBUG: should probably put stuff in to return only
	// data from whole event, but first see if anything works
	memcpy(tp, &last_touch_point, sizeof(touch_point_t));
	return true;
}


void touch_event_callback(struct input_event *evt, void *user_data)
{
    //printk("touch_event_callback(%p %p): %p %u %u %u %d\n", evt, user_data,
    //        evt->dev, evt->sync, evt->type, evt->code, evt->value);
	if (evt->code == INPUT_ABS_X) {
		last_touch_point.x = evt->value;
	}
	if (evt->code == INPUT_ABS_Y) {
		last_touch_point.y = evt->value;
	}
	if (evt->code == INPUT_BTN_TOUCH) {
		last_touch_point.pressed = evt->value;
	}
	if (evt->sync) {
		k_sem_give(&touch_event_sync);
	}
}
static const struct device *const touch_dev = DEVICE_DT_GET(DT_CHOSEN(zephyr_touch));
INPUT_CALLBACK_DEFINE(touch_dev, touch_event_callback, NULL);

I added an export:

#if defined(CONFIG_BOARD_ARDUINO_GIGA_R1) && defined(CONFIG_VIDEO)
FORCE_EXPORT_SYM(getVideoTouchEvent)
#endif

I made sure that loop() had a delay(1) in it, as to let other threads have a chance to run.

I now have the touch screen sketch running... Up on Arduino-GIGA-stuff github...

On my Windows machine:

on my Ubuntu machine:

But I am finding that it appears like the orientation of the touch screen versus the display is different between
the two giga/gigaShield setups.

That is on my Windows machine if I click near the bottom left area (red square)
the coordinates returned are something like 59,440 but on Ubuntu it is like 16,10
And the opposite corner (upper right): Windows 722,60, Ubuntu 459,783

So wondering where the difference is? And is there a way to programatically know which orientation the
screen was installed?

[mjs513]

@KurtE

After a torturous couple of days was able to synch up with all the changes for ble-display-touch again so am not getting the weird bus faults.

Ran your latest set of changes for touch and am now seeing what you previously shown

as for coordinates.

0, 0 for touch is the upper left corner but for the boxes seems that it is in the lower left corner?

[KurtE]

Thanks @mjs513 - As I mentioned, I found the coordinates of touch versus the pixels different between my two boards...
I need to look more at it to see if this also happens with the MBED version or not.

[mjs513]

@KurtE
Was able to get it running on my 2nd giga and am seeing the same coordinate of touch and graphics on my 1st giga. Will look at the graphics code see deltas.

Not sure I mentioned this but I changed the way you defined shared buffer from:

buffer = (uint16_t*)shared_multi_heap_aligned_alloc(SMH_REG_ATTR_EXTERNAL, 16, (this->width() * this-> height() * sizeof(uint16_t)));

to

    void* ptrFB = this->display->getFrameBuffer();
    if (ptrFB == nullptr){
      Serial.println("Memory not allocated successfully." );
      while(1){}
    }
    // Cast the void pointer to an int pointer to use it
    buffer = static_cast<uint16_t*>(ptrFB);

[mjs513]

Will look at the graphics code see deltas.

Simple solution. Delete fillscreen and fillrect functions in Arduino_GigaDisplay_GFX and the adafruit gfx lib handle it. The current fillscreen and fillrect functions are setup for rotation as you already know. I tried it and looks like with that change the coordinates should align with the touch coordinates. Fingers crossed.... :)

Actually probably can just deleted fillRect function

[KurtE]

Thanks @mjs513 - made the same changes as you mentioned in the touch paint sketch... Will push it up soon...

@iabdalkader @facchinm @pillo79

Keep wondering about usage patterns and the how or if to use the to be two buffers.
CONFIG_STM32_LTDC_FB_SMH_ATTRIBUTE=2
With the current code, I believe, that if I pass in a full size buffer to the display write function, it will set it as the new (pending)
buffer. If for example compute the address of the 2nd buffer and then alternate passing in the first and second one,
Each time I do this, I will be starting off with the FB containing the contents from two updates before. i.e. the contents of
what was in it the previous time this buffer was used... So either I need to copy the contents of the other buffer into it
and/or the code needs to regenerate the whole frame...

Note: if you just use the one buffer like mentioned in:

    void* ptrFB = this->display->getFrameBuffer();
    if (ptrFB == nullptr){
      Serial.println("Memory not allocated successfully." );
      while(1){}
    }
    // Cast the void pointer to an int pointer to use it
    buffer = static_cast<uint16_t*>(ptrFB);

And do something like:

  rgb.on(128, 0, 0);
  display.fillScreen(GC9A01A_RED);
  rgb.on(0, 128, 0);  //turn on blue pixel
  display.fillScreen(GC9A01A_GREEN);
  rgb.on(0, 0, 128);  //turn on blue pixel
  display.fillScreen(GC9A01A_BLUE);
  rgb.on(128, 128, 128);
  display.fillScreen(GC9A01A_WHITE);
  rgb.off();  //turn off all pixels
  display.fillScreen(GC9A01A_BLACK);
  delay(500);

You will get a completely garbled screen...

Note: If I have the code alternate between the two simply, the above code will probably also give you some garbled stuff
and/or missed complete updates.

Could add function to say wait until the new pending buffer is the current buffer, before continuing...
Like waiting until: getFrameBuffer() retuns the address of the last buffer I passed to it...

But again wondering what you all believe is the proper usage pattern. and/or should there be some way for a sketch
to dictate this as an option...

[iabdalkader]

CONFIG_STM32_LTDC_FB_SMH_ATTRIBUTE=2

I don't follow, but this is the shared memory attribute not the number of buffers.