Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QR code application #181

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open

QR code application #181

wants to merge 12 commits into from

Conversation

petterhs
Copy link
Contributor

@petterhs petterhs commented Jan 25, 2021

QR code viewer

With this application you can send text from a companion app (e.g. URL's, ticket's) and view it as a QR code on the PineTime.

Increases image size from 399kB to 418kB, mainly from the QR library. The app does not store the QR codes but is instead encoding on the fly when needed. This way companion apps does not need any QR encoders. The library supports all 40 standard QR code sized and all 4 error correction levels but we can possibly limit it to reduce image size.

The app can store 4 text strings that can be encoded to QR and displayed. (Can be increased in the future)

Default strings now: (Let me know if you got other suggestions)

Displayed like this:
IMG_20210403_211429

Example QR code:
IMG_20210403_211453

Testing: (WARNING: Don't test on sealed device unless you know what you are doing. Can crash and cause bricked watch)

Not implemented in any of the companion apps yet but can be tested with nRFConnect.

  • Go to QR app on page 3 of applications on the watch
  • Press screen once and the initial QR-code should be visible(should be "https://github.com/JF002/Pinetime/" when scanned)
  • Go to the list of services on nRFConnect when you are connected to the watch
  • Press service with UUID: 00040000-78fc-.................
  • Press the icon with arrow pointing upwards on the characteristic with UUID: 00040001-78fc-.........
  • Change value-type to TEXT
  • Type your URL or text you want to encode to QR in this format:
  • id|name|text (example: 0|Github|https://github.com/JF002/InfiniTime/)
    • id: position in the list of qr codes (number from 1 to 4)
    • name: displayed name in app
    • text: the text you want to encode to a QR code.
    • the three field are separated by "|"
  • Press send

Things I need help with

  • Review code for possible memory leaks.
  • Placing it in a suitable spot in the application list

@petterhs
Copy link
Contributor Author

petterhs commented Feb 9, 2021

As suspected, there are still memory issues here as two more tried this PR (from the chat) experienced freezes that needed watch reboot

Copy link
Contributor

@MFAshby MFAshby left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a fun little app 😄

I had a quick play with this after rebasing the change to latest develop. I noticed a couple of things:

  • sending any malformatted request causes a crash (so sending something like hello! instead of 0|hello|world)
  • sending item 4 doesn't extend the menu.
  • the menu background gets brighter every time I open a QR then leave it again (took a video but GitHub complains it's too large, will find another way to share it if necessary)

I Am Not A C++ developer, but I had a read through the code for educational purposes and I think I spotted one bug, see inline comment about qrId variable comparison.

I am also not a UX designer, but I had a thought about this: I don't think it makes too much sense to have a dedicated QR option in the menu. I think instead maybe:

  1. Include the project link QR codes in the settings/about screen - currently the GitHub project URL is there as text but it would be way cooler have the QR code as well 😎
  2. When a QR is received over the bluetooth service, generate and display it immediately on the screen instead of assigning it to a slot in the menu. I can't think of a reason to save it as a menu entry (to be fair, I also can't think of a really good reason to send a QR code to the watch via bluetooth, but this doesn't mean we shouldn't do it 😸 )

@Izaic
Copy link

Izaic commented Aug 14, 2021

Bump?

@geekbozu geekbozu marked this pull request as draft October 3, 2021 03:00
@geekbozu geekbozu added the needs more work This PR needs more work label Oct 3, 2021
@uniformbuffer
Copy link

uniformbuffer commented Oct 8, 2021

I don't know if it could be actually integrated, but looking this PR make me think about a related feature that could be very useful: in a lot of EU countries to enter in public building (and to do a lot of other things) is required to show the DCG Certificate (also known as Green Pass), it certificate the vaccination against COVID 19. This certification is represented as a QR code and verified from the requesting entity using a Android/iOS specialized app. Those applications simply translate the QR code to the string that create it and send the string to a server to query the status of the person.

So using this watchface it would be possible, in theory, to do the reverse process. It is just needed to get the string that have generated the green pass and make this watchface to translate it again as QR code. It should be the same as the one in the actual certificate (that it is shipped as pdf with all the information or as png image with just the qr code), so in theory it should validate correctly if requested from the mentioned Android/iOS applications.

It would be cool to have the possibility to show it from the Pinetime to save time.

Still, since the certificate contains some sensitive informations, it would be required to have some kind of security for bluetooth communication before proceeding for such feature.

petterhs and others added 5 commits November 9, 2021 23:43
add qrcode library

add 29x29 QR viewer

viewer support for all QR sizes

change to c library for qrcodegeneration

add ble QrService

Add Qr app to application screen 3

improve Qr::drawQr

update QR code automatically

center QR code on screen

add possibility for 4 different qr codes
@petterhs
Copy link
Contributor Author

petterhs commented Nov 9, 2021

Thanks for the review and comments!! I'm sorry I have not been active here, and not kept the PR up to date. I have worked a little on it the last days though:

  • rebased onto develop branch
  • changed UUID to not interfere with the Motion UUID
  • fixed suggestions above
  • clang-formated files
  • Fix some edgecases when sending data not on the right format. When you send with the wrong format it overrides the nr.1 slot now.
  • debugging of new bugs....

The PR is quite a mess now with the rebases and force-pushes, I will try to keep it cleaner from now so it is easier to review.

Status:

  • It is still not stable, having some inconstant bugs still (Testing at your own risk)
  • New display bug in bottom right corner when displaying codes
  • Temporarily hijacked the Metronome position in the app menu instead of creating a new page. (If we want a new page at all)
  • Should the qrencode library be a git submodule instead?
  • Needs to add a doc file and add the service UUID to BLE docs

I think the app could be useful to me, but i realize that not everyone agrees, so we might just want to close the PR and I can maintain it on a separate branch myself. Up to the rest of the community 😃

QR codes i want to have on my watch:

  • Boarding Pass when traveling
  • Matrix account info (Or other chat apps/social media)
  • Spotify Group session??
  • Links to personal website/portfolio?
  • BTC Lightning node (Scan to connect to my node)

@Avamander
Copy link
Collaborator

Avamander commented Nov 10, 2021

QR codes i want to have on my watch:

GPG pubkey and WiFi password for me :)

I wouldn't increase the scope of this PR and some kind of "use preset and display" UI would be nice to add in the future. Without the need of a companion.

@Avamander
Copy link
Collaborator

Needs to add a doc file and add the service UUID to BLE docs

I think a barebones one for now is fine. It's much easier to update and improve docs than it is to constantly keep a PR from bitrot when it hasn't been merged. I really want to make it less effort for contributors.

@petterhs
Copy link
Contributor Author

I think I have figured out the crashes. Its because the application uses to much memory when I try to support all QR sizes. (up to version 40, 177x177 modules, max 2953 bytes, which is probably not readable on the 240x240 screen anyway)

The Qr.cpp screen creates two buffers of size dependent on max QR version. with version 40 that is 3918 bytes per buffer which is alot.. So the app crashes if there is not enough memory for these buffers, so dependent on other memory usage and that is why it was unstable.

From nRFConnect i'm only able to send 162 bytes to the watch (not sure where this limit comes from, and if we can increase it). So it makes sense to reduce the max QR version down to just support the 162 bytes, at least for now.

The choice is then what error correction to chose:
image

ECC Level Max QR version Module size Pixels per module max size(bytes) buffer sizes(bytes)
L 8 49x49 4x4 192 2*302
M 9 53x53 4x4 180 2*353
Q 11 61x61 3x3 177 2*467
H 13 69x69 3x3 177 2*597

All of these are easily readable by my phone. I have not been able to test with more than 162 bytes, so not sure when it gets problematic, if we want to support bigger QR codes in the future.

So L and Q have the same pixel sizes as M and H so the choice is probably either M or H.

The buffers:
M : 2x353 bytes
H: 2x597 bytes

Buffers are deleted when exiting app so these should both be small enough to always work.

For now I choose QR version 9 with error correction Medium. As i think the extra pixel per module is worth more than having correction level HIGH.

Another potential problem is the QrService that actually store the text that gets encoded. If we send 4 max size codes to the watch we take up 4*162 = 648 bytes of RAM that can potentially be needed by other applications. This is the same for other services that we send data to; Music, navigation, notifications, etc. Hopefully we can move this to external memory soon. (#321 #750)

@petterhs
Copy link
Contributor Author

From nRFConnect i'm only able to send 162 bytes to the watch (not sure where this limit comes from, and if we can increase it). So it makes sense to reduce the max QR version down to just support the 162 bytes, at least for now.

This was just a MTU thing, with nRFConnect i'm able to increase it to 517 bytes.

I also tested with the QR Version 40, 177*177 modules, with only one pixel per qr module.. Was not able to scan with my phone.. but version 25 was.

Version 25 is 117*177 so its the biggest version we can fit into our screen (240x240) with 2 pixels per module. Interestingly it also supports 535 bytes with error correction HIGH. (Max 1,273 bytes with error correction low)

Then if we want to support the biggest, actually readable QR code version it is probably 25. with buffer size 2*1713B. Seems to work fine now, RAM vice, but maybe not in the future.. Not sure what to choose.

IMG_20211113_121905_edit_9333989304825

Of course the app generates the smallest QR code needed, so all codes will not look like this

area.x2 = qrModuleSize * (x + 1) + offset - 1;
area.y2 = qrModuleSize * (y + 1) + offset - 1;
lvgl.SetFullRefresh(Components::LittleVgl::FullRefreshDirections::None);
lvgl.FlushDisplay(&area, colorBuffer);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this OK even if colorBuffer is larger than area?? It works but wonder if I overwrite something on the SPI

@petterhs petterhs marked this pull request as ready for review November 13, 2021 15:54
@Avamander
Copy link
Collaborator

Recent changes have fixed the qualms I had with the code. Now it would be great to test this out further.

Qr.h was missing in CMakeLists.txt
@Avamander Avamander added needs testing Needs testing on a physical device and removed needs more work This PR needs more work labels Dec 6, 2021
FintasticMan added a commit to FintasticMan/InfiniTime that referenced this pull request Dec 13, 2021
Squashed commit of the following:

commit 326dfc5
Author: petter <[email protected]>
Date:   Sat Nov 13 16:51:28 2021 +0100

    move qrgenerator library to a git submodule

commit 1eca515
Author: petter <[email protected]>
Date:   Sat Nov 13 16:26:19 2021 +0100

    fix visual bug in bottom right corner

commit da3cc29
Author: petter <[email protected]>
Date:   Fri Nov 12 17:40:42 2021 +0100

    update external qr library

commit 58488dc
Author: petter <[email protected]>
Date:   Fri Nov 12 15:47:29 2021 +0100

    reduce max QR version to 9 with error correction MEDIUM

commit a3b1519
Author: petter <[email protected]>
Date:   Tue Nov 9 23:43:45 2021 +0100

    update to newer way to assign BLE UUIDs, fix suggested changes from PR

commit d389f22
Author: petter <[email protected]>
Date:   Tue Nov 9 14:47:30 2021 +0100

    change service UUID to not be the same as MotionService

commit 562c87d
Author: petter <[email protected]>
Date:   Sun Nov 7 13:52:35 2021 +0100

    clang-format files

commit 69538bd
Author: Petter Hoem Sletsjøe <[email protected]>
Date:   Wed Nov 3 08:45:52 2021 +0100

    rebase changes

commit 29be2d9
Author: petter <[email protected]>
Date:   Sun Jan 17 11:25:45 2021 +0100

    add QR viewer application with library

    add qrcode library

    add 29x29 QR viewer

    viewer support for all QR sizes

    change to c library for qrcodegeneration

    add ble QrService

    Add Qr app to application screen 3

    improve Qr::drawQr

    update QR code automatically

    center QR code on screen

    add possibility for 4 different qr codes
FintasticMan added a commit to FintasticMan/InfiniTime that referenced this pull request Dec 13, 2021
Squashed commit of the following:

commit 326dfc5
Author: petter <[email protected]>
Date:   Sat Nov 13 16:51:28 2021 +0100

    move qrgenerator library to a git submodule

commit 1eca515
Author: petter <[email protected]>
Date:   Sat Nov 13 16:26:19 2021 +0100

    fix visual bug in bottom right corner

commit da3cc29
Author: petter <[email protected]>
Date:   Fri Nov 12 17:40:42 2021 +0100

    update external qr library

commit 58488dc
Author: petter <[email protected]>
Date:   Fri Nov 12 15:47:29 2021 +0100

    reduce max QR version to 9 with error correction MEDIUM

commit a3b1519
Author: petter <[email protected]>
Date:   Tue Nov 9 23:43:45 2021 +0100

    update to newer way to assign BLE UUIDs, fix suggested changes from PR

commit d389f22
Author: petter <[email protected]>
Date:   Tue Nov 9 14:47:30 2021 +0100

    change service UUID to not be the same as MotionService

commit 562c87d
Author: petter <[email protected]>
Date:   Sun Nov 7 13:52:35 2021 +0100

    clang-format files

commit 69538bd
Author: Petter Hoem Sletsjøe <[email protected]>
Date:   Wed Nov 3 08:45:52 2021 +0100

    rebase changes

commit 29be2d9
Author: petter <[email protected]>
Date:   Sun Jan 17 11:25:45 2021 +0100

    add QR viewer application with library

    add qrcode library

    add 29x29 QR viewer

    viewer support for all QR sizes

    change to c library for qrcodegeneration

    add ble QrService

    Add Qr app to application screen 3

    improve Qr::drawQr

    update QR code automatically

    center QR code on screen

    add possibility for 4 different qr codes
@JF002
Copy link
Collaborator

JF002 commented Dec 30, 2021

@petterhs Thanks for this PR! I'm really sorry I didn't review it sooner. It looks quite nice and useful, especially in those time of "covid pass".
However, from my measurement, it increases the size of the binary of 15KB, which, while not critical, is quite big knowing that the FLASH memory is already quite packed.

I'll delay this PR for some time, until we have a better view on how we can reduce the memory usage and keep it under control.

@petterhs
Copy link
Contributor Author

@JF002 Completely understandable :) If there is a usecase for smaller qr-codes it is possible to reduce the memory usage by removing the ability to create large codes. I will try to keep this up to date for people that want to use it in the meantime

@Danimations
Copy link

Danimations commented Apr 30, 2022

Nice concept. I'd like to share another use case. I would use mine to display QR codes from e-tickets to various events, bought from vendors like Eventbrite. It would definitely be preferable than digging through emails on a phone to enter an event with. Thanks for this initiative!

@Riksu9000 Riksu9000 added the new app This thread is about a new app label May 2, 2022
@devnoname120
Copy link

I'd love to use this to quickly share my contact card and my Wi-Fi profile.

@Riksu9000
Copy link
Contributor

Displaying QR codes seems like a neat idea, but perhaps there's a better way to implement it. As this feature is already dependent on the companion app to enter the data, the conversion to a qr code could also be done on the companion, where memory is not so limited. I was reminded about this after seeing #1237, which could also be used for this, so simply send the codes as images to the external flash.

@Eloitor
Copy link

Eloitor commented Aug 18, 2022

Normal barcodes could also be useful, to store multiple customer cards.

@yannickulrich yannickulrich mentioned this pull request Oct 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs testing Needs testing on a physical device new app This thread is about a new app
Projects
None yet
Development

Successfully merging this pull request may close these issues.