Latest Development Version

This release contains the latest version of Photon Vision based on the code on the master branch. This code is not as well-tested as the latest stable release! Use at your own risk.

v2024.3.1

First major release in way too long! Includes critical bug fixes for Orange Pi object detection crashing/showing no stream/heartbeat freezing, and a new base Orange Pi image. The new base image boots faster without networking and includes other improvements.

Important

Teams with Orange Pis will need to reflash using the v2024.3.1 image. Make sure to export your settings (and keep a backup somewhere safe) prior to doing this. If you've got a spare SD card, try to flash it instead so you've got a known-good hot-spare.

Important

The Orange Pi NetworkManager interface name changed to "netplan-zz-all-eth" -- make sure to have this selected in the dropdown

Note

Teams not using Orange Pi's probably don't need to update, and base images have not changed so a reflash is not required.

Major changes

• Run resize in CPU to avoid RGA-crash/lockup by @mcm001 in #1287
• Bump orange pi images to v1.33 by @mcm001 in #1276

Other changes

• Bump WPILib to 2024.3.2 by @mcm001 in #1283
• Opencv cal: CALIB_USE_LU and use camera focal length guess by @mcm001 in #1268
• Disable camera orientation option when camera is calibrated by @philltran in #1277
• Add proper exposure range for OV2311 by @gautvm in #1282
• Set default pipeline idx in PipelineManager constructor by @mcm001 in #1286
• Free native resources in apriltag pipelines by @mcm001 in #1272
• Release letterboxed frame in rknn pipe by @mcm001 in #1289
• Recreate user pipeline on type change by @mcm001 in #1290
• Fix wpiformat by @mcm001 in #1291

New Contributors

• @philltran made their first contribution in #1277

Full Changelog: v2024.2.9...v2024.3.1

v2024.2.9

Fixes mrcal calibration bug with non-square calibration targets, and implements a incrementally expanding optimization approach to match the behavior of mrcal-calibrate-cameras. We solve first for chessboard pose only, then board pose and pinhole coefficients, then board pose and full camera intrinsics with outlier rejection (but not board warp), and then finally everything.

What's Changed

• Create smoketest mode by @mcm001 in #1264
• Pipe cscore logs through photonvision by @mcm001 in #1260
• Bump mrcal to include solver fixes by @mcm001 in #1265

Full Changelog: v2024.2.8...v2024.2.9

v2024.2.8

Addresses a bug in 2024.2.7 that prevented configs for CSI cameras being saved.

What's Changed

• Fix CSI camera matching by @mcm001 in #1258

Full Changelog: v2024.2.7...v2024.2.8

v2024.2.5

What's Changed

• Check empty mean errors in calibration card by @mcm001 in #1229 . This manifests as a weird empty calibration table with the wrong resolutions.
• fix latency calculation by @gerth2 in #1227
• Fix cpp and java photoncamera names in "Aim at Target" example by @DeltaDizzy in #1230
• Upload docs to VPS via SFTP by @mcm001 in #1235

Full Changelog: v2024.2.4...v2024.2.5

v2024.2.4

What's Changed

Includes a fix for Spinel OV9281s that report duplicate video modes/calibration happening at the wrong resolution. Please create issues/drop by Discord if you still have issues!

• Fix for Spinel camera video mode selection bug (Fixes #1219) by @crschardt in #1221
• fix chessboard gen by @gerth2 in #1217
• Store the last pose on update by @ArchdukeTim in #1207

New Contributors

• @ArchdukeTim made their first contribution in #1207

Full Changelog: v2024.2.3...v2024.2.4

v2024.2.3

More small bugfixes, including reduced bandwidth on initial connection and during calibration to address instability in bandwidth limited setups.

What's Changed

• Add v4l utils to installation script by @gautvm in #1201
• Dont send targets or cal images to UI by @mcm001 in #1200
• Add support for different RKNN YOLO models in the backend by @james20902 in #1205
• Python tweaks by @gerth2 in #1211
• Fix inverted condition in camera calibration table by @mcm001 in #1214
• Add NPU usage to metrics on supported platforms by @mcm001 in #1215

New Contributors

• @gautvm made their first contribution in #1201
• @james20902 made their first contribution in #1205

Full Changelog: v2024.2.2...v2024.2.3

v2024.2.2

What's Changed

Minor QOL changes and object detection bugfixes!

• Sort object detection results and expose controls in UI by @mcm001 in #1173
• Fixed cpp sim apriltag layout and cleaned up cpp sim example by @r4stered in #1190
• Remove extra println that was used for debugging by @crschardt in #1193

Full Changelog: v2024.2.1...v2024.2.2

v2024.2.1

What's Changed

• Fix total ram reporting by @rzblue in #1161
• Fix UI bugs from RKNN PR by @srimanachanta in #1169
• Removed the print statement by @Tsunami014 in #1171
• Make java examples independent by adding GradleRIO version by @DeltaDizzy in #1158
• Add Orange Pi 5 Plus by @BytingBulldogs3539 in #1170
• Disable the network controls when networkingIsDisabled is true by @crschardt in #1118
• Draw calibration rainbow and scale thickness based on image size by @mcm001 in #1174
• Dont flush settings on exit after import by @mcm001 in #1179
• [photonlib-py] Begin implementing PhotonPoseEstimator in Python by @EWall25 in #1178
• Bump WPILib to 2024.2.1 by @crschardt in #1188
• Temperature monitoring for RK3588 by @crschardt in #1186
• Fix positioning of multitarget struct in pipelineresult unpack by @theVerySharpFlat in #1181

New Contributors

• @Tsunami014 made their first contribution in #1171
• @DeltaDizzy made their first contribution in #1158
• @EWall25 made their first contribution in #1178
• @theVerySharpFlat made their first contribution in #1181

Full Changelog: v2024.2.0...v2024.2.1

v2024.2.0

What's Changed

First big feature update of 2024! Brings (still incubating!) object detection using RKNN with a hard-coded YOLOv5 model trained on notes, and an (also new) LimeLight 3 image. Now that features work we'll be changing gears to focus on documenting new things. As always please reach out on Discord with journalctl logs if you encounter any issues!

• Add RKNN / Object Detection Pipeline by @mdurrani808 in #1144
• LL3 by @BytingBulldogs3539 in #1166

The nitty gritty

RKNN is the neural network accelerator hardware built into rockchip CPUs using on SBCs like the Orange Pi 5. We're able to use this hardware to massively accelerate certain math operations like those needed for running ML-based object detection. The code for that lives here, and publishes to Maven. Our pre-trained note model lives here, and NeuralNetworkModelManager deals with extracting that to disk. If a file named "note-640-640-yolov5s.rknn" and labels.txt does not exist at photonvision_config/models/, it'll be extracted from the JAR -- otherwise, the one already on disk will be used. This technically allows power users to replace the model and label files with new ones without rebuilding photon, though this feature is still largely untested, too.

Full Changelog: v2024.1.5...v2024.2.0