Add first 2 notes of 2024

2024/2024-01-09.md

@@ -0,0 +1,71 @@
+---
+tags: Tuesday
+---
+
+# 2024-01-09
+
+Attending: J-M, Petr, Seb, Chris, Jason, Andreas, Kyle, Kevin, Melissa, David, Khaled, June, Tom, Erin, Dom
+
+Start: 2:00 pm UK
+
+## Accepting minutes from [last meeting](https://hackmd.io/team/ome?nav=overview)
+
+## Project Status
+
+(2-3 minutes each)
+
+- IDR
+    - NGFF upgrade:
+        - Will: pending re-processing of a handful of NGFF Plates by BioStudies, I have hosted these on our own uk1s3 buckets and [used these from idr-next](https://github.com/IDR/idr-metadata/issues/675#issuecomment-1878828866) so all NGFF data should now be valid.
+        - Will: I have [run check_pixels.py](https://github.com/IDR/idr-utils/pull/55#issuecomment-1879859634) on 1 image per Plate or 2 images per Dataset across ALL data on idr-next. I see various Exceptions with `getPlane()` but the only mismatching pixels detected were from idr0009. Seb: "The problem is that this data was originally loaded with a version of the ScanR reader that represents missing wells as black images while the default behavior in Bio-Formats is to represent these as sparse plates. When this was backported to OME Bio-Formats, an option was introduced to support both behaviors - see  https://bio-formats.readthedocs.io/en/latest/formats/options.html
+I think the action is to update all plates in this study to set scanr.skip_missing_wells  to false , delete & regenerate the memo files and retest". Will: [tried this](https://github.com/IDR/idr-metadata/issues/675#issuecomment-1881406444) but no joy.
+        - Also Petr found that idr0001 images fail (except for the *first* Acquisition of each Plate), so we either need to convert to NGFF or reopen [Flex: merge IDR changes](https://github.com/ome/bioformats/pull/3537). Will: converted one Plate but [has issues with Field count per Well](https://github.com/IDR/idr-metadata/issues/683#issuecomment-1882669352). Needs more testing etc. Conversion of 192 Plates might take 6 days with bioformats2raw (don't know with omero-cli-zarr) and total 9TB. We probably don't want to block release for this (TBD)?
+    - J-M: to be discussed tomorrow after standup and make decisions.
+    - Seb: storage in good place. Working on upgrading IDR to Rocky Linux 9. Plan for Q1
+    - Jason: Systems came back after power outage ahead of time. TOO: J-M add links to mirror in our doc for future reference.
+
+
+- OMERO
+  - Issue with publication of ansible workflow to ansible galaxy: see [discussion](https://github.com/ome/ansible-role-postgresql-client/pull/13)
+      - rename repository or
+      - re-activate own GHA with workaround
+
+  - Upgrading [omero-test-infra](https://github.com/ome/omero-test-infra/pull/66) so we can release applications like omero-figure 
+  - Upgrade page for migrating systems
+  - OMERO.web looking at open PRs and cleaning up backlog.
+
+- Bio-Formats
+  - Melissa: PRs review next week for release timeline. Backlog is "acceptable" state.
+  - David: Reviewing state to plan release timeline for ZarrReader to include 0.5 spec 
+
+- NGFF
+  - cf. ZarrReader timeline
+
+- GerBI
+  - Tom: Working KVP scripts. Presentation next week of the current status. No progress on Web 
+
+- Glencoe
+  - Work on OMERO.tables 
+      - Documentation
+      - API changes: dependencies issues using OMERO.py (client vs server ones).
+      - Review requirements in OMERO.web. Better minimum for dependencies
+  - Seb: CentOS 7/RHEL 7 upgrade
+  - Seb: Python 3.12 support coming, Python 3.8 EOL in 9 months
+  - Erin: Upcoming Harvard tissue workshop end of January: OME.zarr (public data) with Webnossos+OMERO plus.
+      - presentation after workshop
+
+- Community
+    - Continuing efforts with UGM team
+      - no issue reported so far with the migration
+      - More tests to be done by Dewa.
+    - Grants submitted for working with HCS data (Wellcome), IDR at PSC (NIH), in progress for IDR maintenance (NIH). 
+    - Contact from Erick Ratamero and Ken Ho-- alternative hosts for OME meeting.
+      - looking at options  
+
+## AOB
+
+(5 min. max; tech. Discussion should be highlighted to relevant people and rescheduled)
+
+## Main Topic
+
+(20-25 minutes plus 15 minutes questions max)

2024/2024-01-16.md

@@ -0,0 +1,74 @@
+---
+tags: Tuesday
+---
+
+# 2024-01-16
+
+Attending:  Josh, Will, Dom, David Gault, Chris, Frances, Andreas, Khaled, David Stirling,  June, Melissa, Jena Wendt (Münster), Tom, Seb, Kyle, Kevin, Maadhav, Petr, Jean-Marie, Jason
+
+Start: 2:00 pm UK
+
+## Accepting minutes from [last meeting](https://hackmd.io/team/ome?nav=overview)
+
+## Project Status
+
+(2-3 minutes each)
+
+- IDR:
+    - Next release: idr0138 (NGFF) and idr0154
+        - idr0138 imported over holidays, ROIs now created - rendering settings needed
+        - idr0154 import will be quick
+    - NGFF release: "stress testing" this morning. Some images felt like they are loading a bit slower than on IDR, but difficult to quantify. Performance seemed to improve with more browsing (caching??).
+        - idr0011 Plate failing, due to renaming/moving on BIA s3.
+        - Various other plates resubmitted to BIA are now available (we have hosted these on our custom s3 buckets in the meantime for idr-next). Now looking at updating these to use BIA data...
+        - Running mkngff again to update NGFF Fileset will add `_mkngff` to templatePrefix, e.g. `demo_2/2016-07/28/21-44-02.006_mkngff_mkngff`.
+        - Some images didn't load, but appeared OK when re-loaded. 
+
+- OMERO
+  - upcoming release omero-py: 3.8 Python is the minimum version.
+      - Fixes test-infra (Pillow version issue)
+
+- Bio-Formats (David)
+    - PR reviews -
+        - Aim for 1st week of Feb for next 7.2.0 release (or 2 weeks later if date is missed)
+    - Nico Quick czi reader progressing
+    - ZarrReader moving towards NGFF v0.5 spec support
+        - will keep changes up to date with spec changes
+
+- NGFF
+  - No word on transforms. Time limit/cutoff?
+      - implementations progressing, python, JavaScript, etc
+      - We should look at ome-ngff-validator (schemas now available)
+  - [RFC](https://forum.image.sc/t/ngff-rfc-process-proposal-draft-now-available/90181) proposal opened. Next step: identifer reviewers.
+    - Additionally [ZEP0](https://zarr.dev/zeps/active/ZEP0000.html) is also undergoing review. Hopefully get this signed off by spring.
+  - Starting [Office hours](https://forum.image.sc/t/ome-files-office-hours-wednesdays-at-alternative-times/90850) to restart the website meetings
+    - Will: currently open issues? Josh: ping me URLs or we can meet at a different time?
+    - Upcoming questions around implementations (e.g., ilastik) 
+
+- Glencoe Chris
+    - Django 5.0 released. More async features, some things deprecated. Requires python 3.10.
+    - omero-web cleanup needed, old apis, interactions etc.
+    - Seb: python 2 compatibility code cleanup, 'future' etc.
+    - character encoding in NGFF, reserved chars, unicode etc. How to handle during conversion?
+
+- Community
+  - Josh: "OME data compatibility", next Wednesday (the 24th), 10AM EST/3PM UK/4PM EU
+      - Ping Josh if you need invite/more info?
+
+- GerBI (Tom)
+    - Key-Value pair scripts... See https://github.com/German-BioImaging/omero-scripts/tree/xtnd_support_kvpairs
+    - Editing exisitng KV scripts 
+    - Users start KV workflow in webclient, but doesn't work for SPW Wells.
+    - Single script to handle CSV -> P/D/I S/P/A/W and vice-versa (export KVP to csv)
+    - Also need Delete KVP script
+    - Namespace update "editable" vv protected from client editing
+    - Also option to add Tags instead of KVP
+    - Jens: Q: using Tags - should we try to use existing Tags? - input option for users to choose
+
+## AOB
+
+(5 min. max; tech. Discussion should be highlighted to relevant people and rescheduled)
+
+## Main Topic
+
+(20-25 minutes plus 15 minutes questions max)