Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace the ray-to-dem intersection algorithm with the secant method #5408

Open
wants to merge 1 commit into
base: dev
Choose a base branch
from

Conversation

oleg-alexandrov
Copy link
Contributor

This is a proposed fix for the issue of intersecting a ray from a rover camera with a DEM. There were several problems:

  • The rays emanating from the camera rover had wrong direction (my bug, Bufixes for MSL sensor intrinsics ale#589)
  • As @acpaquette noticed, the zero elevation datum is not a great initial guess for a tiny DEM which is 4,000 m below the datum and oblique rays
  • The intersection method uses the fixed point method. This is fragile. There was a comment in the code saying that current method fails at the limb, which I think is because of that.

This fix first finds a representative elevation based on sampling the DEM, then uses the secant method.

One issue that needs looking into is the stopping criterion. I left it the same, at 100th of pixel resolution. For MSL the computed resolution ranged between 2 m and 10 m, which is a bit too coarse I think for MSL. I am not sure if the provided resolution() function is accurate enough.

To test this, one should first merge the ALE code, regenerate the ISD, and reattach it to the cub file with same shapefile as before. Must fetch latest ASP if doing mapprojection, as there's a bug fix there too.

I tested both the old fixed point method and new secant method after initializing with a representative elevation, rather than zero datum. The old approach does well with campt when line and sample are 200 or more, when the ray looks more vertically, but the old approach fails for line and sample between 120 and 200 (approx) when rays are more oblique. The new secant method still does well then. But it breaks down when line and sample is 120 and below (approx) when one looks closer to the horizon. This was done with dataset NLB_450492465EDR_F0310634NCAM00357M1.cub with the fixed ISD and same shape file as what @acpaquette prepared.

This is a large change. It is suggested to have a build with and another without this change and make some sanity checks for orbital sensors. This is my last week on CSM work and I will not have much time for a large amount of work if this change has a big impact.

Licensing

This project is mostly composed of free and unencumbered software released into the public domain, and we are unlikely to accept contributions that are not also released into the public domain. Somewhere near the top of each file should have these words:

This work is free and unencumbered software released into the public domain. In jurisdictions that recognize copyright laws, the author or authors of this software dedicate any and all copyright interest in the software to the public domain.

  • I dedicate any and all copyright interest in this software to the public domain. I make this dedication for the benefit of the public at large and to the detriment of my heirs and successors. I intend this dedication to be an overt act of relinquishment in perpetuity of all present and future rights to this software under copyright law.

@acpaquette
Copy link
Collaborator

Just to keep you in the loop with tests on this @oleg-alexandrov, here is partial output from one of the camstats tests:

The difference between (double) group.findKeyword("LatitudeMinimum") and 9.928647861 is 2.0655599897168031e-08, which exceeds 1e-8, where
(double) group.findKeyword("LatitudeMinimum") evaluates to 9.9286478816555999,
9.928647861 evaluates to 9.928647861, and
1e-8 evaluates to 1e-08.

/tmp/jenkins-7a57a89e/workspace/ISIS_PR-5408/isis/tests/FunctionalTestsCamstats.cpp:29: Failure
The difference between (double) group.findKeyword("LatitudeMaximum") and 10.434709879 is 2.5338000142482997e-08, which exceeds 1e-8, where
(double) group.findKeyword("LatitudeMaximum") evaluates to 10.434709904338,
10.434709879 evaluates to 10.434709879, and
1e-8 evaluates to 1e-08.

/tmp/jenkins-7a57a89e/workspace/ISIS_PR-5408/isis/tests/FunctionalTestsCamstats.cpp:35: Failure
The difference between (double) group.findKeyword("LongitudeMaximum") and 256.146069653 is 5.8120008361584041e-08, which exceeds 1e-8, where
(double) group.findKeyword("LongitudeMaximum") evaluates to 256.14606959487998,
256.146069653 evaluates to 256.14606965299998, and
1e-8 evaluates to 1e-08.

/tmp/jenkins-7a57a89e/workspace/ISIS_PR-5408/isis/tests/FunctionalTestsCamstats.cpp:37: Failure
The difference between (double) group.findKeyword("LongitudeStandardDeviation") and 0.106583406 is 1.8490559999806422e-08, which exceeds 1e-8, where
(double) group.findKeyword("LongitudeStandardDeviation") evaluates to 0.10658338750944001,
0.106583406 evaluates to 0.10658340600000001, and
1e-8 evaluates to 1e-08.

/tmp/jenkins-7a57a89e/workspace/ISIS_PR-5408/isis/tests/FunctionalTestsCamstats.cpp:40: Failure
The difference between (double) group.findKeyword("SampleResolutionMinimum") and 18.840683397 is 1.4566600015086806e-07, which exceeds 1e-8, where
(double) group.findKeyword("SampleResolutionMinimum") evaluates to 18.840683542666,
18.840683397 evaluates to 18.840683396999999, and
1e-8 evaluates to 1e-08.

/tmp/jenkins-7a57a89e/workspace/ISIS_PR-5408/isis/tests/FunctionalTestsCamstats.cpp:41: Failure
The difference between (double) group.findKeyword("SampleResolutionMaximum") and 18.985953926 is 3.6378700229988681e-07, which exceeds 1e-8, where
(double) group.findKeyword("SampleResolutionMaximum") evaluates to 18.985953562212998,
18.985953926 evaluates to 18.985953926000001, and
1e-8 evaluates to 1e-08.

/tmp/jenkins-7a57a89e/workspace/ISIS_PR-5408/isis/tests/FunctionalTestsCamstats.cpp:42: Failure
The difference between (double) group.findKeyword("SampleResolutionAverage") and 18.908165593 is 5.2355002111426074e-08, which exceeds 1e-8, where
(double) group.findKeyword("SampleResolutionAverage") evaluates to 18.908165645355002,
18.908165593 evaluates to 18.908165593, and
1e-8 evaluates to 1e-08.

/tmp/jenkins-7a57a89e/workspace/ISIS_PR-5408/isis/tests/FunctionalTestsCamstats.cpp:43: Failure
The difference between (double) group.findKeyword("SampleResolutionStandardDeviation") and 0.038060025 is 1.4495013002380208e-08, which exceeds 1e-8, where
(double) group.findKeyword("SampleResolutionStandardDeviation") evaluates to 0.038060039495013,
0.038060025 evaluates to 0.038060024999999997, and
1e-8 evaluates to 1e-08.

/tmp/jenkins-7a57a89e/workspace/ISIS_PR-5408/isis/tests/FunctionalTestsCamstats.cpp:46: Failure
The difference between (double) group.findKeyword("LineResolutionMinimum") and 18.840683397 is 1.4566600015086806e-07, which exceeds 1e-8, where
(double) group.findKeyword("LineResolutionMinimum") evaluates to 18.840683542666,
18.840683397 evaluates to 18.840683396999999, and
1e-8 evaluates to 1e-08.

/tmp/jenkins-7a57a89e/workspace/ISIS_PR-5408/isis/tests/FunctionalTestsCamstats.cpp:47: Failure
The difference between (double) group.findKeyword("LineResolutionMaximum") and 18.985953926 is 3.6378700229988681e-07, which exceeds 1e-8, where
(double) group.findKeyword("LineResolutionMaximum") evaluates to 18.985953562212998,
18.985953926 evaluates to 18.985953926000001, and
1e-8 evaluates to 1e-08.

/tmp/jenkins-7a57a89e/workspace/ISIS_PR-5408/isis/tests/FunctionalTestsCamstats.cpp:48: Failure
The difference between (double) group.findKeyword("LineResolutionAverage") and 18.908165593 is 5.2355002111426074e-08, which exceeds 1e-8, where
(double) group.findKeyword("LineResolutionAverage") evaluates to 18.908165645355002,
18.908165593 evaluates to 18.908165593, and
1e-8 evaluates to 1e-08.

/tmp/jenkins-7a57a89e/workspace/ISIS_PR-5408/isis/tests/FunctionalTestsCamstats.cpp:49: Failure
The difference between (double) group.findKeyword("LineResolutionStandardDeviation") and 0.038060025 is 1.4495013002380208e-08, which exceeds 1e-8, where
(double) group.findKeyword("LineResolutionStandardDeviation") evaluates to 0.038060039495013,
0.038060025 evaluates to 0.038060024999999997, and
1e-8 evaluates to 1e-08.

We are exceptionally close to within our test tolerance but I don't know if there is much we can do about it

@oleg-alexandrov
Copy link
Contributor Author

oleg-alexandrov commented Jan 16, 2024

One may try to see if decreasing the tolerance in the stopping criterion in the secant method changes anything, or one could just call these the new reference results. Given that one peeks all the way from orbit, 1e-8 is small, whether these are pixels or lon-lat.

@oleg-alexandrov
Copy link
Contributor Author

I realized that jigsaw also uses this function. I am not sure what it is used for, but there's a possibility that the current stopping tolerance, of 1/100th of a pixel, may be too high. Especially if one has to do numerical differentiation based on the result of this.

In CSM ground-to-image, the tolerance used is about 1e-8. Here likely 1/1000 of a pixel may be a better tolerance, and likely will result in at most another iteration, given that the method converges very fast.

@jlaura
Copy link
Collaborator

jlaura commented Feb 8, 2024

@acpaquette What is the status on this? More work needed or waiting on bandwidth for additional review?

@acpaquette
Copy link
Collaborator

@jlaura @oleg-alexandrov I still need to look into all of the failing tests here and do some manual verification. I should be able to get to this under continuous support but it will take a but longer to fully explore the changes

@acpaquette
Copy link
Collaborator

Just to loop back around here, this seems like a good change for ISIS as it allows ISIS to use more local DEMs for both landed and orbital data. Still need to test on a network through jigsaw to see how the results differ.

@oleg-alexandrov
Copy link
Contributor Author

Sounds good. I would suggest making the tolerance for convergence smaller, maybe by a factor of 10, and seeing the effect. For ground-level DEMs, whose grid can grow with the distance from the rover, current tolerance looked to be on the high side.

@acpaquette
Copy link
Collaborator

@oleg-alexandrov I tested it against orbital data/networks and everything seems nominal. I was also able to replicate the images produced through ASP for the landed data. I think this PR from a functional stand point is good to go.

When you have time Oleg, would you be able to update the tests? No rush on our end as the grounded support hasn't been widely publicized.

Thanks for an amazing addition to the code base!

@oleg-alexandrov
Copy link
Contributor Author

I will not be coming back to CSM / ISIS work till middle of June. I have never worked with the ISIS test suite and know very little about its structure.

Can the ISIS maintainers pick up the slack? :) Likely not the first time the regression suite broke.

@Kelvinrr
Copy link
Collaborator

Kelvinrr commented Apr 5, 2024

@oleg-alexandrov we've always asked PR authors to write and update their own tests. People would be pretty updates if we suddenly made an exception. The process for updating docs is pretty well documented and new outside contributors seem to be able to pick up it pretty fast: https://astrogeology.usgs.gov/docs/how-to-guides/isis-developer-guides/writing-isis-tests-with-ctest-and-gtest/

@oleg-alexandrov
Copy link
Contributor Author

oleg-alexandrov commented Apr 8, 2024

Thanks for the user guide.

I believe I updated all the tests. I also pushed making the tolerance for convergence 1/1000 pixel rather than 1/100 pixel. This had the added bonus that more tests passed even before I started looking at them.

I brought up an issue in #5459. Currently some unit tests take a very long time while not doing much than computing a number or two. Multiply by the very large number of tests and potentially several iterations and that becomes a pain to deal with.

@oleg-alexandrov
Copy link
Contributor Author

Any thoughts on this?

@acpaquette
Copy link
Collaborator

@oleg-alexandrov Apologies for little activity on this, Jenkins has been dead for sometime now. I will work in the background this week to double check tests and provide a review here

@acpaquette
Copy link
Collaborator

Looking into this now that I have a handle on test failures on mac with a generic mac test config from #5517

@oleg-alexandrov
Copy link
Contributor Author

This is reminder to perhaps attempt to pull this in. This pull request has been around for 6 months.

A few months ago I fixed all the tests I could see failing. That meant dealing with large amount of editing of files and plugging in new numbers. Not fun. I can do that second time, if provided if a list of failures (after merging into latest), but then hoping it will be merged promptly afterwards.

@amystamile-usgs
Copy link
Contributor

@oleg-alexandrov I am unable to run tests until merge conflicts have been resolved.

merge

Save with debug info removed

Add code with no debug info

Wipe more debug logic

Tweak some comments

Punctuation

Hardening the logic for intersecting ray with DEM

DemShape: Let tolerance bet 1/1000 pixel, as that is more robust

Fix FunctionalTestsCamdev and FunctionalTestsCamrange

Fix FunctionalTestsCampt and FunctionalTestsCamstats
@oleg-alexandrov
Copy link
Contributor Author

I fixed the conflict. Look that the test results changed in different ways in my branch and the main one, so surely some tests will fail now. When you have the test results I will fix what is failing.

@amystamile-usgs
Copy link
Contributor

I fixed the conflict. Look that the test results changed in different ways in my branch and the main one, so surely some tests will fail now. When you have the test results I will fix what is failing.

I will run the tests now. It takes about an hour. I will post the results in the PR.

@amystamile-usgs
Copy link
Contributor

The following tests FAILED:
--
30829 | 65 - isis_unit_test_Camera (Failed)
30830 | 71 - isis_unit_test_CameraPointInfo (Failed)
30833 | 199 - isis_unit_test_PixelFOV (Failed)
30834 | 275 - isis_unit_test_Sensor (Failed)
30835 | 405 - cosi_app_test_default (Failed)
30837 | 491 - fx_app_test_bandDependent (Failed)
30838 | 492 - fx_app_test_camera (Failed)
30839 | 658 - photomet_app_test_clemMaxemission (Failed)
30840 | 660 - photomet_app_test_clementine (Failed)
30841 | 667 - photomet_app_test_useDem (Failed)
30842 | 672 - photrim_app_test_usedem (Failed)
30843 | 674 - pixel2map_app_test_crismpolyoverlap (Failed)
30846 | 779 - cnetref_app_test_interestNoOverlaps (Failed)
30847 | 780 - cnetref_app_test_interestOverlaps (Failed)
30848 | 790 - cnettable_app_test_allowErrors (Failed)
30849 | 791 - cnettable_app_test_append (Failed)
30850 | 792 - cnettable_app_test_default (Failed)
30851 | 855 - isis_module_test_CropCam2map (Failed)
30852 | 856 - isis_module_test_CropEnlargeCam2map (Failed)
30853 | 857 - isis_module_test_CropReduceCam2map (Failed)
30854 | 858 - isis_module_test_EnlargeCropCam2map (Failed)
30855 | 860 - isis_module_test_ReduceCropCam2map (Failed)
30856 | 862 - apollo_unit_test_ApolloMetricCamera (Failed)
30857 | 892 - chandrayaan1_unit_test_Chandrayaan1M3Camera (Failed)
30858 | 896 - clementine_unit_test_UvvisCamera (Failed)
30860 | 908 - dawn_unit_test_DawnFcCamera (Failed)
30864 | 954 - kaguya_unit_test_KaguyaMiCamera (Failed)
30865 | 955 - kaguya_unit_test_KaguyaTcCamera (Failed)
30866 | 960 - lo_unit_test_LoHighCamera (Failed)
30867 | 961 - lo_unit_test_LoMediumCamera (Failed)
30868 | 978 - lrowacpho_app_test_default (Failed)
30873 | 1029 - mgs_unit_test_MocWideAngleCamera (Failed)
30874 | 1036 - mocproc_app_test_case01 (Failed)
30875 | 1037 - mocproc_app_test_case02 (Failed)
30876 | 1038 - mocproc_app_test_case03 (Failed)
30878 | 1041 - mro_unit_test_CTXCamera (Failed)
30879 | 1042 - mro_unit_test_CrismCamera (Failed)
30880 | 1069 - hijitter_app_test_default (Failed)
30881 | 1070 - hijitter_app_test_withCrop (Failed)
30883 | 1099 - odyssey_unit_test_ThemisVisCamera (Failed)
30886 | 1106 - thmproc_app_test_ir (Failed)
30887 | 1107 - thmproc_app_test_vis (Failed)
30888 | 1111 - odyssey_module_test_themis (Failed)
30890 | 1125 - tgo_unit_test_TgoCassisCamera (Failed)
30891 | 1246 - ThreeImageNetwork.FunctionalTestFindfeaturesFastGeomGridDefault (Failed)
30892 | 1248 - ThreeImageNetwork.FunctionalTestFindimageoverlapsNoOverlap (Failed)
30893 | 1426 - DefaultCube.FunctionalTestCamrangeMeta (Failed)
30894 | 1431 - DefaultCube.FunctionalTestCamstatsDefaultParameters (Failed)
30895 | 1491 - DefaultCube.FunctionalTestNoprojFromUser (Failed)
30896 | 1494 - DefaultCube.FunctionalTestPhocubeAllBands (Failed)
30897 | 1524 - LineScannerCube.FunctionalTestNoprojLineScanner (Failed)
30898 | 1773 - TempTestingFiles.UnitTestImagePolygonCross (Failed)
30899 | 1976 - ApolloNetwork.FunctionalTestJigsawApollo (Failed)
30900 | 2052 - Lrowacphomap.FunctionalTestLrowacphomapNoBack (Failed)
30901 | 2054 - Lrowacphomap.FunctionalTestLrowacphomapDefaultAlgoAndParCubeNoBack (Failed)
30902 | 2356 - TgoCassisModuleKernels.TgoCassisSingleFrameletProjection (Failed)
30903 | 2357 - TgoCassisModuleKernels.TgoCassisTestColorMosaic (Failed)
30904 | 2359 - TgoCassisModuleKernels.TgoCassisSingleColorMosaicReingest (Failed)
30905 | 2360 - TgoCassisModuleKernels.TgoCassisTestProjSingleStitchedFrame (Failed)
30906 | 2362 - TgoCassisModuleTests.TgoCassisUncontrolledSingleColorMosaic (Failed)

@oleg-alexandrov
Copy link
Contributor Author

Were these tests passing before? Most of these have no relation to the function I changed, which projects points from image to ground.

@amystamile-usgs
Copy link
Contributor

@oleg-alexandrov I have been running tests for all the PRs today and have not run across these test failures. I can run the tests again and see if anything changes.

Currently, there are 19 test failures in the dev branch that I did not include in this list.

@oleg-alexandrov
Copy link
Contributor Author

oleg-alexandrov commented Jul 16, 2024 via email

@oleg-alexandrov
Copy link
Contributor Author

oleg-alexandrov commented Jul 16, 2024 via email

@amystamile-usgs
Copy link
Contributor

amystamile-usgs commented Jul 17, 2024

Here are the test logs:

tests-part1.log
tests-part2.log

Please ignore the follow tests that are failing on dev:

Screenshot 2024-07-15 at 2 12 24 PM

@oleg-alexandrov
Copy link
Contributor Author

oleg-alexandrov commented Jul 17, 2024

Thank you. The useful information I was able to extract from those is:

EVENTS 1721165603679 The difference between hist->Average() and 127.46759871644132 is 3.0619243602814095e-06, which exceeds .000001, where
EVENTS 1721165603679 The difference between hist->Sum() and 41629898 is 1, which exceeds .0001, where
EVENTS 1721165607710 The difference between hist->Sum() and bandSum[i-1] is 1.9193115008420136e-06, which exceeds 0.000001, where
EVENTS 1721165625868 The difference between hist->Average() and 26.259947527749951 is 4.372687521225771e-05, which exceeds .000001, where
EVENTS 1721165625868 The difference between hist->Sum() and 78070.824000000604 is 0.13000000000465661, which exceeds .0001, where
EVENTS 1721165887540 The difference between csvLine[i].toDouble() and compareMe[i].toDouble() is 0.00056271000005381211, which exceeds 0.000001, where
EVENTS 1721165887540 The difference between csvLine[i].toDouble() and compareMe[i].toDouble() is 0.00033369999982824083, which exceeds 0.000001, where
EVENTS 1721165887540 The difference between csvLine[i].toDouble() and compareMe[i].toDouble() is 0.0013663400000041293, which exceeds 0.000001, where
EVENTS 1721165887540 The difference between csvLine[i].toDouble() and compareMe[i].toDouble() is 0.0003895800000464078, which exceeds 0.000001, where
EVENTS 1721165887540 The difference between csvLine[i].toDouble() and compareMe[i].toDouble() is 8.171000013135199e-05, which exceeds 0.000001, where
EVENTS 1721165887540 The difference between csvLine[i].toDouble() and compareMe[i].toDouble() is 0.00096860000007836788, which exceeds 0.000001, where
EVENTS 1721165887540 The difference between csvLine[i].toDouble() and compareMe[i].toDouble() is 0.00036263000004055357, which exceeds 0.000001, where
EVENTS 1721165887540 The difference between csvLine[i].toDouble() and compareMe[i].toDouble() is 0.00032482999995409045, which exceeds 0.000001, where
EVENTS 1721165887540 The difference between csvLine[i].toDouble() and compareMe[i].toDouble() is 0.00044334000000390006, which exceeds 0.000001, where
EVENTS 1721165947957 The difference between hist->Sum() and expectedSum is 0.04754066004534252, which exceeds 0.001, where
EVENTS 1721165949970 The difference between hist->Sum() and expectedSum is 0.046142582286847755, which exceeds 0.001, where
EVENTS 1721166103015 The difference between hist->Sum() and 70857.19977273792 is 0.083809107542037964, which exceeds 0.0001, where
EVENTS 1721166103015 The difference between hist->Sum() and 78150.645788893104 is 0.097585096955299377, which exceeds 0.0001, where
EVENTS 1721166103015 The difference between hist->Sum() and 78810.883871480823 is 0.29421606659889221, which exceeds 0.0001, where
EVENTS 1721166103015 The difference between hist->Sum() and 42226.834142448381 is 0.050979908555746078, which exceeds 0.0001, where
EVENTS 1721166121057 The difference between hist->Sum() and 441491.9287794265 is 0.2429589219391346, which exceeds 0.001, where
EVENTS 1721166263188 The difference between (double) group.findKeyword("EmissionMinimum") and 6.5873900702382002 is 4.2110721071348003, which exceeds 1e-8, where
EVENTS 1721166263188 The difference between (double) group.findKeyword("EmissionMaximum") and 26.933664313876001 is 13.431033218835001, which exceeds 1e-8, where
EVENTS 1721166263188 The difference between (double) group.findKeyword("EmissionAverage") and 14.577805073101 is 2.4263181069420003, which exceeds 1e-8, where
EVENTS 1721166263188 The difference between (double) group.findKeyword("EmissionStandardDeviation") and 1.9856895689242999 is 1.42025136788749, which exceeds 1e-8, where
EVENTS 1721166263188 The difference between (double) group.findKeyword("IncidenceMinimum") and 53.332131708106999 is 16.608964322613005, which exceeds 1e-8, where
EVENTS 1721166263188 The difference between (double) group.findKeyword("IncidenceMaximum") and 73.850688669405002 is 3.5387436018540086, which exceeds 1e-8, where
EVENTS 1721166263188 The difference between (double) group.findKeyword("IncidenceAverage") and 66.178552518139 is 3.9489066163740034, which exceeds 1e-8, where
EVENTS 1721166263188 The difference between (double) group.findKeyword("IncidenceStandardDeviation") and 1.7434732723265001 is 1.6409828310974302, which exceeds 1e-8, where
EVENTS 1721166281230 The difference between 214.399354 and centroid->getX() is 0.037849262457228861, which exceeds 1e-6, where
EVENTS 1721166281230 The difference between 67.471751 and centroid->getY() is 0.016809208945247178, which exceeds 1e-6, where

These numbers are much larger than would be expected, and I don't understand what they mean, and even if they even have anything to do do with the code I changed.

Last time around I spent at least a day on this.

I regretfully no longer have time to go through this process again. The tests take 2-3 hours to run on my machine. Then one has to plug and chug the right numbers in the right files. Mistakes are unavoidable so one has to rerun the tests. One has to also configure rclone and fetch gigabytes of test data, set up the env, fetch ISIS, and wait for a couple of hours for it to build.

I hope the effort I spent modifying the image-to-ground code, which required a new algorithm, is appreciated, and that under the umbrella of the relevant project the ISIS developers can take it from there.

@jlaura
Copy link
Collaborator

jlaura commented Jul 17, 2024

This is an important contribution to have merged in support of work that I am doing. Given the challenges getting access to the test data #5550, the inability for external contributors to merge updates to the test data, and the fact that @oleg-alexandrov had GTests passing on or about April 8, I would propose that the remaining updates be addressed under an upcoming continuous support sprint.

Additionally, I would love to see process improvements that (1) make the test data more accessible, e.g., the proposed solution from #5550, (2) process improvements such that the turn around between contributions and review are significantly shorter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants