Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Could not process" batchDirectory #134

Open
arelkins1 opened this issue Dec 14, 2024 · 18 comments
Open

"Could not process" batchDirectory #134

arelkins1 opened this issue Dec 14, 2024 · 18 comments
Labels
0_Code for Zenhub Workspace Filtering end user issue - processing bug

Comments

@arelkins1
Copy link

Provide Log Files

(ping) PS C:\Users\arelkins\PythonPackages3\PINGMapper> python gui_main_batchDirectory.py
WARNING:tensorflow:From C:\Users\arelkins\AppData\Local\miniconda3\envs\ping\Lib\site-packages\tf_keras\src\losses.py:2976: The name tf.losses.sparse_softmax_cross_entropy is deprecated. Please use tf.compat.v1.losses.sparse_softmax_cross_entropy instead.

Version: 2.18.0
Eager mode: True
2024-12-14T19:08:31.030ZE [49660:NonCelloThread] registry_win.h:57:GetProtoFromRegistryValue Opening registry key Software\Google\DriveFS\Share failed with 0x2
2024-12-14T19:08:31.030ZE [37672:NonCelloThread] registry_win.h:57:GetProtoFromRegistryValue Opening registry key Software\Google\DriveFS\Share failed with 0x2
2024-12-14T19:08:38.476ZE [60068:NonCelloThread] registry_win.h:57:GetProtoFromRegistryValue Opening registry key Software\Google\DriveFS\Share failed with 0x2
inDir C:/Users/arelkins/Documents/2024data

Browse C:/Users/arelkins/Documents/2024data

0 C:/Users/arelkins/Documents/2024data/Outputs

proj C:/Users/arelkins/Documents/2024data/Outputs

prefix All

suffix

project_mode True

tempC 20.0

nchunk 500

exportUnknown False

fixNoDat False

threadCnt 0

cropRange 0.0

max_heading_deviation 0.0

max_heading_distance 0.0

min_speed 0.0

max_speed 0.0

6

aoi

x_offset 0.1

y_offset 0.1

egn False

egn_stretch Percent Clip

egn_stretch_factor 0.5

wcp False

wcr False

tileFile .jpg

lbl_set False

spdCor 0.0

maxCrop False

remShadow False

detectDep Sensor

smthDep False

adjDep 0.0

pltBedPick False

pix_res_son 0.05

rect_wcp False

rect_wcr False

son_colorMap Greys_r

pix_res_map 0.25

map_sub True

export_poly True

pltSubClass True

mosaic_nchunk 0

mosaic GTiff

map_mosaic GTiff

banklines True

coverage False

0 : C:/Users/arelkins/Documents/2024data\R00004\R00004.DAT
1 : C:/Users/arelkins/Documents/2024data\R00005\R00005.DAT
2 : C:/Users/arelkins/Documents/2024data\R00006\R00006.DAT
3 : C:/Users/arelkins/Documents/2024data\R00008\R00008.DAT
4 : C:/Users/arelkins/Documents/2024data\R00009\R00009.DAT
5 : C:/Users/arelkins/Documents/2024data\R00010\R00010.DAT
6 : C:/Users/arelkins/Documents/2024data\R00015\R00015.DAT
7 : C:/Users/arelkins/Documents/2024data\R00016\R00016.DAT
8 : C:/Users/arelkins/Documents/2024data\R00017\R00017.DAT
9 : C:/Users/arelkins/Documents/2024data\R00018\R00018.DAT
1

User Parameters
| project_mode : 1 |
| tempC : 20.0 |
| nchunk : 500 |
| cropRange : 0.0 |
| exportUnknown : False |
| fixNoDat : False |
| threadCnt : 0 |
| aoi : False |
| max_heading_deviation : 0.0 |
| max_heading_distance : 0.0 |
| min_speed : 0.0 |
| max_speed : 0.0 |
| pix_res_son : 0.05 |
| pix_res_map : 0.25 |
| x_offset : 0.1 |
| y_offset : 0.1 |
| egn : False |
| egn_stretch : 2 |
| egn_stretch_factor : 0.5 |
| wcp : False |
| wcr : False |
| tileFile : .jpg |
| lbl_set : 0 |
| spdCor : 0.0 |
| maxCrop : False |
| remShadow : 0 |
| detectDep : 0 |
| smthDep : False |
| adjDep : 0.0 |
| pltBedPick : False |
| rect_wcp : False |
| rect_wcr : False |
| son_colorMap : Greys_r |
| pred_sub : True |
| pltSubClass : True |
| map_sub : True |
| export_poly : True |
| map_class_method : max |
| map_predict : 0 |
| mosaic_nchunk : 0 |
| mosaic : 1 |
| map_mosaic : 1 |
| banklines : True |
| coverage : False |
sonPath C:/Users/arelkins/Documents/2024data\R00004\R00004

+++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++++++++++++++++++++++++++
***** Working On *****
C:/Users/arelkins/Documents/2024data\R00004\R00004.DAT
Start Time: 2024-12-14 14:08

===========================================

***** READING *****

Getting DAT Metadata...
C:/Users/arelkins/Documents/2024data\R00004\R00004.DAT

Done!
Time (s): 0.0

!!!!!!!!!!!!!!!!!
An Error Occured.

Please consult the logfile at:
C:/Users/arelkins/Documents/2024data/Outputs\AllR00004\logs\error_2024-12-14_1408.txt

Could not process: C:/Users/arelkins/Documents/2024data\R00004\R00004.DAT

Total Processing Time: 0:00:00
1

User Parameters
| project_mode : 1 |
| tempC : 20.0 |
| nchunk : 500 |
| cropRange : 0.0 |
| exportUnknown : False |
| fixNoDat : False |
| threadCnt : 0 |
| aoi : False |
| max_heading_deviation : 0.0 |
| max_heading_distance : 0.0 |
| min_speed : 0.0 |
| max_speed : 0.0 |
| pix_res_son : 0.05 |
| pix_res_map : 0.25 |
| x_offset : 0.1 |
| y_offset : 0.1 |
| egn : False |
| egn_stretch : 2 |
| egn_stretch_factor : 0.5 |
| wcp : False |
| wcr : False |
| tileFile : .jpg |
| lbl_set : 0 |
| spdCor : 0.0 |
| maxCrop : False |
| remShadow : 0 |
| detectDep : 0 |
| smthDep : False |
| adjDep : 0.0 |
| pltBedPick : False |
| rect_wcp : False |
| rect_wcr : False |
| son_colorMap : Greys_r |
| pred_sub : True |
| pltSubClass : True |
| map_sub : True |
| export_poly : True |
| map_class_method : max |
| map_predict : 0 |
| mosaic_nchunk : 0 |
| mosaic : 1 |
| map_mosaic : 1 |
| banklines : True |
| coverage : False |
| sonFiles : [] |
| logfilename : C:/Users/arelkins/Documents/2024data/Outputs\AllR00004\logs\log_2024-12-14_1408.txt |
| script : ['C:\Users\arelkins\PythonPackages3\PINGMapper\gui_main_batchDirectory.py', 'gui_main_batchDirectory_2024-12-14_1408.py'] |
| projDir : C:/Users/arelkins/Documents/2024data/Outputs\AllR00004 |
| inFile : C:/Users/arelkins/Documents/2024data\R00004\R00004.DAT |
sonPath C:/Users/arelkins/Documents/2024data\R00005\R00005

+++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++++++++++++++++++++++++++
***** Working On *****
C:/Users/arelkins/Documents/2024data\R00005\R00005.DAT
Start Time: 2024-12-14 14:08

===========================================

***** READING *****

Getting DAT Metadata...
C:/Users/arelkins/Documents/2024data\R00005\R00005.DAT

Done!
Time (s): 0.0

!!!!!!!!!!!!!!!!!
An Error Occured.

Please consult the logfile at:
C:/Users/arelkins/Documents/2024data/Outputs\AllR00005\logs\error_2024-12-14_1408.txt

Could not process: C:/Users/arelkins/Documents/2024data\R00005\R00005.DAT

Total Processing Time: 0:00:00
1

User Parameters
| project_mode : 1 |
| tempC : 20.0 |
| nchunk : 500 |
| cropRange : 0.0 |
| exportUnknown : False |
| fixNoDat : False |
| threadCnt : 0 |
| aoi : False |
| max_heading_deviation : 0.0 |
| max_heading_distance : 0.0 |
| min_speed : 0.0 |
| max_speed : 0.0 |
| pix_res_son : 0.05 |
| pix_res_map : 0.25 |
| x_offset : 0.1 |
| y_offset : 0.1 |
| egn : False |
| egn_stretch : 2 |
| egn_stretch_factor : 0.5 |
| wcp : False |
| wcr : False |
| tileFile : .jpg |
| lbl_set : 0 |
| spdCor : 0.0 |
| maxCrop : False |
| remShadow : 0 |
| detectDep : 0 |
| smthDep : False |
| adjDep : 0.0 |
| pltBedPick : False |
| rect_wcp : False |
| rect_wcr : False |
| son_colorMap : Greys_r |
| pred_sub : True |
| pltSubClass : True |
| map_sub : True |
| export_poly : True |
| map_class_method : max |
| map_predict : 0 |
| mosaic_nchunk : 0 |
| mosaic : 1 |
| map_mosaic : 1 |
| banklines : True |
| coverage : False |
| sonFiles : [] |
| logfilename : C:/Users/arelkins/Documents/2024data/Outputs\AllR00005\logs\log_2024-12-14_1408.txt |
| script : ['C:\Users\arelkins\PythonPackages3\PINGMapper\gui_main_batchDirectory.py', 'gui_main_batchDirectory_2024-12-14_1408.py'] |
| projDir : C:/Users/arelkins/Documents/2024data/Outputs\AllR00005 |
| inFile : C:/Users/arelkins/Documents/2024data\R00005\R00005.DAT |
sonPath C:/Users/arelkins/Documents/2024data\R00006\R00006

+++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++++++++++++++++++++++++++
***** Working On *****
C:/Users/arelkins/Documents/2024data\R00006\R00006.DAT
Start Time: 2024-12-14 14:08

===========================================

***** READING *****

Getting DAT Metadata...
C:/Users/arelkins/Documents/2024data\R00006\R00006.DAT

Done!
Time (s): 0.0

!!!!!!!!!!!!!!!!!
An Error Occured.

Please consult the logfile at:
C:/Users/arelkins/Documents/2024data/Outputs\AllR00006\logs\error_2024-12-14_1408.txt

Could not process: C:/Users/arelkins/Documents/2024data\R00006\R00006.DAT

Total Processing Time: 0:00:00
1

User Parameters
| project_mode : 1 |
| tempC : 20.0 |
| nchunk : 500 |
| cropRange : 0.0 |
| exportUnknown : False |
| fixNoDat : False |
| threadCnt : 0 |
| aoi : False |
| max_heading_deviation : 0.0 |
| max_heading_distance : 0.0 |
| min_speed : 0.0 |
| max_speed : 0.0 |
| pix_res_son : 0.05 |
| pix_res_map : 0.25 |
| x_offset : 0.1 |
| y_offset : 0.1 |
| egn : False |
| egn_stretch : 2 |
| egn_stretch_factor : 0.5 |
| wcp : False |
| wcr : False |
| tileFile : .jpg |
| lbl_set : 0 |
| spdCor : 0.0 |
| maxCrop : False |
| remShadow : 0 |
| detectDep : 0 |
| smthDep : False |
| adjDep : 0.0 |
| pltBedPick : False |
| rect_wcp : False |
| rect_wcr : False |
| son_colorMap : Greys_r |
| pred_sub : True |
| pltSubClass : True |
| map_sub : True |
| export_poly : True |
| map_class_method : max |
| map_predict : 0 |
| mosaic_nchunk : 0 |
| mosaic : 1 |
| map_mosaic : 1 |
| banklines : True |
| coverage : False |
| sonFiles : [] |
| logfilename : C:/Users/arelkins/Documents/2024data/Outputs\AllR00006\logs\log_2024-12-14_1408.txt |
| script : ['C:\Users\arelkins\PythonPackages3\PINGMapper\gui_main_batchDirectory.py', 'gui_main_batchDirectory_2024-12-14_1408.py'] |
| projDir : C:/Users/arelkins/Documents/2024data/Outputs\AllR00006 |
| inFile : C:/Users/arelkins/Documents/2024data\R00006\R00006.DAT |
sonPath C:/Users/arelkins/Documents/2024data\R00008\R00008

+++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++++++++++++++++++++++++++
***** Working On *****
C:/Users/arelkins/Documents/2024data\R00008\R00008.DAT
Start Time: 2024-12-14 14:09

===========================================

***** READING *****

Getting DAT Metadata...
C:/Users/arelkins/Documents/2024data\R00008\R00008.DAT

Done!
Time (s): 0.0

!!!!!!!!!!!!!!!!!
An Error Occured.

Please consult the logfile at:
C:/Users/arelkins/Documents/2024data/Outputs\AllR00008\logs\error_2024-12-14_1409.txt

Could not process: C:/Users/arelkins/Documents/2024data\R00008\R00008.DAT

Total Processing Time: 0:00:00
1

User Parameters
| project_mode : 1 |
| tempC : 20.0 |
| nchunk : 500 |
| cropRange : 0.0 |
| exportUnknown : False |
| fixNoDat : False |
| threadCnt : 0 |
| aoi : False |
| max_heading_deviation : 0.0 |
| max_heading_distance : 0.0 |
| min_speed : 0.0 |
| max_speed : 0.0 |
| pix_res_son : 0.05 |
| pix_res_map : 0.25 |
| x_offset : 0.1 |
| y_offset : 0.1 |
| egn : False |
| egn_stretch : 2 |
| egn_stretch_factor : 0.5 |
| wcp : False |
| wcr : False |
| tileFile : .jpg |
| lbl_set : 0 |
| spdCor : 0.0 |
| maxCrop : False |
| remShadow : 0 |
| detectDep : 0 |
| smthDep : False |
| adjDep : 0.0 |
| pltBedPick : False |
| rect_wcp : False |
| rect_wcr : False |
| son_colorMap : Greys_r |
| pred_sub : True |
| pltSubClass : True |
| map_sub : True |
| export_poly : True |
| map_class_method : max |
| map_predict : 0 |
| mosaic_nchunk : 0 |
| mosaic : 1 |
| map_mosaic : 1 |
| banklines : True |
| coverage : False |
| sonFiles : [] |
| logfilename : C:/Users/arelkins/Documents/2024data/Outputs\AllR00008\logs\log_2024-12-14_1409.txt |
| script : ['C:\Users\arelkins\PythonPackages3\PINGMapper\gui_main_batchDirectory.py', 'gui_main_batchDirectory_2024-12-14_1409.py'] |
| projDir : C:/Users/arelkins/Documents/2024data/Outputs\AllR00008 |
| inFile : C:/Users/arelkins/Documents/2024data\R00008\R00008.DAT |
sonPath C:/Users/arelkins/Documents/2024data\R00009\R00009

+++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++++++++++++++++++++++++++
***** Working On *****
C:/Users/arelkins/Documents/2024data\R00009\R00009.DAT
Start Time: 2024-12-14 14:09

===========================================

***** READING *****

Getting DAT Metadata...
C:/Users/arelkins/Documents/2024data\R00009\R00009.DAT

Done!
Time (s): 0.0

!!!!!!!!!!!!!!!!!
An Error Occured.

Please consult the logfile at:
C:/Users/arelkins/Documents/2024data/Outputs\AllR00009\logs\error_2024-12-14_1409.txt

Could not process: C:/Users/arelkins/Documents/2024data\R00009\R00009.DAT

Total Processing Time: 0:00:00
1

User Parameters
| project_mode : 1 |
| tempC : 20.0 |
| nchunk : 500 |
| cropRange : 0.0 |
| exportUnknown : False |
| fixNoDat : False |
| threadCnt : 0 |
| aoi : False |
| max_heading_deviation : 0.0 |
| max_heading_distance : 0.0 |
| min_speed : 0.0 |
| max_speed : 0.0 |
| pix_res_son : 0.05 |
| pix_res_map : 0.25 |
| x_offset : 0.1 |
| y_offset : 0.1 |
| egn : False |
| egn_stretch : 2 |
| egn_stretch_factor : 0.5 |
| wcp : False |
| wcr : False |
| tileFile : .jpg |
| lbl_set : 0 |
| spdCor : 0.0 |
| maxCrop : False |
| remShadow : 0 |
| detectDep : 0 |
| smthDep : False |
| adjDep : 0.0 |
| pltBedPick : False |
| rect_wcp : False |
| rect_wcr : False |
| son_colorMap : Greys_r |
| pred_sub : True |
| pltSubClass : True |
| map_sub : True |
| export_poly : True |
| map_class_method : max |
| map_predict : 0 |
| mosaic_nchunk : 0 |
| mosaic : 1 |
| map_mosaic : 1 |
| banklines : True |
| coverage : False |
| sonFiles : [] |
| logfilename : C:/Users/arelkins/Documents/2024data/Outputs\AllR00009\logs\log_2024-12-14_1409.txt |
| script : ['C:\Users\arelkins\PythonPackages3\PINGMapper\gui_main_batchDirectory.py', 'gui_main_batchDirectory_2024-12-14_1409.py'] |
| projDir : C:/Users/arelkins/Documents/2024data/Outputs\AllR00009 |
| inFile : C:/Users/arelkins/Documents/2024data\R00009\R00009.DAT |
sonPath C:/Users/arelkins/Documents/2024data\R00010\R00010

+++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++++++++++++++++++++++++++
***** Working On *****
C:/Users/arelkins/Documents/2024data\R00010\R00010.DAT
Start Time: 2024-12-14 14:09

===========================================

***** READING *****

Getting DAT Metadata...
C:/Users/arelkins/Documents/2024data\R00010\R00010.DAT

Done!
Time (s): 0.0

!!!!!!!!!!!!!!!!!
An Error Occured.

Please consult the logfile at:
C:/Users/arelkins/Documents/2024data/Outputs\AllR00010\logs\error_2024-12-14_1409.txt

Could not process: C:/Users/arelkins/Documents/2024data\R00010\R00010.DAT

Total Processing Time: 0:00:00
1

User Parameters
| project_mode : 1 |
| tempC : 20.0 |
| nchunk : 500 |
| cropRange : 0.0 |
| exportUnknown : False |
| fixNoDat : False |
| threadCnt : 0 |
| aoi : False |
| max_heading_deviation : 0.0 |
| max_heading_distance : 0.0 |
| min_speed : 0.0 |
| max_speed : 0.0 |
| pix_res_son : 0.05 |
| pix_res_map : 0.25 |
| x_offset : 0.1 |
| y_offset : 0.1 |
| egn : False |
| egn_stretch : 2 |
| egn_stretch_factor : 0.5 |
| wcp : False |
| wcr : False |
| tileFile : .jpg |
| lbl_set : 0 |
| spdCor : 0.0 |
| maxCrop : False |
| remShadow : 0 |
| detectDep : 0 |
| smthDep : False |
| adjDep : 0.0 |
| pltBedPick : False |
| rect_wcp : False |
| rect_wcr : False |
| son_colorMap : Greys_r |
| pred_sub : True |
| pltSubClass : True |
| map_sub : True |
| export_poly : True |
| map_class_method : max |
| map_predict : 0 |
| mosaic_nchunk : 0 |
| mosaic : 1 |
| map_mosaic : 1 |
| banklines : True |
| coverage : False |
| sonFiles : [] |
| logfilename : C:/Users/arelkins/Documents/2024data/Outputs\AllR00010\logs\log_2024-12-14_1409.txt |
| script : ['C:\Users\arelkins\PythonPackages3\PINGMapper\gui_main_batchDirectory.py', 'gui_main_batchDirectory_2024-12-14_1409.py'] |
| projDir : C:/Users/arelkins/Documents/2024data/Outputs\AllR00010 |
| inFile : C:/Users/arelkins/Documents/2024data\R00010\R00010.DAT |
sonPath C:/Users/arelkins/Documents/2024data\R00015\R00015

+++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++++++++++++++++++++++++++
***** Working On *****
C:/Users/arelkins/Documents/2024data\R00015\R00015.DAT
Start Time: 2024-12-14 14:09

===========================================

***** READING *****

Getting DAT Metadata...
C:/Users/arelkins/Documents/2024data\R00015\R00015.DAT

Done!
Time (s): 0.0

!!!!!!!!!!!!!!!!!
An Error Occured.

Please consult the logfile at:
C:/Users/arelkins/Documents/2024data/Outputs\AllR00015\logs\error_2024-12-14_1409.txt

Could not process: C:/Users/arelkins/Documents/2024data\R00015\R00015.DAT

Total Processing Time: 0:00:00
1

User Parameters
| project_mode : 1 |
| tempC : 20.0 |
| nchunk : 500 |
| cropRange : 0.0 |
| exportUnknown : False |
| fixNoDat : False |
| threadCnt : 0 |
| aoi : False |
| max_heading_deviation : 0.0 |
| max_heading_distance : 0.0 |
| min_speed : 0.0 |
| max_speed : 0.0 |
| pix_res_son : 0.05 |
| pix_res_map : 0.25 |
| x_offset : 0.1 |
| y_offset : 0.1 |
| egn : False |
| egn_stretch : 2 |
| egn_stretch_factor : 0.5 |
| wcp : False |
| wcr : False |
| tileFile : .jpg |
| lbl_set : 0 |
| spdCor : 0.0 |
| maxCrop : False |
| remShadow : 0 |
| detectDep : 0 |
| smthDep : False |
| adjDep : 0.0 |
| pltBedPick : False |
| rect_wcp : False |
| rect_wcr : False |
| son_colorMap : Greys_r |
| pred_sub : True |
| pltSubClass : True |
| map_sub : True |
| export_poly : True |
| map_class_method : max |
| map_predict : 0 |
| mosaic_nchunk : 0 |
| mosaic : 1 |
| map_mosaic : 1 |
| banklines : True |
| coverage : False |
| sonFiles : [] |
| logfilename : C:/Users/arelkins/Documents/2024data/Outputs\AllR00015\logs\log_2024-12-14_1409.txt |
| script : ['C:\Users\arelkins\PythonPackages3\PINGMapper\gui_main_batchDirectory.py', 'gui_main_batchDirectory_2024-12-14_1409.py'] |
| projDir : C:/Users/arelkins/Documents/2024data/Outputs\AllR00015 |
| inFile : C:/Users/arelkins/Documents/2024data\R00015\R00015.DAT |
sonPath C:/Users/arelkins/Documents/2024data\R00016\R00016

+++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++++++++++++++++++++++++++
***** Working On *****
C:/Users/arelkins/Documents/2024data\R00016\R00016.DAT
Start Time: 2024-12-14 14:09

===========================================

***** READING *****

Getting DAT Metadata...
C:/Users/arelkins/Documents/2024data\R00016\R00016.DAT

Done!
Time (s): 0.0

!!!!!!!!!!!!!!!!!
An Error Occured.

Please consult the logfile at:
C:/Users/arelkins/Documents/2024data/Outputs\AllR00016\logs\error_2024-12-14_1409.txt

Could not process: C:/Users/arelkins/Documents/2024data\R00016\R00016.DAT

Total Processing Time: 0:00:00
1

User Parameters
| project_mode : 1 |
| tempC : 20.0 |
| nchunk : 500 |
| cropRange : 0.0 |
| exportUnknown : False |
| fixNoDat : False |
| threadCnt : 0 |
| aoi : False |
| max_heading_deviation : 0.0 |
| max_heading_distance : 0.0 |
| min_speed : 0.0 |
| max_speed : 0.0 |
| pix_res_son : 0.05 |
| pix_res_map : 0.25 |
| x_offset : 0.1 |
| y_offset : 0.1 |
| egn : False |
| egn_stretch : 2 |
| egn_stretch_factor : 0.5 |
| wcp : False |
| wcr : False |
| tileFile : .jpg |
| lbl_set : 0 |
| spdCor : 0.0 |
| maxCrop : False |
| remShadow : 0 |
| detectDep : 0 |
| smthDep : False |
| adjDep : 0.0 |
| pltBedPick : False |
| rect_wcp : False |
| rect_wcr : False |
| son_colorMap : Greys_r |
| pred_sub : True |
| pltSubClass : True |
| map_sub : True |
| export_poly : True |
| map_class_method : max |
| map_predict : 0 |
| mosaic_nchunk : 0 |
| mosaic : 1 |
| map_mosaic : 1 |
| banklines : True |
| coverage : False |
| sonFiles : [] |
| logfilename : C:/Users/arelkins/Documents/2024data/Outputs\AllR00016\logs\log_2024-12-14_1409.txt |
| script : ['C:\Users\arelkins\PythonPackages3\PINGMapper\gui_main_batchDirectory.py', 'gui_main_batchDirectory_2024-12-14_1409.py'] |
| projDir : C:/Users/arelkins/Documents/2024data/Outputs\AllR00016 |
| inFile : C:/Users/arelkins/Documents/2024data\R00016\R00016.DAT |
sonPath C:/Users/arelkins/Documents/2024data\R00017\R00017

+++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++++++++++++++++++++++++++
***** Working On *****
C:/Users/arelkins/Documents/2024data\R00017\R00017.DAT
Start Time: 2024-12-14 14:09

===========================================

***** READING *****

Getting DAT Metadata...
C:/Users/arelkins/Documents/2024data\R00017\R00017.DAT

Done!
Time (s): 0.0

!!!!!!!!!!!!!!!!!
An Error Occured.

Please consult the logfile at:
C:/Users/arelkins/Documents/2024data/Outputs\AllR00017\logs\error_2024-12-14_1409.txt

Could not process: C:/Users/arelkins/Documents/2024data\R00017\R00017.DAT

Total Processing Time: 0:00:00
1

User Parameters
| project_mode : 1 |
| tempC : 20.0 |
| nchunk : 500 |
| cropRange : 0.0 |
| exportUnknown : False |
| fixNoDat : False |
| threadCnt : 0 |
| aoi : False |
| max_heading_deviation : 0.0 |
| max_heading_distance : 0.0 |
| min_speed : 0.0 |
| max_speed : 0.0 |
| pix_res_son : 0.05 |
| pix_res_map : 0.25 |
| x_offset : 0.1 |
| y_offset : 0.1 |
| egn : False |
| egn_stretch : 2 |
| egn_stretch_factor : 0.5 |
| wcp : False |
| wcr : False |
| tileFile : .jpg |
| lbl_set : 0 |
| spdCor : 0.0 |
| maxCrop : False |
| remShadow : 0 |
| detectDep : 0 |
| smthDep : False |
| adjDep : 0.0 |
| pltBedPick : False |
| rect_wcp : False |
| rect_wcr : False |
| son_colorMap : Greys_r |
| pred_sub : True |
| pltSubClass : True |
| map_sub : True |
| export_poly : True |
| map_class_method : max |
| map_predict : 0 |
| mosaic_nchunk : 0 |
| mosaic : 1 |
| map_mosaic : 1 |
| banklines : True |
| coverage : False |
| sonFiles : [] |
| logfilename : C:/Users/arelkins/Documents/2024data/Outputs\AllR00017\logs\log_2024-12-14_1409.txt |
| script : ['C:\Users\arelkins\PythonPackages3\PINGMapper\gui_main_batchDirectory.py', 'gui_main_batchDirectory_2024-12-14_1409.py'] |
| projDir : C:/Users/arelkins/Documents/2024data/Outputs\AllR00017 |
| inFile : C:/Users/arelkins/Documents/2024data\R00017\R00017.DAT |
sonPath C:/Users/arelkins/Documents/2024data\R00018\R00018

+++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++++++++++++++++++++++++++
***** Working On *****
C:/Users/arelkins/Documents/2024data\R00018\R00018.DAT
Start Time: 2024-12-14 14:09

===========================================

***** READING *****

Getting DAT Metadata...
C:/Users/arelkins/Documents/2024data\R00018\R00018.DAT

Done!
Time (s): 0.0

!!!!!!!!!!!!!!!!!
An Error Occured.

Please consult the logfile at:
C:/Users/arelkins/Documents/2024data/Outputs\AllR00018\logs\error_2024-12-14_1409.txt

Could not process: C:/Users/arelkins/Documents/2024data\R00018\R00018.DAT

Total Processing Time: 0:00:00

Provide Recording Files

https://drive.google.com/drive/folders/1iPeV6BPJpxjt86vepaaxoOZvMSfI8Pbj?usp=drive_link

Additional Information

We have gotten batch directory to run these files before, but when looking at substrate mosaics from the output, all of the output for the tracks is identical. The "could not process" error was showing in this output as well. After attempting to run the tracks again one by one and in a batch, I am receiving the same error. I hope that when resolved, it will solve the duplicate output issue.

@arelkins1 arelkins1 added 0_Code for Zenhub Workspace Filtering end user issue - processing bug labels Dec 14, 2024
@arelkins1
Copy link
Author

Error log:
Error Thrown:
Traceback (most recent call last):
File "C:\Users\arelkins\PythonPackages3\PINGMapper\gui_main.py", line 325, in
read_master_func(**params)
File "C:\Users\arelkins\PythonPackages3\PINGMapper\src\main_readFiles.py", line 284, in read_master_func
sonar_obj = hum2pingmapper(inFile, projDir, nchunk, tempC, exportUnknown)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\arelkins\AppData\Local\miniconda3\envs\ping\Lib\site-packages\pingverter\converter.py", line 119, in hum2pingmapper
del s, beam
^
UnboundLocalError: cannot access local variable 's' where it is not associated with a value

@CameronBodine
Copy link
Owner

Hi, thanks for trying pingmapper. Can you please attach the log files? They are inside of the project folders output by pingmapper. What you shared is the console output. While helpful, it does not show me what the error is. Thanks!

@CameronBodine
Copy link
Owner

Ah sorry. Just saw the error. Can you share one of the recordings with me to test?

@CameronBodine
Copy link
Owner

Can you also confirm that there are son files in similarly named subfolder in the same directory as the dat files?

As shown here:

https://cameronbodine.github.io/PINGMapper/docs/advanced/HumFileStructure.html#sd-card-files

@arelkins1
Copy link
Author

arelkins1 commented Dec 15, 2024 via email

@CameronBodine
Copy link
Owner

I see the issue. The dat file is inside the recording folder. Pingmapper expects the file to be outside the folder, at the same level as the recording folder. Just move the dat files and everything should work.

@CameronBodine
Copy link
Owner

Let me know how it goes.

@CameronBodine
Copy link
Owner

Here's an example. Son and idx files are in the corresponding folder.
image

@CameronBodine
Copy link
Owner

@arelkins1 were you able to successfully process your recordings?

@arelkins1
Copy link
Author

arelkins1 commented Dec 22, 2024 via email

@CameronBodine
Copy link
Owner

Make sure you have selected to export water column present (wcp) or removed (wcr) georectified imagery in addition to exporting the mosaic.

There are mosaics exported to the output folder that you linked. I can take a closer at your data this week if you continue to have issues.

@CameronBodine
Copy link
Owner

Also I don't see the attached pingmapper settings you referenced.

@arelkins1
Copy link
Author

arelkins1 commented Dec 30, 2024 via email

@arelkins1
Copy link
Author

arelkins1 commented Dec 30, 2024 via email

@CameronBodine
Copy link
Owner

CameronBodine commented Dec 30, 2024

Hi Arianna, thanks for following up. Unfortunately, I still do not see the referenced files. Perhaps it is because you are replying to the email from GitHub rather than adding to the issue directly on GitHub? Can you go ahead and email them to me, or share in a cloud drive? Thanks!

@CameronBodine
Copy link
Owner

@arelkins1 thanks for providing the files in an email. Attaching them here for reference. Also adding your notes for reference. Note that you can attach files by replying to this issue directly in GitHub: Issue 134.

I will take a look at this and provide further troubleshooting steps. Thanks!

Just to recap, ive gotten 7 of 10 tracks to spit out sonar raster mosaics, leaving me with tracks 5,8, and 15 unprocessed. With 15 being of most importance.
Additionally, I am missing some data from tracks 10 and 16, very close to a dam. It very well could be due to user error in the field, although the remaining 75% of those specific tracks provide data. This is shown in the ArcGIS image i've attached.

PING settings

Screenshot 2024-12-30 110047

12 22FileFormat

@CameronBodine
Copy link
Owner

Hi Arianna, see my findings below. It looks like most of your issues can be remedied by checking Locate and flag missing pings. Also, any missing data are due to poor scan conditions or presence of acoustic shadows, as noted below. Please let me know if this resolves your issues!

R00005

I was unable to process this recording and received the following error:

error_2024-12-31_1014.txt

There is no starboard sonar (B003.SON) channel available in the shared Google Drive, only the index file (B003.IDX). PINGMapper is unable to process the recording without this file.

R00015

I am able to process R00015 by checking Locate and flag missing pings. Note that this should not be checked by default as it can produce some strange anomalies (see #119 for more information).

image

image

R00008

I am able to process R00008 by checking Locate and flag missing pings. Note that this should not be checked by default as it can produce some strange anomalies (see #119 for more information).

image

R00010

I am able to process R00010 by checking Locate and flag missing pings. Note that this should not be checked by default as it can produce some strange anomalies (see #119 for more information). Areas of missing data are due to the poor data quality in the scan. Note the dark areas in the sonar mosaic near the dam. This is likely the result of the highly aerated water resulting from the dam. Highly aerated waters will degrade scan quality.

image

image

R00016

I am unable to reproduce the issue. Any missing data is due to acoustic shadows caused by bridge pilings, vegetation, etc. If you are exporting the data to a shared drive (Google, OneNote, etc.), I would recommend trying to export directly to a local drive. This has been a recent issue for others (#133).

image

CameronBodine added a commit that referenced this issue Jan 1, 2025
CameronBodine added a commit that referenced this issue Jan 1, 2025
@CameronBodine
Copy link
Owner

Hi Arianna, did this resolve your issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0_Code for Zenhub Workspace Filtering end user issue - processing bug
Projects
None yet
Development

No branches or pull requests

2 participants