Skip to content

Commit

Permalink
Update and renamed whitelists for hydrovis S3 put
Browse files Browse the repository at this point in the history
  • Loading branch information
RobHanna-NOAA authored Dec 23, 2022
1 parent 0355e2c commit 9116498
Show file tree
Hide file tree
Showing 4 changed files with 19 additions and 1 deletion.
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
hydroTable{}.csv
gw_catchments_reaches_filtered_addedAttributes{}.tif
rem_zeroed_masked{}.tif
usgs_elev_table.csv
4 changes: 3 additions & 1 deletion data/aws/s3.py
Original file line number Diff line number Diff line change
Expand Up @@ -180,9 +180,10 @@ def bulk_upload(self, src_folder_path, aws_target_path, whitelist_file_names = [
if __name__ == '__main__':

# Sample Usage
#python3 /foss_fim/data/aws/s3.py -a put -c /data/config/aws_creds.env -s /data/previous_fim/fim_4_0_7_0 -t "s3://example-us-east-1/fim_4_0_7_0" -w /foss_fim/config/aws_s3_put_whitelist.lst
#python3 /foss_fim/data/aws/s3.py -a put -c /data/config/aws_creds.env -s /data/previous_fim/fim_4_0_13_1 -t "s3://example-us-east-1/fim_4_0_13_1"

# You can leave the -w flag off to load all files/folders from a directory
# but default is to -w /foss_fim/config/aws_s3_put_fim4_hydrovis_whitelist.lst

# this works for hydroviz but can work with other s3 sites as well (ie esip)

Expand All @@ -198,6 +199,7 @@ def bulk_upload(self, src_folder_path, aws_target_path, whitelist_file_names = [
help='s3 bucket address and folder', required=True)
parser.add_argument('-w','--whitelist_file_path',
help='A file with the last of file names to be copied up (line delimited)',
default='/foss_fim/config/aws_s3_put_fim4_hydrovis_whitelist.lst',
required=False)
parser.add_argument('-v','--is_verbose',
help='Adding this flag will give additional tracing output',
Expand Down
15 changes: 15 additions & 0 deletions docs/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,21 @@
All notable changes to this project will be documented in this file.
We follow the [Semantic Versioning 2.0.0](http://semver.org/) format.

## v4.0.14.2 - 2022-12-22 - [PR #772](https://github.com/NOAA-OWP/inundation-mapping/pull/772)

Added `usgs_elev_table.csv` to hydrovis whitelist files. Also updated the name to include the word "hydrovis" in them (anticipating more s3 whitelist files).

### Changes

- `config`
- `aws_s3_put_fim4_hydrovis_whitelist.lst`: File name updated and added usgs_elev_table.csv so it gets push up as well.
- `aws_s3_put_fim3_hydrovis_whitelist.lst`: File name updated

- `data/aws`
- `s3.py`: added `/foss_fim/config/aws_s3_put_fim4_hydrovis_whitelist.lst` as a default to the -w param.

<br/><br/>

## v4.0.14.1 - 2022-12-03 - [PR #753](https://github.com/NOAA-OWP/inundation-mapping/pull/753)

Creates a polygon of 3DEP DEM domain (to eliminate errors caused by stream networks with no DEM data in areas of HUCs that are outside of the U.S. border) and uses the polygon layer to clip the WBD and stream network (to a buffer inside the WBD).
Expand Down

0 comments on commit 9116498

Please sign in to comment.