Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

set_geometry #889

Merged
merged 226 commits into from
Jul 9, 2024
Merged
Changes from all commits
Commits
Show all changes
226 commits
Select commit Hold shift + click to select a range
8cf5e0a
Add geodataframe to centroids: first commit
Apr 27, 2023
e2a2a05
Merge branch 'develop' into feature/centroids_as_gdf
Jun 21, 2023
89b4051
Remove raster methods, obsolete methods, and fix some.
Jun 21, 2023
70ea2fb
Remove test for removed methods
Jun 21, 2023
e34bc0f
Fix from geodataframe method
Jun 21, 2023
02b4a8b
Remove set meta to lat lon
Jun 22, 2023
5ede9bd
Update all I/O methods
Jun 29, 2023
98a898e
Remove area pixel
Jul 14, 2023
c77a2d9
Add method to compute the area per pixel (not attribute)
Jul 17, 2023
913c0f1
Make get_area_pixel
Jul 18, 2023
5ec2859
Remove not often used dist_coast and elevation
Jul 18, 2023
e2ac9b7
Remove unused meta
Aug 7, 2023
5dd4ffe
Change Centroids call to new signature
Aug 7, 2023
950a722
Remove vector - raster hazard methods
Aug 8, 2023
923b066
Restore legacy excell reader
Aug 8, 2023
abfff60
Remove clear method
Aug 8, 2023
8e360e6
Remove clear test
Aug 8, 2023
73c4392
Update read/write centroids in hazard
Aug 8, 2023
0dc0abb
Support empty on land or region id
Aug 8, 2023
c2f626a
Check for empty region id
Aug 8, 2023
bdf77d2
Set region id correctly
Aug 8, 2023
cf0128d
Remove scheduler
Aug 8, 2023
4f40def
Add docstring
Aug 8, 2023
5f520fb
Add legacy read hdf5
Aug 8, 2023
7c24df0
Replace incorrect dot product
Aug 8, 2023
e4bb789
Remove meta in assign centroids
Aug 11, 2023
3df5a4b
Remove matlab test file
Aug 11, 2023
7206446
Update set lat/lon
Aug 15, 2023
d4d5e02
Update test with init
Aug 15, 2023
a94d7af
Add note on method
Sep 22, 2023
df6c3f4
Merge tag 'v4.0.1' into feature/centroids_as_gdf
Sep 28, 2023
cbab08e
Add from_meta class method
Sep 28, 2023
5aeb35e
Update centroids init
Sep 28, 2023
fb374d3
Update read raster intensity / fraction data
Sep 28, 2023
4119f33
Merge branch 'develop' into feature/centroids_as_gdf
Sep 28, 2023
2390c5a
undo changelog duplications
emanuel-schmid Sep 29, 2023
4b1279e
Merge branch 'develop' into feature/centroids_as_gdf
Oct 13, 2023
315f7d9
Update test for points outside of raster within threshold
Oct 13, 2023
2b1ac74
Docstring and cosmetics
Oct 13, 2023
bab8c70
Add to default crs method
Oct 13, 2023
fe96975
Fix legacy from hdf5 for empty extra values
Oct 13, 2023
a4cf3ce
Update legacy from_hdf5 to exclude 'latitude'/'longitude' from extra
Oct 13, 2023
9ce8e56
Merge remote-tracking branch 'origin/develop' into feature/centroids_…
Nov 21, 2023
b22677f
Update centroids.select to work properly with mask and indices
Nov 21, 2023
1946bca
Fix forecast translate bug
Nov 21, 2023
b2c0c74
Remove _set_centroids
Nov 21, 2023
6226bff
Remove reproject raster
Nov 21, 2023
dc8d638
Remove test hazard raster
Nov 21, 2023
7002a52
Update naming for plot fraction centroids
Nov 21, 2023
73ee79e
Replace HAZ_DEMO_MAT with HAZ_TEST_TC
Nov 21, 2023
0cabeac
Replace mat file with hdf5 test file
Nov 22, 2023
3729248
Rewrite write raster method for hazard
Nov 22, 2023
6c3d1ac
Add a default for sel_cen in mask
Nov 22, 2023
8b15107
Correct indent typos
Nov 22, 2023
f3af2dd
Change duplicate test function name
Nov 23, 2023
5888dea
Replace mat file with hdf5
Nov 23, 2023
e22c857
Remove unused mat file import
Nov 23, 2023
847736b
Add legacy code to read old centroid hdf5 files.
Nov 23, 2023
756c587
Update access to centroids dist coast in gdf
Nov 23, 2023
e015000
test_trop_cyclone: get test centroids from data api
emanuel-schmid Nov 23, 2023
b0dfd66
Merge remote-tracking branch 'origin/develop' into feature/centroids_…
emanuel-schmid Nov 23, 2023
cfecf10
Add method to estimate meta raster from centroids
Nov 24, 2023
510f53f
Remove unecessary conversion to ne_geom
Nov 24, 2023
e108432
Add good lat/lon values for unit test
Nov 24, 2023
5571755
centroids.centr.write_hdf5: eliminate side effect
emanuel-schmid Nov 29, 2023
baf88a0
centroids.centr.write_hdf5: elimination of side effect futile for pan…
emanuel-schmid Nov 29, 2023
5b37116
centroids.test.TestCentroidsFuns.test_select_pass: adapt to downsized…
emanuel-schmid Nov 29, 2023
78811c5
test_vec_ras: fix TestCentroids.test_centroids_check_pass
emanuel-schmid Nov 29, 2023
84de9bf
test_vec_ras: remove TestReader.test_write_read_points_h5 as it is re…
emanuel-schmid Nov 29, 2023
1ab1b69
fix typo in from_exposures method
sarah-hlsn Dec 4, 2023
19b02a5
centroids.centr.from_csv: write class method
sarah-hlsn Dec 4, 2023
5e4e03d
centroids.centr.from_csv: correct order of xy
sarah-hlsn Dec 5, 2023
a426247
centroids.centr.from_excel: update method for gdf
sarah-hlsn Dec 5, 2023
c60e50f
centroids.centr. update csv and excel methods
sarah-hlsn Dec 5, 2023
ae6a0cc
Update tests in test_vec_ras
Dec 5, 2023
fbf28d4
Update more tests and remove some with meta
Dec 5, 2023
b3ae8c4
Merge remote-tracking branch 'origin/feature/centroids_as_gdf' into f…
sarah-hlsn Dec 5, 2023
929f0e1
centroids.centr.from_excel fix columnn indexing
sarah-hlsn Dec 5, 2023
17ee8b1
Add possibility to NOT recenter crs for countires.
Dec 5, 2023
e58aef2
Add to_csre convenience method
Dec 5, 2023
b70a0f4
Prepare the stage for setting other than admin0 region id
Dec 5, 2023
494d416
Remove uneeded import
Dec 5, 2023
448cf77
cenctroids.centr: fix the obvious
emanuel-schmid Dec 6, 2023
c736356
gencenctroids.centr: change crs of the geodataframe instead of the ge…
emanuel-schmid Dec 6, 2023
2880bcc
white space cosmetics
emanuel-schmid Dec 6, 2023
8cc88e7
PEP8
emanuel-schmid Dec 6, 2023
11a6439
Add solid region id and on land tests
Dec 6, 2023
a1fce1d
Merge branch 'feature/centroids_as_gdf' of https://github.com/CLIMADA…
Dec 6, 2023
bfcfc01
Auto stash before merge of "feature/centroids_as_gdf" and "origin/fea…
Dec 6, 2023
4a4aed7
Update distance and area tests
Dec 6, 2023
4460955
Update set crs in load vector shape file
Dec 6, 2023
37c54ea
centr.Centroids.from_excel: deal with region_id column
emanuel-schmid Dec 6, 2023
e4bc0db
Merge branch 'feature/centroids_as_gdf' of github.com:CLIMADA-project…
emanuel-schmid Dec 6, 2023
b02ae1d
Fix write hazard raster with centroids.get_meta
Dec 7, 2023
cf3f6e5
Merge branch 'feature/centroids_as_gdf' of https://github.com/CLIMADA…
sarah-hlsn Dec 7, 2023
b3448d0
Area pixel now correctly uses CEA
Dec 7, 2023
2beb6a3
Add actual NE CRS test.
Dec 7, 2023
a7eb166
Make on_land and region_id always part of centroids
Dec 7, 2023
147ee0d
Update docstring
Dec 7, 2023
445194b
Update import path
Dec 7, 2023
425b9ce
Add overwrite argument to set region id and on land
Dec 7, 2023
b879ca6
Remove from base grid method
Dec 7, 2023
d63fc8b
Fix some linting
Dec 7, 2023
70928bd
Add some docstrings
Dec 7, 2023
b97491c
Avoid costly computation in from geodataframe
Dec 7, 2023
d9e98ce
Add comment
Dec 7, 2023
274cf20
Fix typo
Dec 7, 2023
8101e3d
Merge branch 'feature/centroids_as_gdf' of https://github.com/CLIMADA…
sarah-hlsn Dec 7, 2023
42634bb
Fix set crs in from geodataframe
Dec 7, 2023
8727895
Make properties return arrays and not series
Dec 7, 2023
429e74c
Remove not needed matlab vars
Dec 7, 2023
fb7c7cc
Revert return numpy arrays.
Dec 7, 2023
dd212db
update from excel
sarah-hlsn Dec 8, 2023
5996714
Merge branch 'feature/centroids_as_gdf' of https://github.com/CLIMADA…
sarah-hlsn Dec 8, 2023
164e689
update from_excel and from_csv
sarah-hlsn Dec 8, 2023
4079f3f
centroids.centr: from_excel fix column renaming and adapt to "new arg…
emanuel-schmid Dec 8, 2023
dd7e4cd
Centroids.from_excel fix: switch key,val in test_storm_europe rather …
emanuel-schmid Dec 8, 2023
f6b72ed
Return numpy arrays instead of series
Dec 11, 2023
61f3c6b
Add some basic tests
Dec 12, 2023
adce3c1
Add tests for meta
Dec 12, 2023
9653e98
Add tentative changelog.
Dec 12, 2023
8e13e61
Make keywords argument only : name lat/lon : remove set_*
Dec 12, 2023
43c2f1b
Auto stash before revert of "Make keywords argument only : name lat/l…
Dec 12, 2023
32a4be9
remove file
Dec 12, 2023
d1475d8
Merge branch 'local/geodataframe_temp' into feature/centroids_as_gdf
Dec 12, 2023
2464066
Fix exposures method test
Dec 12, 2023
bd5629c
Merge remote-tracking branch 'origin/develop' into feature/centroids_…
Dec 13, 2023
d031a79
Fix setting region id and on land at init.
Dec 13, 2023
a1040f8
Fix typo
Dec 13, 2023
a748ee0
Fix all close values for big numbers
Dec 13, 2023
652a2b8
Improve cosmetics.
Dec 13, 2023
6806331
update docstrings, doc cosmetics
idetring Dec 14, 2023
25a0930
update from_csv method, add test
sarah-hlsn Dec 18, 2023
5849acb
remove DEF_VAR_CSV
sarah-hlsn Dec 18, 2023
84107ee
Update from_excel method and test
sarah-hlsn Dec 20, 2023
bfb24d7
remove unnecessary import
sarah-hlsn Dec 20, 2023
67a1d60
fix storm_europe tests
sarah-hlsn Dec 20, 2023
8f25d1f
improve csv and excel tests
sarah-hlsn Jan 3, 2024
0f54053
Apply suggestions from code review
chahank Jan 3, 2024
c4bb540
add excel, csv write methods and tests
sarah-hlsn Jan 4, 2024
3f4ebb6
Merge branch 'feature/centroids_as_gdf' of https://github.com/CLIMADA…
sarah-hlsn Jan 4, 2024
c7e6ffd
Replace gdf with _gdf
Jan 4, 2024
79cd84b
Revert "Replace gdf with _gdf"
Jan 4, 2024
e5dffd9
Update to_crs methods
Jan 5, 2024
1f6ec54
Raise error if wrong exposures
Jan 5, 2024
f59278c
Make from geodataframe more restrictive.
Jan 5, 2024
4c641dc
Use consistently to_crs method
Jan 5, 2024
e5c1c73
Improve read centroids
Jan 5, 2024
26e2702
Add literal to import
Jan 8, 2024
e6fa996
Update to crs with inplace argument
Jan 8, 2024
37b5d19
improve excel, csv method, test based on review
sarah-hlsn Jan 8, 2024
622d326
Update typing
Jan 8, 2024
a7311a0
Fix equal centroids for different gdf columns ordering
Jan 8, 2024
99203ad
Make consistent gdf column ordering
Jan 8, 2024
e5cd4ad
Simplify from excel/csv
Jan 8, 2024
58360a5
Add support for legacy hazard excell
Jan 8, 2024
2f248b8
Avoid single column loading error for df
Jan 8, 2024
19cae86
fix path in write_excel, write_csv
sarah-hlsn Jan 9, 2024
19fe589
fixing docstrings in centr.py
idetring Jan 9, 2024
5ccc278
Update code cosmetics
Jan 10, 2024
ee512ca
Fix typo
Jan 10, 2024
4053565
Allo for kwargs in init and update from exposures
Jan 18, 2024
fc0c7ec
Merge branch 'develop' into feature/centroids_as_gdf
emanuel-schmid Jan 18, 2024
a30a7c4
Merge branch 'feature/centroids_as_gdf' of https://github.com/CLIMADA…
emanuel-schmid Jan 18, 2024
5404dba
Merge branch 'develop' into feature/centroids_as_gdf
emanuel-schmid Jan 22, 2024
4d62101
code linting
Jan 24, 2024
1ba2ade
Centroids: test from_meta
Jan 24, 2024
62df2dd
Centroids: refined tests
Jan 24, 2024
a7b8cf9
Centroids: fix from_exposures
Jan 24, 2024
e514195
Centroids: implement get_pixel_shapes
Jan 24, 2024
ec11611
Centroids: code linting and docstrings
Jan 25, 2024
e77af81
hazard.test.test_base: remove undefined classes from main
emanuel-schmid Jan 29, 2024
bba7e49
hazard.base: remove unused imports
emanuel-schmid Jan 30, 2024
655b749
hazard.base: clean up imports
Jan 30, 2024
03676d8
Merge branch 'develop' into feature/centroids_as_gdf
emanuel-schmid Jan 30, 2024
7dfa631
cosmetics
emanuel-schmid Jan 30, 2024
eda2755
hazard.base.write_htdf5: add inline comment.
emanuel-schmid Jan 30, 2024
8fcf3b3
Centroids: fix _gdf_from_legacy_hdf signature
emanuel-schmid Jan 31, 2024
57fd369
Centroids._gdf_from_legacy_hdf5: static, not classmethod
Jan 31, 2024
4ffd744
Merge branch 'develop' into feature/centroids_as_gdf
emanuel-schmid Jan 31, 2024
ddb6dcc
hazard.centroids.centr: pylint
emanuel-schmid Jan 31, 2024
bd1f9fd
centroids.centr : fix pydoc typos and pylint
emanuel-schmid Jan 31, 2024
31aba1c
climada.hazard.base: pylint
emanuel-schmid Jan 31, 2024
b84a76d
hazard.centroids.centr: pylint
emanuel-schmid Jan 31, 2024
b1b0efa
climada.test.test_calicbration: use hzard test file with hdf5 format
emanuel-schmid Jan 31, 2024
473bc9d
Update climada/hazard/centroids/test/test_vec_ras.py
tovogt Feb 1, 2024
017864e
Hazard: fix write_raster, and some docstrings
Feb 1, 2024
441fbb1
test.test_calibration: fix test file name
emanuel-schmid Feb 7, 2024
0cd3530
Merge branch 'feature/centroids_as_gdf' of github.com:CLIMADA-project…
emanuel-schmid Feb 7, 2024
db748c7
test_api_client: added test for basic centroids plotting
emanuel-schmid Feb 7, 2024
5863cb2
centr.Centroids.plot: all changes reverted
emanuel-schmid Feb 8, 2024
3dd4a50
Hazard tutorial: use centroids.get_meta() instead of the attribute meta
emanuel-schmid Feb 8, 2024
065c868
Hazard tutorial: use Centroids() instead of from_lat_lon()
emanuel-schmid Feb 8, 2024
5e1d9ff
Merge branch 'develop' into feature/centroids_as_gdf
emanuel-schmid Feb 8, 2024
f6f7db8
hazard tutorial: update Hazard from raster section
emanuel-schmid Feb 9, 2024
ead5e05
hazard tutorial: fix dist_coast access
emanuel-schmid Feb 9, 2024
693af77
climada.hazard.Hazard: remove broken vector file support
emanuel-schmid Feb 9, 2024
1f45167
Merge branch 'develop' into feature/centroids_as_gdf
emanuel-schmid Feb 14, 2024
bc75e0f
climada.hazard.centr: add deprecated methods section
emanuel-schmid Feb 15, 2024
31409cd
Merge branch 'develop' into feature/centroids_as_gdf
emanuel-schmid Feb 15, 2024
e59eb18
fix from_tracks method, remove side effect
emanuel-schmid Feb 15, 2024
0ad0804
climada.hazard.trop_cyclone.TropCytlone.from_tracks: option for prede…
emanuel-schmid Feb 15, 2024
88b320a
TropicalCyclone.from_tracks pydoc
emanuel-schmid Feb 15, 2024
b543636
white space cosmetics
emanuel-schmid Feb 16, 2024
5f3904b
CHANGELOG: list deprecated and removed methods
emanuel-schmid Feb 16, 2024
7b009e9
Refactor and add to centroids testing
idetring Feb 20, 2024
eac96d8
Merge branch 'develop' into feature/centroids_as_gdf
emanuel-schmid Feb 21, 2024
daa9812
Merge branch 'develop' into feature/centroids_as_gdf
emanuel-schmid Feb 21, 2024
53bd89c
Merge branch 'develop' into feature/centroids_as_gdf
emanuel-schmid Feb 21, 2024
5fba7f6
trop_cyclone.TropCyclone.from_tracks: fix dist_coast type
emanuel-schmid Feb 21, 2024
62f2637
climada.hazard.Centroids: raise exception in deprecated methods that …
emanuel-schmid Feb 23, 2024
cd960c2
changelog: list added Centroids methods
emanuel-schmid Feb 26, 2024
05676cb
hazard.centr: depreecated methods: pydoc string instaed of pass
emanuel-schmid Mar 14, 2024
5988922
Merge branch 'develop' into feature/centroids_as_gdf
emanuel-schmid Mar 14, 2024
d6d2859
fixing geodataframe issues
emanuel-schmid Mar 14, 2024
da98ff6
Merge branch 'develop' into feature/centroids_as_gdf
emanuel-schmid Mar 14, 2024
ab03d2a
Revert "fixing geodataframe issues"
emanuel-schmid Mar 14, 2024
9873dea
proposed fix for set_geometry warning
emanuel-schmid Mar 14, 2024
6246172
Merge branch 'develop' into feature/set_geometry_fix
emanuel-schmid May 8, 2024
9208acc
Merge branch 'develop' into feature/set_geometry_fix
emanuel-schmid May 15, 2024
f5cc9de
u_coord: use set_geometry instead of direct column assignment for Geo…
emanuel-schmid Jun 7, 2024
329bf61
Merge branch 'develop' into feature/set_geometry_fix
emanuel-schmid Jun 7, 2024
5412099
Merge branch 'feature/set_geometry' into feature/set_geometry_fix
emanuel-schmid Jun 7, 2024
c9d7c95
white space cosmetics
emanuel-schmid Jun 7, 2024
2cdcdb7
Merge branch 'develop' into feature/set_geometry
emanuel-schmid Jul 9, 2024
fe378cd
simplification
emanuel-schmid Jul 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 15 additions & 14 deletions climada/util/coordinates.py
Original file line number Diff line number Diff line change
Expand Up @@ -2412,13 +2412,16 @@ def apply_box(df_exp):
LOGGER.info('Raster from resolution %s to %s.', res, raster_res)
df_poly = gpd.GeoDataFrame(points_df[val_names])
if not scheduler:
df_poly['geometry'] = apply_box(points_df)
df_poly['_-geometry-prov'] = apply_box(points_df)
else:
ddata = dd.from_pandas(points_df[['latitude', 'longitude']],
npartitions=cpu_count())
df_poly['geometry'] = ddata.map_partitions(apply_box, meta=Polygon) \
df_poly['_-geometry-prov'] = ddata.map_partitions(apply_box, meta=Polygon) \
.compute(scheduler=scheduler)
df_poly.set_crs(crs if crs else points_df.crs if points_df.crs else DEF_CRS, inplace=True)
df_poly.set_geometry('_-geometry-prov',
crs=crs if crs else points_df.crs if points_df.crs else DEF_CRS,
inplace=True,
drop=True)

# renormalize longitude if necessary
if equal_crs(df_poly.crs, DEF_CRS):
Expand Down Expand Up @@ -2651,30 +2654,28 @@ def set_df_geometry_points(df_val, scheduler=None, crs=None):
LOGGER.info('Setting geometry points.')

# keep the original crs if any
if crs is None:
try:
crs = df_val.geometry.crs
except AttributeError:
crs = None
crs = df_val.crs if crs is None else crs # crs might now still be None

# work in parallel
if scheduler:
def apply_point(df_exp):
return df_exp.apply(lambda row: Point(row.longitude, row.latitude), axis=1)

ddata = dd.from_pandas(df_val, npartitions=cpu_count())
df_val['geometry'] = ddata.map_partitions(
df_val['_-geometry-prov'] = ddata.map_partitions(
apply_point,
meta=('geometry', gpd.array.GeometryDtype)
).compute(scheduler=scheduler)

# single process
else:
df_val['geometry'] = gpd.GeoSeries(
gpd.points_from_xy(df_val.longitude, df_val.latitude), index=df_val.index, crs=crs)
df_val['_-geometry-prov'] = gpd.GeoSeries(
gpd.points_from_xy(df_val.longitude, df_val.latitude),
index=df_val.index)

# set crs
if crs:
df_val.set_crs(crs, inplace=True)
# A 'geometry' column must not be created in a GeoDataFrame except through the constructor
# or with set_geometry. That's why we first made a temporary columns with a weird name
df_val.set_geometry('_-geometry-prov', inplace=True, drop=True, crs=crs)


def fao_code_def():
Expand Down
Loading