Numpy Batch Operations in Bboxes and Keypoints #1396

GdoongMathew · 2023-02-10T09:17:19Z

Changes

Replace internal bbox & keypoint object with BboxesInternalType & KeypointsInternalType. Bboxes and keypoints will be converted to internal type in each DataProcessor's preprocess and back to original form in its postprocess.
Reimplement most bboxes and keypoints related transformation function in numpy operation.
Convert bboxes and keypoints internal target object from list to np.array as well. (updated on 12/24/2023)

Updated Transforms

Crop Transforms
CoarseDropout
Geometric Transform (except ElasticTransform, OpticalDistortion & GridDistortion)

Benchmark

10 bboxes

Transformation	imgaug	albumentations(numpy)	albumentations(for loop)
HorizontalFlip	0.203	0.317	0.824
VerticalFlip	0.168	0.276	1.298
Flip	0.267	0.273	1.13
Rotate	1.686	1.967	7.866
SafeRotate	-	2.965	10.541
RandomRotate90	1.05	0.886	4.009
ShiftScaleRotate	-	2.069	6.831
Transpose	-	1.064	3.369
Pad	1.1835	1.263	3.601
Perspective	3.2145	2.492	10.504
RandomCropNearBBox	-	0.183	0.666
BBoxSafeRandomCrop	-	0.372	1.368
CenterCrop	-	0.238	0.933
Crop	1.735	0.238	0.768
CropAndPad	-	1.004	3.455
RandomCropFromBorders	-	0.299	1.235
Affine	1.496	1.629	8.192
PiecewiseAffine	1254.1	369.816	1874.854
Sequence	8.541	7.021	8.599

unit: seconds / image

40 keypoints

Transformation	imgaug	albumentations(numpy)	albumentations(for loops)
HorizontalFlip	0.223	0.346	0.24
VerticalFlip	0.1775	0.319	0.208
Flip	0.261	0.349	0.225
Rotate	1.7625	1.982	1.972
SafeRotate	-	3.055	3.741
RandomRotate90	1.017	0.909	0.912
ShiftScaleRotate	-	2.037	2.264
Transpose	-	1.163	1.078
Pad	1.132	1.295	1.309
Perspective	2.312	2.308	2.618
RandomCropNearBBox	-	0.13	0.116
CenterCrop	-	0.119	0.11
Crop	1.595	0.143	0.116
CropAndPad	-	1.058	1.129
RandomCropFromBorders	-	0.304	0.205
Affine	1.495	1.562	2.483
PiecewiseAffine	1146.51	227.35	1151.898
Sequence	9.135	6.993	8.159

unit: seconds / image

Sequence: HorizontalFlip -> VerticalFlip -> Rotate -> RandomRotate90 -> Perspective -> Affine -> PadIfNeeded

Hardware

CPU: Intel i7-13700H
RAM: 16GB

…_from_albumentations with batch of bboxes in numpy operations.

2. added implementation of bboxes_flip, bboxes_vflip, bboxes_hflip and bboxes_transpose in numpy operation.

…bboxes operations.

…, VerticalFlip, HorizontalFlip, Flip's apply_to_bbox to numpy operations.

2. update ShiftScaleRotate apply_to_bboxes method.

…w_bbox_numpy_op

2. add bboxes_transpose implementation.

…d normalize_bboxes_np. 2. add test cases for test_bboxes_flip, test_crop_bboxes_by_coords, test_bboxes_center_crop, test_bboxes_crop, test_bboxes_random_crop. 3. Add methods in benchmark_bbox.py.

2. optimize some array operation.

2. store label field data into label_buffer.

2. add imgaug bounding boxes augmentation benchmark.

2. add filter_label method in DataProcessor. 3. change filter_keypoints and filter_bboxes return value to a resulting data and the filtered indices.

2. decorate functional bboxes transformation with ensure_and_convert_bboxes

tests/test_bbox.py

tests/test_serialization.py

albumentations/core/utils.py

albumentations/core/transforms_interface.py

albumentations/core/keypoints_utils.py

…nternal_format. 2. add BoxInternalType and KeypointInternalType as apply_to_bbox and apply_to_keypoint input. 3. raise error if comparing different object in BBoxesInternalType and KeypointsInternalType.

2. reformat parametrize arguments format.

GdoongMathew · 2023-02-15T11:18:13Z

I've added a few fixes.

converting bboxes / keypoints to internal type when testing transforming function.
add targets back in the unittests.
some requested changes.

point 1 and 2 were missing from the initial review, sorry for the additional corrections.

albumentations/core/transforms_interface.py

albumentations/core/keypoints_utils.py

…` if the input is not a np.ndarray or InteralType.

…lType and KeypointsInternalType.

GdoongMathew · 2023-02-21T10:23:21Z

@Dipet Hi, changes have been made as requested.

…onvert_to_original_type` methods. 2. change `convert_to_albumentations` and `convert_from_albumentations` input/output's type hint to `InternalDtype`.

GdoongMathew · 2023-03-01T15:34:11Z

Hi @Dipet,
since I haven't got further comments on this change request, I think currently it's best to just add the correct type hint instead of combining combine_bboxes_from_albumentations, convert_to_original_type & combine_bboxes_to_albumentations, convert_to_internal_type.

Feel free to give any further suggestions.

GdoongMathew · 2023-03-07T04:23:57Z

Hi @Dipet
I think there was a mistake when merging master into this branch, accidentally renamed convert_bboxes_to_albumentations to a in bbox_utils.py?

Dipet · 2023-03-07T06:04:56Z

@GdoongMathew Please, fix it. I can not properly checkout to your branch and push to this PR from my workstation.

GdoongMathew · 2023-03-07T07:17:53Z

@Dipet
Done.

GdoongMathew · 2023-03-07T11:21:29Z

Hi @Dipet
I think the version of scikit-image is the reason why the unittest failed.
Should I fix its version to scikit-image>=0.16.1,<0.20.0 in the setup file?
Or should I open an issue for this?

GdoongMathew · 2023-06-06T02:31:44Z

@Dipet
Hi, is there anything that still needs to be changed or that I can help with?

…eypointsInternalType default array value.

GdoongMathew and others added 30 commits January 5, 2023 18:51

1. reimplement filter_bboxes in numpy operations.

b6bb78a

1. implement normalize, denormalize, check_bboxes, and convert_bboxes…

070bef8

…_from_albumentations with batch of bboxes in numpy operations.

1. reimplement convert_bboxes_to_albumentations in numpy operation.

fba7f51

1. added docstring in bbox_utils.py

39a4737

2. added implementation of bboxes_flip, bboxes_vflip, bboxes_hflip and bboxes_transpose in numpy operation.

1. reimplement union_of_bboxes in numpy operation.

b599d2c

1. reimplement Crop related Transforms in transforms to adapt numpy b…

a147759

…bboxes operations.

1. reimplement bboxes rotation in transforms to adapt numpy operations.

05ee945

1. add benchmark_bbox.py.

5a626dc

1. update RandomRotate90 and Rotate, Perspective, Affine, PadIfNeeded…

fe7b448

…, VerticalFlip, HorizontalFlip, Flip's apply_to_bbox to numpy operations.

1. implement bboxes_shift_scale_rotate numpy operations.

bcc6cf0

2. update ShiftScaleRotate apply_to_bboxes method.

1. code cleanup.

f727843

1. add benchmark_bbox.py.

17479a3

Merge remote-tracking branch 'origin/mathew_bbox_numpy_op' into mathe…

a490648

…w_bbox_numpy_op

1. fixing false implementations in bboxes_vflip and bboxes_hflip.

a79293e

1. add bboxes_vflip and bboxes_hflip test cases.

6df8ba6

2. add bboxes_transpose implementation.

1. adding bboxes benchmark script.

82a10f6

1. add more bbox benchmark methods.

f8b9b4e

1. optimize cropping related transform method.

632444a

1. add new bbox benchmark method.

1e167d0

1. add test cases for calculate_bboxes_area, denormalize_bboxes_np an…

d3021c3

…d normalize_bboxes_np. 2. add test cases for test_bboxes_flip, test_crop_bboxes_by_coords, test_bboxes_center_crop, test_bboxes_crop, test_bboxes_random_crop. 3. Add methods in benchmark_bbox.py.

1. add docstring.

6c46147

2. optimize some array operation.

1. implement bboxes_safe_rotate in numpy operations.

3056ff6

1. add numpy operation benchmark code.

b371750

1. add assertion to KeypointsParams

f2dce08

2. store label field data into label_buffer.

1. fix mypy error.

0fba8a7

Merge branch 'master' into bbox_numpy

dda9e44

1. update angle_to_2pi_range type hints.

5b3b53a

2. add imgaug bounding boxes augmentation benchmark.

Merge remote-tracking branch 'origin/master' into bbox_numpy

c1510fd

1. store label_data in label_buffer.

221d0fb

2. add filter_label method in DataProcessor. 3. change filter_keypoints and filter_bboxes return value to a resulting data and the filtered indices.

1. add convert bbox to numpy function.

78fde34

2. decorate functional bboxes transformation with ensure_and_convert_bboxes

1. remove self branch from github Actions.

2c06ba7

Dipet requested changes Feb 12, 2023

View reviewed changes

GdoongMathew added 3 commits February 15, 2023 10:23

1. unify ensure_bboxes_format and ensure_keypoints_format as ensure_i…

be35347

…nternal_format. 2. add BoxInternalType and KeypointInternalType as apply_to_bbox and apply_to_keypoint input. 3. raise error if comparing different object in BBoxesInternalType and KeypointsInternalType.

1. fix test.

d09c82e

1. fix tests.

1546b30

2. reformat parametrize arguments format.

Merge branch 'master' into bbox_numpy

d1aea78

Dipet mentioned this pull request Feb 15, 2023

Affine has no rotate_method in 1.3.0 #1399

Open

Dipet requested changes Feb 15, 2023

View reviewed changes

albumentations/core/transforms_interface.py Outdated Show resolved Hide resolved

albumentations/core/keypoints_utils.py Show resolved Hide resolved

GdoongMathew and others added 3 commits February 16, 2023 03:03

1. raise TypeError in use_bboxes_ndarray and `use_keypoints_ndarray…

172a961

…` if the input is not a np.ndarray or InteralType.

1. fix test cases error in test_calculate_bboxes_area.

35774d9

1. add Floats4Internal to implement common methods in BBoxesInterna…

ca3458f

…lType and KeypointsInternalType.

1. add type hint in DataProcessor's convert_to_internal_type and `c…

7b31304

…onvert_to_original_type` methods. 2. change `convert_to_albumentations` and `convert_from_albumentations` input/output's type hint to `InternalDtype`.

Merge branch 'master' into bbox_numpy

e254544

1. fix naming error in convert_bboxes_to_albumentations.

e8c21a4

Dipet added 3 commits March 8, 2023 11:06

Merge branch 'master' into bbox_numpy

012b1a0

Merge branch 'master' into bbox_numpy

826a62d

Merge branch 'master' into bbox_numpy

cd7b149

GdoongMathew added 3 commits December 15, 2023 15:49

Merge remote-tracking branch 'Albumentations/master' into bbox_numpy

c2f8581

fix: fix mutable numpy array not accepted in BboxesInternalType and K…

60aae88

…eypointsInternalType default array value.

feat: update target in BatchInternalType from list to np.ndarray.

f900fe2

Dipet mentioned this pull request Mar 12, 2024

Vectorized interface for bboxes and keypoints #1577

Open

ternaus mentioned this pull request May 10, 2024

[Performance] Vecorize bounding boxes and keypoints #1718

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Numpy Batch Operations in Bboxes and Keypoints #1396

Numpy Batch Operations in Bboxes and Keypoints #1396

GdoongMathew commented Feb 10, 2023 •

edited

GdoongMathew commented Feb 15, 2023

GdoongMathew commented Feb 21, 2023

GdoongMathew commented Mar 1, 2023

GdoongMathew commented Mar 7, 2023

Dipet commented Mar 7, 2023

GdoongMathew commented Mar 7, 2023

GdoongMathew commented Mar 7, 2023 •

edited

GdoongMathew commented Jun 6, 2023

Numpy Batch Operations in Bboxes and Keypoints #1396

Are you sure you want to change the base?

Numpy Batch Operations in Bboxes and Keypoints #1396

Conversation

GdoongMathew commented Feb 10, 2023 • edited

Changes

Updated Transforms

Benchmark

10 bboxes

40 keypoints

Hardware

GdoongMathew commented Feb 15, 2023

GdoongMathew commented Feb 21, 2023

GdoongMathew commented Mar 1, 2023

GdoongMathew commented Mar 7, 2023

Dipet commented Mar 7, 2023

GdoongMathew commented Mar 7, 2023

GdoongMathew commented Mar 7, 2023 • edited

GdoongMathew commented Jun 6, 2023

GdoongMathew commented Feb 10, 2023 •

edited

GdoongMathew commented Mar 7, 2023 •

edited