[improve][fn] Implement pip 401: Support set batching configurations for Pulsar Functions&Sources

[jiangpengcheng]

Fixes #xyz

Main Issue: #xyz

PIP: #23793

Motivation

Modifications

Verifying this change

• Make sure that the change passes the CI checks.

• This change added tests and can be verified as follows:

  • Added integration tests for creating function without batching config
  • Added integration tests for creating function with custom batching config
  • Added integration tests for creating function with disable batching config
  • Added integration tests for creating source without batching config
  • Added integration tests for creating source with custom batching config
  • Added integration tests for creating source with disable batching config

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
  • it make the functions&sources creating/updating command's --producer-config argument to include a batchingConfig field
• The metrics
• Anything that affects deployment

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

Matching PR in forked repository

PR in forked repository: jiangpengcheng#37

[lhotari]

LGTM, good work @jiangpengcheng

[nlu90]

Thank you for the work!

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 73.77049% with 16 lines in your changes missing coverage. Please review.

Project coverage is 74.19%. Comparing base (bbc6224) to head (926e35f).
Report is 861 commits behind head on master.

Files with missing lines	Patch %	Lines
...g/apache/pulsar/functions/utils/BatchingUtils.java	72.97%	2 Missing and 8 partials ⚠️
...sar/functions/instance/ProducerBuilderFactory.java	68.42%	1 Missing and 5 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #23860      +/-   ##
============================================
+ Coverage     73.57%   74.19%   +0.61%     
+ Complexity    32624    32215     -409     
============================================
  Files          1877     1854      -23     
  Lines        139502   143660    +4158     
  Branches      15299    16331    +1032     
============================================
+ Hits         102638   106587    +3949     
+ Misses        28908    28670     -238     
- Partials       7956     8403     +447     

Flag	Coverage Δ
inttests	26.58% <0.00%> (+2.00%)	⬆️
systests	23.19% <70.49%> (-1.13%)	⬇️
unittests	73.70% <47.54%> (+0.85%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...ulsar/functions/instance/JavaInstanceRunnable.java	73.29% <100.00%> (+0.24%)	⬆️
...he/pulsar/functions/utils/FunctionConfigUtils.java	73.67% <100.00%> (+0.72%)	⬆️
...sar/functions/instance/ProducerBuilderFactory.java	87.01% <68.42%> (ø)
...g/apache/pulsar/functions/utils/BatchingUtils.java	72.97% <72.97%> (ø)

... and 1021 files with indirect coverage changes