You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Apache Beam recommends to use Fakes instead of Mocks, since Mocks cannot serialize over the pipeline.
I am writing unit tests for older code, where the class is using Apache Beam calling other external class services (which I have not written or cannot change). The classes aren't using interfaces.
public class DataAdapter extends DoFn<KV<String, Product>, Void>{
public String processCategory;
public Map<String, String> configs;
public ServiceApi serviceApi;
public DataAdapter(String processCategory, Map<String, String> configs) {
this.processCategory = processCategory;
this.configs = configs;
}
@Setup
public void init() {
serviceApi = new ServiceApi(processCategory, configs);
}
@ProcessElement
public void processData(ProcessContext context, @Element KV<String, Product> input) {
try {
String productId = input.getKey();
serviceApi.getData();
....
}
}
Attempt:
I created the fake below. Calling the constructors with nulls will cause Argument Exception errors in the serviceApi constructor. So I tried a factory create pattern which is probably not ideal. (trying to find another solution)
However, the fake keeps getting overriden, since for each ProcessElement, it calls Setup, with reinitializes Parent Class. We had to initialize in each SetupBatch, since serviceApi was not serializable in running Apache Beam application.
Create Fake:
public class ServiceApiFake extends ServiceApi {
public ServiceApiFake() {
super(null, null);
}
public static ServiceApiFake create() {
try {
return new ServiceApiFake();
} catch (Exception ignore) {
return null;
}
}
@Override
public DataItem getData(String keyId) {
return null;
}
}
Test:
TestStream<KV<String, Product>> testStream = TestStream.create(KvCoder.of(StringUtf8Coder.of(), SerializableCoder.of(Product.class)))
.addElements(KV.of("25", product)).advanceWatermarkToInfinity();
DataAdapter dataAdapter = new DataAdapter("productCategory", configs);
dataAdapter.serviceApi = new ServiceApiFake();
pipeline.apply(testStream).apply(ParDo.of(dataAdapter));
pipeline.run();
Error in test run: ArgumentExceptionError at Setup init, since it return to calling the regular service at Setup instead of fake.
I thought about going back to testing methods individually by calling "processData" Method, without calling setup, and using a mockito Mock on ProcessContext, ElementInput, and serviceApi when(serviceApi.getData()).thenReturn(..) . This strategy actually works, its not ideal, but allows me to test in some degree.
** I wish Apache Beam team had a better, robust alternative in using fakes, or allow mocks somehow. Or allow DoFnTester again (since its currently deprecated). Company policy does not allow deprecated usage anymore.
Resource: I looked at referring stackoverflow article. Their serviceAPI doesn't have constructor variables, or argument exception validators within the serviceAPI, so it works in their situation.
The text was updated successfully, but these errors were encountered:
Apache Beam recommends to use Fakes instead of Mocks, since Mocks cannot serialize over the pipeline.
I am writing unit tests for older code, where the class is using Apache Beam calling other external class services (which I have not written or cannot change). The classes aren't using interfaces.
Attempt:
I created the fake below. Calling the constructors with nulls will cause Argument Exception errors in the serviceApi constructor. So I tried a factory create pattern which is probably not ideal. (trying to find another solution)
However, the fake keeps getting overriden, since for each ProcessElement, it calls Setup, with reinitializes Parent Class. We had to initialize in each SetupBatch, since serviceApi was not serializable in running Apache Beam application.
Create Fake:
Test:
Error in test run: ArgumentExceptionError at Setup init, since it return to calling the regular service at Setup instead of fake.
I thought about going back to testing methods individually by calling "processData" Method, without calling setup, and using a mockito Mock on ProcessContext, ElementInput, and serviceApi when(serviceApi.getData()).thenReturn(..) . This strategy actually works, its not ideal, but allows me to test in some degree.
** I wish Apache Beam team had a better, robust alternative in using fakes, or allow mocks somehow. Or allow DoFnTester again (since its currently deprecated). Company policy does not allow deprecated usage anymore.
Resource: I looked at referring stackoverflow article. Their serviceAPI doesn't have constructor variables, or argument exception validators within the serviceAPI, so it works in their situation.
The text was updated successfully, but these errors were encountered: