Marekhorst 1434 reimplement context importer module by replacing is lookup with RESTful endpoint #1437
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…kup with RESTful endpoint WIP. Proper error code handling needs to be introduced once RESTful context API returns an appropriate HTTP error code. Introducing RESTful context API based concepts importer module. New paramater is required to enable RESTful endpoint based import: `import_context_service_location`. If the parameter is not provided then ISLookup-based importer mode will be enabled making `import_islookup_service_location` parameter required in such case.
…kup with RESTful endpoint Proper handling of non-existing context by addressing 404 HTTP code. Adding relevant test cases.
marekhorst
changed the title
mpol
requested changes
Oct 12, 2023
Comment on lines
6
to
8
| import static eu.dnetlib.iis.wf.importer.concept.ISLookupServiceBasedConceptImporter.CONCEPT_COUNTER_NAME; | ||
| import static eu.dnetlib.iis.wf.importer.concept.ISLookupServiceBasedConceptImporter.PARAM_IMPORT_CONTEXT_IDS_CSV; | ||
| import static eu.dnetlib.iis.wf.importer.concept.ISLookupServiceBasedConceptImporter.PORT_OUT_CONCEPTS; |
| * ISLookup mock providing static concept profiles. | ||
| * | ||
| */ | ||
| private static class MockISLookupFacade implements ContextStreamingFacade { |
| * ISLookup mock providing static concept profiles. | ||
| * | ||
| */ | ||
| private static class MockISLookupFacade implements ContextStreamingFacade { |
| * ISLookup mock providing static concept profiles. | ||
| * | ||
| */ | ||
| private static class MockISLookupFacade implements ContextStreamingFacade { |
| * @throws MalformedURLException | ||
| */ | ||
| public ContextUrlStreamingFacade(String endpointLocation, | ||
| int readTimeout, int connectionTimeout) throws MalformedURLException { |
There was a problem hiding this comment.
Doesn't actually throw the exception.
| //------------------------ PRIVATE -------------------------- | ||
|
|
||
| private static String buildUrl(String endpointLocation, String contextId) { | ||
| return endpointLocation + "/" + contextId; |
There was a problem hiding this comment.
In general xxx//yyy in an URL is not the same as xxx/yyy, but it seems the exporter API returns the same result in both cases, so maybe it's not that important to worry about it here.
There was a problem hiding this comment.
I agree, the best way would be to rely on a UriTemplate or RestTemplate (or even more generic StringSubstitutor) and provide a template instead of endpointLocation but it seems it is not available among included dependencies and I think for this particular, rather static case, I wouldn't overcomplicate the final solution.
I'd stick to simple check regarding the trailing slash e.g.:
private static String buildUrl(String endpointLocation, String contextId) {
StringBuilder urlBuilder = new StringBuilder(endpointLocation);
if (!endpointLocation.endsWith(URL_SLASH)) {
urlBuilder.append(URL_SLASH);
}
urlBuilder.append(contextId);
return urlBuilder.toString();
}
and call it a day.
There was a problem hiding this comment.
Extracted this method to ContextUrlStreamingFacadeUtils and prepared a dedicated test class ContextUrlStreamingFacadeUtilsTest.
…kup with RESTful endpoint Introducing PR related fixes.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces an additional support for RESTful streaming endpoint as one another source providing context details in JSON format, alongside already available ISLookup based importer which is still to be maintained until we are ready to dismiss it (#1435).
An appropriate importer module is to be automatically selected relying on
import_context_service_locationparameter availability.