Skip to content
This repository has been archived by the owner on Jan 13, 2025. It is now read-only.

initial stab at new connector for kinesis streams #58 #102

Merged
merged 2 commits into from
Nov 17, 2022

Conversation

rbraley
Copy link
Contributor

@rbraley rbraley commented Oct 28, 2022

This is the general structure of the Connector for Kinesis data streams, however there are some issues which ought to be resolved. Addresses issue #58

  • proper tests
  • handling of generic records
  • not leaking underlying library types in the signature of ProducerRecords
  • Read side as well.

@CLAassistant
Copy link

CLAassistant commented Oct 28, 2022

CLA assistant check
All committers have signed the CLA.

@rbraley rbraley marked this pull request as draft October 28, 2022 04:53
@adrianfilip
Copy link
Contributor

Looks great 🙏 . Only a few minor observations.

@rbraley rbraley marked this pull request as ready for review October 28, 2022 16:54

trait KinesisDataStreamsConnector[T] {

def sinkChunked(implicit

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👋 zio-kinesis author here.

I think the sinkChunked on the Producer could be changed to a non-chunked version, since ZStreams are chunked under the hood anyway. svroonland/zio-kinesis#791

I would recommend to change that here too, to not expose the Chunks in the interface.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@svroonland I will merge this PR and apply your suggestions in a separate PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@svroonland That PR change is not released yet. We will keep an eye on the releases and update it once 0.30.1 is out.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes, I will try to make a release this week and will let you know here.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adrianfilip I just released 0.30.1, should be on Maven soon.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@svroonland Fantastic. Will upgrade shortly.

@adrianfilip adrianfilip merged commit 04fe666 into zio-archive:master Nov 17, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants