Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

channels-1094: Refactoring diffs #2324

Closed
wants to merge 188 commits into from
Closed

Conversation

pmunin
Copy link
Member

@pmunin pmunin commented Aug 21, 2024

A summary of your pull request, including the what change you're making and why.

Testing

Include any additional information about the testing you have completed to
ensure your changes behave as expected. For a speedy review, please check
any of the tasks you completed below during your testing.

  • Added unit tests for new functionality
  • Tested end-to-end using the local server
  • [Segmenters] Tested in the staging environment

@pmunin pmunin requested a review from a team as a code owner August 21, 2024 17:36
@pmunin pmunin requested review from cogwizzle and removed request for a team August 21, 2024 17:36
Comment on lines 520 to 525
function getOrCatch<T>(getValue: () => T): ValueOrError<T>
function getOrCatch<T>(getValue: () => Promise<T>): Promise<ValueOrError<T>>
function getOrCatch<T>(getValue: () => T | Promise<T>): ValueOrError<T> | Promise<ValueOrError<T>> {
try {
const value = getValue()
if (value instanceof Promise) {
return value.then((value) => ({ value })).catch((e) => ({ error: e }))
}
return { value }
} catch (error) {
return { error }
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After working with this abstraction I like that it works in a similar way to how Rust Results works, but I fear that it does deviate from the normal JavaScript standard way of operating quite significantly. I'm not sure how transparent this abstraction is in comparison to the traditional try catch statements.

I'm not saying we can't use this, but what I do think is this pattern needs to be more carefully considered before we introduce it. I think if we do decide to go this route we should likely include this proposed pattern to a larger audience as well.

Comment on lines 204 to 262
if (!result) {
throw new Error('Cache error: no result and no error')
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to add this in order to prevent errors as result can be undefined according to the type. We could probably fix this type issue, but I think it is due to the ValueOrError type.

)
}

async getOrAddCache<T>(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't hate the reusable nature of this function. It feels a bit out of place in a few ways.

  1. I think the function is doing to much. If we want to think about this in an abstract way we should have a function that handles the retrieval of the cache and then have a strategy function to route to either returning the cached value or another function which executes the call and then adds it to a cache. Writing test for this function would be very difficult.

  2. This feels like a very OOP approach to this problem where we've written a generic class in one function. Should we consider building that class?

Copy link

@iamsuneeth iamsuneeth Aug 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a first time reviewer ( and hence inexperienced) of this repo:

I second @cogwizzle 's opinion here. While the function is keeping itself generic, I feel it is doing too much and the multi level getOrCatch conditions seems to be affecting the readability of the code. The benefit of having a getOrCatch over multiple try catch statements is not clear enough to me.

We could go with a generic class as @cogwizzle suggested.
I was also thinking whether we could inject a cache to this function and then this function can be used to handle any caching scenario in future and is not tied to engageDestinationCache and may help in doing away with definite assignment assertion. Probably an overkill if we don't anticipate a reuse of this function in future though

Copy link
Member Author

@pmunin pmunin Aug 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

feel free to suggest your implementation that will keep the same clarity of where the error came from

@cogwizzle
Copy link
Contributor

Fixed the last batch of TypeScript issues. I think that if we're going to use patterns like this one we should be very strict with types and ensure that we're writing completely type complete code even though it increases the verbosity of our code. The language patterns you've introduced mirror very strongly typed languages and the more TS hacks we use the more likely it is to break.

Copy link
Member Author

@pmunin pmunin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cogwizzle plz see comments. I didn't find bugs you were talking about, let's zoom after meetings

Comment on lines 149 to 152
const engageDestinationCache = this.engageDestinationCache
if (!engageDestinationCache) return createValue()

const cacheRead = await getOrCatch(() => engageDestinationCache.getByKey(key))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

too much code to make typescript happy for questionable value coming out of it

Suggested change
const engageDestinationCache = this.engageDestinationCache
if (!engageDestinationCache) return createValue()
const cacheRead = await getOrCatch(() => engageDestinationCache.getByKey(key))
if (!this.engageDestinationCache) return createValue()
const cacheRead = await getOrCatch(() => this.engageDestinationCache!.getByKey(key))

if (!engageDestinationCache) return createValue()

const cacheRead = await getOrCatch(() => engageDestinationCache.getByKey(key))
const cacheReadValue = 'value' in cacheRead ? cacheRead.value : undefined
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here - too much code and complexity for questionable value. increases cognitive load is the price we pay for perfectionism :)

Suggested change
const cacheReadValue = 'value' in cacheRead ? cacheRead.value : undefined
const cacheReadValue = cacheRead.value

Comment on lines 157 to 166
} else if (cacheReadValue) {
let parsedCache: ValueOrError<T> | undefined = undefined
let parsingError: any | undefined = undefined
const result = getOrCatch(() => serializer.parse(cacheReadValue))
if ('error' in result) {
parsingError = result.error
}
if ('value' in result) {
parsedCache = result.value || undefined
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here. this one line
const { value: parsedCache, error: parsingError } = getOrCatch(() => serializer.parse(cacheRead.value!)) became this large paragraph with bunch of explicit typings just to make typescript happy

Comment on lines 176 to 181
if ('error' in parsedCache) {
this.statsIncr('cache_hit', 1, [`cached_error:${!!parsedCache?.error}`])
if (parsedCache?.error) throw parsedCache.error
}
if ('value' in parsedCache) {
return parsedCache.value
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here. Also cache_hit metric tag does not make sense in this implementation. I think cache_hit should include erroneous value which we could not parse

Comment on lines 204 to 195
const result = getOrCatch(() => this.engageDestinationCache?.setByKey(key, stringified.value))
if ('error' in result && result.error) {
this.logError('cache_saving_error', { key, error: result.error })
this.statsIncr('cache_saving_error')
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bug in the code and notably happy typescript didn't help :-)

Suggested change
const result = getOrCatch(() => this.engageDestinationCache?.setByKey(key, stringified.value))
if ('error' in result && result.error) {
this.logError('cache_saving_error', { key, error: result.error })
this.statsIncr('cache_saving_error')
}
const result = await getOrCatch(() => this.engageDestinationCache?.setByKey(key, stringified.value))
if ('error' in result && result.error) {
...
}

also var name result is confusable with line 193. And again, this is less readable code, comparing to what it was:

      const { error: cacheSavingError } = await getOrCatch(() => this.engageDestinationCache!.setByKey(key, stringified.value!)
      if (cacheSavingError) {
          ...
      }
      

Comment on lines 525 to 527
type Ok<T> = { value: T }
type Err<E> = { error: E }
type ValueOrError<T> = Ok<T> | Err<T>
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks unnecessary to add so much types, i'd just keep it as it was

Suggested change
type Ok<T> = { value: T }
type Err<E> = { error: E }
type ValueOrError<T> = Ok<T> | Err<T>
type ValueOrError<T> = { value?: T; error?: any }

Copy link

codecov bot commented Aug 21, 2024

Codecov Report

Attention: Patch coverage is 96.82540% with 2 lines in your changes missing coverage. Please review.

Project coverage is 77.89%. Comparing base (d682103) to head (84f4fe8).
Report is 73 commits behind head on channels-1094.

Files Patch % Lines
...ssaging-twilio/__tests__/__helpers__/test-utils.ts 71.42% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@                Coverage Diff                @@
##           channels-1094    #2324      +/-   ##
=================================================
+ Coverage          77.47%   77.89%   +0.42%     
=================================================
  Files                905      944      +39     
  Lines              15584    17384    +1800     
  Branches            2853     3372     +519     
=================================================
+ Hits               12074    13542    +1468     
- Misses              2574     2904     +330     
- Partials             936      938       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

itsarijitray and others added 28 commits September 23, 2024 11:22
* Test commit

* Test commit

* Test commit

* Remove Faulty Test Assetions
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* index

* add fields

* start creating email upload function

* add updateAudience endpoint

* add logic to know which payload arrays to push

* audience update

* testing permissions to collab

* fix array bug and add dynamic ID field

* testing

* resolving errors and adding types + refactor

* few more changes

* generating types

* committing changes

* refactor

* added functions back into the refactored files

* cleaned out excess logs

* tested locally

* removing token

* adding a unit test

* add more test cases and fix email hashing bug

* removing snapshot tests

---------

Co-authored-by: Austin Hsueh <[email protected]>
Co-authored-by: Manik Mehta <[email protected]>
Co-authored-by: ManikMM <[email protected]>
Co-authored-by: Joe Ayoub <[email protected]>
* Postscript initial commit

* progress from working session

* Initial completed version of syncAudiences with tests

* minor refactor to get CI to pass

* minor refactor

---------

Co-authored-by: Joe Ayoub <[email protected]>
* moving pixel id to settings

* updating tests
* adding 2 tests for custom events

* adding more tests for customEvents

* Add subscription metadata to execute bundle

* starting to add caching

* reverting cache stuff

* reverting cache

* dfsdf

* saving progress

* not test - caching for custom function

* tested locally - ci failing

* adding instrumentation

* cache test added

* fixing auth token

* updating readme

---------

Co-authored-by: Varadarajan V <[email protected]>
* Braze syncmode for createAlias action

* Fixes build; generates types

* Splits off V2 action into it's own directory

* Updates snapshot for v1 action unit tests

* Removes unneeded git diffs

* Updates unit test with better test names and a check for the expected error message when testing if syncMode != add

* Updates generated snapshot with new unit test names
* list_id fields added in klaviyo

* decriptions changed

* Unit test cases added
…rt (#2440)

* Duplicate action:

* Add syncMode

* Add snapshot tests

* fix unit test

* Remove update_existing_only field

* generate types

* removing duplicate registration

---------

Co-authored-by: Joe Ayoub <[email protected]>
Co-authored-by: Joe Ayoub <[email protected]>
* Track Purchase V2

* Track Purchase V2 unit tests

---------

Co-authored-by: Joe Ayoub <[email protected]>
Squash of these commits:

AAD-427 Added new identify action

AAD-427 Added new track action

AAD-427 Added new group action

AAD-427 Added new group action

AAD-427 Added new page action

AAD-427 Added new screen action

AAD-427 Removed generic postToAccoil action

AAD-427 Fleshed out identify call

AAD-427 Fleshed out group call

AAD-427 Fleshed out track call

AAD-427 Added extra test to ensure proper type is forwarded

AAD-427 Fleshed out page call

AAD-427 Fleshed out screen call

AAD-427 Commonised timestamp field

AAD-427 Optionally send traits object

AAD-427 Added util functions to handle staging keys and urls

AAD-427 Removed unnecessary auth header in testAuthentication

It calls extendRequest which was already setting it

AAD-427 Description changes, added accountStatus to identify

AAD-427 Timestamp snapshot fix

AAD-427 Correctly filter group calls
* Track Event V2

* Passes syncMode to sendTrackEvent

* Updates snapshot unit tests for trackEvent2

* Updates unit test with better test names and a check for the expected error message when testing if syncMode != add or upsert; regenerates snapshots

* Update packages/destination-actions/src/destinations/braze/trackEvent2/index.ts

Co-authored-by: maryamsharif <[email protected]>

* Update packages/destination-actions/src/destinations/braze/trackEvent2/index.ts

Co-authored-by: maryamsharif <[email protected]>

* Updates unit test case with correct expected error message

---------

Co-authored-by: maryamsharif <[email protected]>
Co-authored-by: Joe Ayoub <[email protected]>
* Braze syncmode for identifyUser action

* Fixes a build error

* Generates types

* Splits off V2 action into it's own directory

* Manyally updates updateUserProfile snapshot test to hopefully pass?

* Removes unneeded git diffs

* Updates unit test with better test names and a check for the expected error message when testing if syncMode != add or upsert

---------

Co-authored-by: Joe Ayoub <[email protected]>
Co-authored-by: Manik Mehta <[email protected]>
* fixing breaking snapshots for accoil

* another fix for bad snapshot for Accoil

* update
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@pmunin
Copy link
Member Author

pmunin commented Sep 24, 2024

Closing towards #2325

@pmunin pmunin closed this Sep 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.