New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[wip] - do-not-merge - Add test PlacementTableVersionNotEventuallyIncreased #7571
base: master
Are you sure you want to change the base?
Conversation
Signed-off-by: ItalyPaleAle <[email protected]>
Signed-off-by: ItalyPaleAle <[email protected]>
Signed-off-by: ItalyPaleAle <[email protected]>
Signed-off-by: ItalyPaleAle <[email protected]>
Signed-off-by: ItalyPaleAle <[email protected]>
Signed-off-by: ItalyPaleAle <[email protected]>
Signed-off-by: ItalyPaleAle <[email protected]>
Signed-off-by: Lee Fowler <[email protected]>
Signed-off-by: Lee Fowler <[email protected]>
…apr into fowlerlee-7490-placement-api
…lerlee/dapr into fowlerlee-7490-placement-api
…apr into fowlerlee-7490-placement-api
Signed-off-by: Guido Spadotto <[email protected]> Co-authored-by: Guido Spadotto <[email protected]>
pkg/api/errors/placement.go
Outdated
kiterrors "github.com/dapr/kit/errors" | ||
) | ||
|
||
func PlacementTableVersionNotEventuallyIncreased(tableVersion int, msg string) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hey @fowlerlee - so you'll want to call this func in place of where the relevant err is thrown. Here is a direct example of me doing that for the pubsub api, but youll need to find where this error is thrown in dapr and do the same thing. You'll want to pass in all relevant err details like maybe the appID (which you can typically find off the universal struct) and any other metadata that may be relevant to a user. Specifically, for the kiterrors code - you'll need to open a PR to dapr/kit
to add the placement kiterrors.CodePrefixPlacement
similar to this PR. Then you'll want to use that kit error prefix for the .WithErrorInfo()
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@cicoyle : I added the error handling to this PR, and fixed some typos I had that were not conforming to the format tests. The issue still requires integration tests, but I'm not certain the form that these should take, since many already exist? I also 'git rebase' to get the master into the feature branch, hence the many commits from the master to this issue PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cc @cicoyle
* ci: add 'mikeee' to dapr-bot owners Signed-off-by: mikeee <[email protected]> * fix: alphabetise list of bot owner gh handles Signed-off-by: mikeee <[email protected]> --------- Signed-off-by: mikeee <[email protected]>
* Make injector resilient to sentry unavailability (dapr#7507) * make injector resilient to sentry unavailability Signed-off-by: yaron2 <[email protected]> * remove redundant line Signed-off-by: yaron2 <[email protected]> --------- Signed-off-by: yaron2 <[email protected]> * sentry retry up to 30s (dapr#7508) Signed-off-by: yaron2 <[email protected]> * Update contrib to 1.13.0-rc.3 (dapr#7509) * update contrib to 1.13.0-rc.2 Signed-off-by: yaron2 <[email protected]> * update to rc.3 Signed-off-by: yaron2 <[email protected]> --------- Signed-off-by: yaron2 <[email protected]> * Injector: add option to add `DAPR_HOST_IP` env var to daprd (dapr#7511) The `DAPR_HOST_IP` env var is used in various places in Dapr for a sidecar to know its own IP address, for example for service invocation or actor invocation. When using the Dapr injector to add the daprd container, we can use the downstream APIs to add the `DAPR_HOST_IP` env var based on data from the controller This option can be enabled by setting the Helm option `dapr_sidecar_injector.enableK8sDownwardAPIs=true` Signed-off-by: ItalyPaleAle <[email protected]> * Fix state encryption regression + add integration test (dapr#7517) * fix state encryption regression + add intg test Signed-off-by: yaron2 <[email protected]> * linter Signed-off-by: yaron2 <[email protected]> * review feedback Signed-off-by: yaron2 <[email protected]> * linter Signed-off-by: yaron2 <[email protected]> * just use sha256, remove nolint:gosec Signed-off-by: yaron2 <[email protected]> --------- Signed-off-by: yaron2 <[email protected]> * Test Integration: speed up tests 10% (dapr#7528) * Test Integration: speed up tests 10% Speed up integration tests by changing poll intervals `100*time.Millisecond` to `10*time.Millisecond`. ~4.50m to ~4.20m Signed-off-by: joshvanl <[email protected]> * Wait for operator healthz before exiting to ensure no exit error Signed-off-by: joshvanl <[email protected]> --------- Signed-off-by: joshvanl <[email protected]> * Revert selfhosted disk loader to not respect namespace (dapr#7527) * Revert selfhosted disk loader to not respect namespace Revert the selfhosted component disk loader to not respect the namespace. This will allow the selfhosted disk loader to load components from any namespace from file. Fixes dapr#7523 Signed-off-by: joshvanl <[email protected]> * Revert component disk loader behaviour to respect component namespace when NAMESPACE env var is set. Signed-off-by: joshvanl <[email protected]> --------- Signed-off-by: joshvanl <[email protected]> * [1.13] Add warning that Dapr state store encryption could lead to catastrophic failures (dapr#7524) * [1.13] Add warning that Dapr state store encryption could lead to catastrophic failures If the same key is used to encrypt more than 2^32 values (ie. more than 2^32 "Save" operations), it can lead to the private keys being exposed. See: dapr#6027 Signed-off-by: ItalyPaleAle <[email protected]> * Changed per review feedback Signed-off-by: ItalyPaleAle <[email protected]> --------- Signed-off-by: ItalyPaleAle <[email protected]> * Add content-length to http channel (dapr#7537) * add content-length to http channel Signed-off-by: yaron2 <[email protected]> * update tests Signed-off-by: yaron2 <[email protected]> --------- Signed-off-by: yaron2 <[email protected]> * Updates components-contrib to 1.13.0-rc.4 (dapr#7538) * Updates components-contrib to 1.13.0-rc.3 Signed-off-by: joshvanl <[email protected]> * mod-tidy-all Signed-off-by: joshvanl <[email protected]> --------- Signed-off-by: joshvanl <[email protected]> Co-authored-by: Yaron Schneider <[email protected]> * Subscriptions: Fix panic when match rule is empty (dapr#7539) Prevents a panic resulting when a subscription route rule match (`spec.routes.rules.match`) is empty. An empty match rule is valid and is considered "default". Error occurs because of Go interface vs implementation struct pointer nil check foot-gun. Adds integration tests for both HTTP and gRPC subscribers. Signed-off-by: joshvanl <[email protected]> * Hot Reloading: don't watch files if not enabled (dapr#7521) * Hot Reloading: don't watch files if not enabled Update hot reloading so that we don't setup file watchers on the component directory if hot reloading is not enabled. Signed-off-by: joshvanl <[email protected]> * Remove `Close` paradigm from hot reloader Signed-off-by: joshvanl <[email protected]> * Linting Signed-off-by: joshvanl <[email protected]> --------- Signed-off-by: joshvanl <[email protected]> Co-authored-by: Yaron Schneider <[email protected]> * Remove pubsub content-length test which has been removed from (dapr#7550) release-1.13 Signed-off-by: joshvanl <[email protected]> * Revert ApiLevel controlling vnodes back to context metadata (dapr#7547) * Revert ApiLevel controlling vnodes back to context metadata Signed-off-by: Elena Kolevska <[email protected]> * Lint Signed-off-by: Elena Kolevska <[email protected]> * Fixes after review Signed-off-by: Elena Kolevska <[email protected]> * Set back api level to 10, just for completeness purposes Signed-off-by: Elena Kolevska <[email protected]> * Don’t specify APILevelSpecify api level Signed-off-by: Elena Kolevska <[email protected]> * Apply suggestions from code review Co-authored-by: Josh van Leeuwen <[email protected]> Signed-off-by: Elena Kolevska <[email protected]> * Cleanup after review Signed-off-by: Elena Kolevska <[email protected]> * Small refactor Signed-off-by: Elena Kolevska <[email protected]> * Refactor multiple parameters into a request object for placement table dissemination Signed-off-by: Elena Kolevska <[email protected]> --------- Signed-off-by: Elena Kolevska <[email protected]> Signed-off-by: Elena Kolevska <[email protected]> Co-authored-by: Josh van Leeuwen <[email protected]> Co-authored-by: Artur Souza <[email protected]> * Actor Reminders: Default JSON serialization. (dapr#7548) * Actor Reminders: Default JSON serialization. To support downgrades to 1.12 from 1.13, this PR changes the reminder serialization storage format back to JSON by default. This means a 1.12 actor reminder client can read reminders written by 1.13 actors. 1.13 will continue to understand both JSON and protobuf. Protobuf serialization can be enabled with the `ActorReminderStorageProtobuf` feature gate. The actor "API Level" has been changed back to 10. Adds test to ensure the default serialization is JSON. Signed-off-by: joshvanl <[email protected]> * Remove ActorReminderStorageProtobuf feature gate in favour of using API level Signed-off-by: joshvanl <[email protected]> * Fix api level tests Signed-off-by: joshvanl <[email protected]> --------- Signed-off-by: joshvanl <[email protected]> Co-authored-by: Yaron Schneider <[email protected]> * Update contrib to 1.13.0-rc.6 (dapr#7553) * update contrib to 1.13.0-rc.5 Signed-off-by: yaron2 <[email protected]> * update to rc.6 Signed-off-by: yaron2 <[email protected]> --------- Signed-off-by: yaron2 <[email protected]> * Updates components-contrib to 1.13.0-rc.7 (dapr#7562) Signed-off-by: joshvanl <[email protected]> * update contrib to 1.13.0-rc.8 (dapr#7567) * Add metadata in binding response even in case of error (dapr#7572) * Add binding metadata even in case of error. Signed-off-by: Artur Souza <[email protected]> * ADD IT. Signed-off-by: Artur Souza <[email protected]> * Fix lint. Signed-off-by: Artur Souza <[email protected]> --------- Signed-off-by: Artur Souza <[email protected]> * [Release-1.13] Upgrade to contrib `v1.13.0-rc.10` (dapr#7577) * Upgrade to contrib 1.13.0-rc.9 Signed-off-by: Bernd Verst <[email protected]> * Use contrib 1.13.0-rc.10 for latest kafka sarama patch version Signed-off-by: Bernd Verst <[email protected]> --------- Signed-off-by: Bernd Verst <[email protected]> * Fix for issue 7576 (dapr#7581) (dapr#7587) Signed-off-by: Guido Spadotto <[email protected]> Co-authored-by: Guido Spadotto <[email protected]> Co-authored-by: Guido Spadotto <[email protected]> * Adds v1.13.0 release notes (dapr#7586) * Adds v1.13.0 release notes Adds docs/release_notes/v1.13.0.md Signed-off-by: joshvanl <[email protected]> * Update docs/release_notes/v1.13.0.md Co-authored-by: Paul Yuknewicz <[email protected]> Signed-off-by: Josh van Leeuwen <[email protected]> * Move Actor Reminder Performance higher in notes Signed-off-by: joshvanl <[email protected]> * Update v1.13.0.md Signed-off-by: Artur Souza <[email protected]> --------- Signed-off-by: joshvanl <[email protected]> Signed-off-by: Josh van Leeuwen <[email protected]> Signed-off-by: Artur Souza <[email protected]> Co-authored-by: Paul Yuknewicz <[email protected]> Co-authored-by: Artur Souza <[email protected]> * chore: bump go to 1.21.8 and protobuf lib to 1.33.0 (dapr#7591) * ci: force go1.21.8 in workflows instead of go.mod Signed-off-by: mikeee <[email protected]> * chore: bump google.golang.org/protobuf to 1.33.0 Signed-off-by: mikeee <[email protected]> * make modtidy-all Signed-off-by: Artur Souza <[email protected]> --------- Signed-off-by: mikeee <[email protected]> Signed-off-by: Artur Souza <[email protected]> Co-authored-by: Artur Souza <[email protected]> --------- Signed-off-by: yaron2 <[email protected]> Signed-off-by: ItalyPaleAle <[email protected]> Signed-off-by: joshvanl <[email protected]> Signed-off-by: Elena Kolevska <[email protected]> Signed-off-by: Elena Kolevska <[email protected]> Signed-off-by: Artur Souza <[email protected]> Signed-off-by: Bernd Verst <[email protected]> Signed-off-by: Guido Spadotto <[email protected]> Signed-off-by: Josh van Leeuwen <[email protected]> Signed-off-by: mikeee <[email protected]> Co-authored-by: Yaron Schneider <[email protected]> Co-authored-by: Alessandro (Ale) Segala <[email protected]> Co-authored-by: Elena Kolevska <[email protected]> Co-authored-by: Artur Souza <[email protected]> Co-authored-by: Bernd Verst <[email protected]> Co-authored-by: Guido Spadotto <[email protected]> Co-authored-by: Guido Spadotto <[email protected]> Co-authored-by: Paul Yuknewicz <[email protected]> Co-authored-by: Mike Nguyen <[email protected]>
) This reverts commit 1421069.
* make injector resilient to sentry unavailability Signed-off-by: yaron2 <[email protected]> * remove redundant line Signed-off-by: yaron2 <[email protected]> --------- Signed-off-by: yaron2 <[email protected]>
Signed-off-by: yaron2 <[email protected]>
* update contrib to 1.13.0-rc.2 Signed-off-by: yaron2 <[email protected]> * update to rc.3 Signed-off-by: yaron2 <[email protected]> --------- Signed-off-by: yaron2 <[email protected]>
The `DAPR_HOST_IP` env var is used in various places in Dapr for a sidecar to know its own IP address, for example for service invocation or actor invocation. When using the Dapr injector to add the daprd container, we can use the downstream APIs to add the `DAPR_HOST_IP` env var based on data from the controller This option can be enabled by setting the Helm option `dapr_sidecar_injector.enableK8sDownwardAPIs=true` Signed-off-by: ItalyPaleAle <[email protected]>
chore: convert crypto APIs to net/http
Before v1.12, Daprd's would connect to the sentry service on port 80. This was changed to 443 in v1.12, but we kept the port 80 open for backwards compatibility. This PR removes the port 80 from the sentry service. Signed-off-by: joshvanl <[email protected]> Co-authored-by: Dapr Bot <[email protected]>
* Refactor api/errors into builders Refactors api/errors into functional builders. The current API for api/errors is heavily string concatenation based with long parameter arguments. This is fragile/error prone to the developer and difficult to understand for the consumer. This change should make it clearer to the reader/consumer what the intended errors are and how to use them. Moving implementation details of the error protocol codes into the error package also helps to clean up API implementation code. Signed-off-by: joshvanl <[email protected]> * Remove use of `stderror` `errors` alias Signed-off-by: joshvanl <[email protected]> --------- Signed-off-by: joshvanl <[email protected]> Co-authored-by: Dapr Bot <[email protected]>
Signed-off-by: ItalyPaleAle <[email protected]>
* Test Integration: reserve ports in ordered blocks Dapr opens a lot of ports. Integration tests use upwards of a 1000 ports through a run. Tests are run in parallel and ports are reserved on demand through port 0 then being released before being used by a test. There was a high chance that a port would attempted to be used by multiple tests/processes at the same time causing conflict errors and false test failures. To prevent these conflicts, the ports utils has been moved to a process which will reserve 300 blocks of ports to be used by tests. These ports are reserved by incrementing through port ranges 1024 to 65535 in a ring. This always prevents conflicts on this test runner, and should also decrease port conflicts across multiple simultaneous test runners. Signed-off-by: joshvanl <[email protected]> * Rename intPort to port Signed-off-by: joshvanl <[email protected]> --------- Signed-off-by: joshvanl <[email protected]> Co-authored-by: Loong Dai <[email protected]> Co-authored-by: Dapr Bot <[email protected]>
Adds youtube link to a talk about the integration tests which might be useful for developers to learn more. Signed-off-by: joshvanl <[email protected]> Co-authored-by: Dapr Bot <[email protected]>
* Runtime Manifest Loader Refactor The current implementation of Component and HTTPEndpoint disk & Kubernetes manifest loader spanned across disjointed packages with semi generic functionality. PR continues the existing generic manifest loader pattern with a shared internal package. Moves both Component and HTTPEndpont loader implementations into shared internal generic package. Existing Subscription, Configuration, and Resiliency loaders are left as-is but should will be brought into this shared generic loader in future PRs. Signed-off-by: joshvanl <[email protected]> * Adds namespace filter to disk manifest loader Signed-off-by: joshvanl <[email protected]> * Fix disk loader unit test Signed-off-by: joshvanl <[email protected]> --------- Signed-off-by: joshvanl <[email protected]> Co-authored-by: Dapr Bot <[email protected]>
It is sometimes the case that Dapr services are suspended for a period of time, for example when developer closes their laptop over night, before being resumed after a significant period of time. Due to the renewal timer being a Go Timer, it will also be suspended during this time and only resume after the process is resumed. Because X.509 certificates expiration validity follow wall-clock time and the renewal timer follows process up-time time, it is possible for a Dapr identity certificates to expire and not be renewed until enough observed time has passed for the renewal timer to trigger. This commit introduces a safety check whereby the renewal timer will always trigger either every minute or at renewal time, whichever is sooner. Once ticked, the renew loop will check to ensure whether the renewal time has actually been reached before renewing. This change helps out developers by not needing to restart the Dapr cluster when they suspend their Dapr clusters for a period of time. While it is not expected that this change be useful in production, it is possible users are suspending and resuming virtual machines in production. Adds a renewal integration test to ensure Dapr identity certificate is renewed based on expiration. Signed-off-by: joshvanl <[email protected]> Co-authored-by: Dapr Bot <[email protected]> Co-authored-by: Loong Dai <[email protected]>
Adds support for injector process in integration tests. Adds basic healthz & ports tests for injector. Signed-off-by: joshvanl <[email protected]> Co-authored-by: Dapr Bot <[email protected]>
Use correct free port import and interface in injector integration test process. Signed-off-by: joshvanl <[email protected]>
Signed-off-by: mikeee <[email protected]> Co-authored-by: Josh van Leeuwen <[email protected]> Co-authored-by: Dapr Bot <[email protected]>
…ders chore: Convert actor HTTP APIs off fasthttp
…IRE TLS clients and servers. (dapr#7037) * Removes legacy SPIFFE TLS clients and servers in favour of the new SPIRE TLS clients and servers. Signed-off-by: joshvanl <[email protected]> * Fix sentry int tests, and adds test to ensure legacy ID is not longer accepted Signed-off-by: joshvanl <[email protected]> * String match on sentry Kubernetes validator longname test Signed-off-by: joshvanl <[email protected]> * Fix namespace of sentry in operator tests Signed-off-by: joshvanl <[email protected]> * Linting Signed-off-by: joshvanl <[email protected]> * Update integration kubernetes process to use leaf certificate with cluster.local Signed-off-by: joshvanl <[email protected]> * Fix setting correct control plane trust domain on daprd Signed-off-by: joshvanl <[email protected]> * Remove SENTRY_LOCAL_IDENTITY form expected env var Signed-off-by: joshvanl <[email protected]> * Fix control plane trust domain setting in test Signed-off-by: joshvanl <[email protected]> * Fixes int version skew tests using legacy client/server Signed-off-by: joshvanl <[email protected]> * Fix int version-skew patch on v1.13.0 Signed-off-by: joshvanl <[email protected]> * Use correct namespace for sentry in injector integration tests Signed-off-by: joshvanl <[email protected]> --------- Signed-off-by: joshvanl <[email protected]> Co-authored-by: Dapr Bot <[email protected]> Co-authored-by: Yaron Schneider <[email protected]>
…apr#7657) When block shutdown is enabled, Dapr will block the shutdown sequence for a configurable amount of time or until the app becomes unhealthy. During this time the out-bound Dapr APIs remain available but all in-bound requests are blocked. Currently, in-flight PubSub messages are canceled when Dapr receives a SIGTERM, which can result in lost messages. Patch updates the pubsub subscribe message handler policy runner to execute sending the message to the app channel in a background context. This ensures that when Dapr cancels the Subscription due to a blocking shutdown, the in-flight message is not effected. Enables the `dapr_component_pubsub_egress_bulk_count` and `dapr_component_pubsub_egress_bulk_latency` metrics for bulk pubsub egress which were previously not enabled, as they are used to test that an in-flight message is correctly sent after Dapr has begun to shutdown. Signed-off-by: joshvanl <[email protected]>
…pr#7659) * Fix incorrect content-length being sent to HTTP published message PR dapr#7537 reverse revered the change which removed the content-length from being set on HTTP headers on sent messages. We still need to remove the content-length from messages from pubsub subscriptions as the pubsub may report a message with a content-length which does not actually match the size of the delivered message to the app. content-length is only removed on HTTP published messages. Change should be backported to 1.13 Signed-off-by: joshvanl <[email protected]> * Adds grpc app subscriber to content-length tests Signed-off-by: joshvanl <[email protected]> * Adds tests for content-length gRPC subscribed app Signed-off-by: joshvanl <[email protected]> * framework/socket: skip if test is windows Signed-off-by: joshvanl <[email protected]> * Fix socket runtime GOOS import Signed-off-by: joshvanl <[email protected]> --------- Signed-off-by: joshvanl <[email protected]>
On Darwin (MacOS), `os.TempDir()` nicely returns a truly ephemeral process directory is `/var/folders/...` however in order to reap the benefits of Go cache magic in integration tests we need a slightly more permanent directory, i.e. `/tmp`. This directory will persist across integration test runs and speed up the test execution boot times from s to ms. Signed-off-by: joshvanl <[email protected]>
Signed-off-by: Lee Fowler <[email protected]>
Signed-off-by: Lee Fowler <[email protected]>
Signed-off-by: Lee Fowler <[email protected]>
Signed-off-by: Lee Fowler <[email protected]>
Signed-off-by: Lee Fowler <[email protected]>
It looks like the |
By making a contribution to this project, I certify that:
Description
Add a new test called PlacementTableVersionNotEventuallyIncreased
Signed-off-by: Lee Fowler [email protected]
Issue reference
Issue #7490_#7490 (comment)
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list: