Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DMS-92] Basic eTag creation #381

Merged
merged 9 commits into from
Jan 9, 2025
Merged

[DMS-92] Basic eTag creation #381

merged 9 commits into from
Jan 9, 2025

Conversation

simpat-adam
Copy link
Contributor

No description provided.

@@ -20,6 +21,10 @@ public async Task Execute(PipelineContext context, Func<Task> next)
DateTimeOffset utcNow = DateTimeOffset.UtcNow;
string formattedUtcDateTime = utcNow.ToString("yyyy-MM-ddTHH:mm:ssZ");
context.ParsedBody["_lastModifiedDate"] = formattedUtcDateTime;
context.ParsedBody["_etag"] = DateTime
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mimic'd the way ODS API generates etag.

Copy link
Contributor

@bradbanister bradbanister left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do wonder if it would be better to generate etags from a hash of the document. Since DMS could be running on multiple servers, there's a teeny-tiny chance the clocks are out of sync in just the right way for different documents to have the same timestamp. Just something to think about.

using EdFi.DataManagementService.Core.Pipeline;
using Microsoft.Extensions.Logging;

namespace EdFi.DataManagementService.Core.Middleware
{
internal class InjectLastModifiedDateToEdFiDocumentMiddleware(ILogger _logger) : IPipelineStep
internal class InjectVersionMetadataToEdFiDocumentMiddleware(ILogger _logger) : IPipelineStep
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good name change 😎

namespace EdFi.DataManagementService.Core.External.Model;

// A string type branded as an Etag which uniquely identifies a specific version of a document
public record struct Etag(string Value);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice

IUpsertRequest;
IUpsertRequest
{
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops

@@ -16,7 +16,8 @@ public record UpsertResult
/// A successful upsert request that took the form of an insert
/// </summary>
/// <param name="NewDocumentUuid">The DocumentUuid of the new document</param>
public record InsertSuccess(DocumentUuid NewDocumentUuid) : UpsertResult();
/// <param name="Etag">The Etag of the new document</param>
public record InsertSuccess(DocumentUuid NewDocumentUuid, Etag Etag) : UpsertResult();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does a backend really need to return it? Doesn't core set it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point.

@simpat-adam simpat-adam force-pushed the DMS-92 branch 3 times, most recently from 5fb2e75 to aa177b2 Compare January 9, 2025 18:05
@simpat-adam
Copy link
Contributor Author

I do wonder if it would be better to generate etags from a hash of the document. Since DMS could be running on multiple servers, there's a teeny-tiny chance the clocks are out of sync in just the right way for different documents to have the same timestamp. Just something to think about.

Definitely could be done. Couple thoughts

  • Is it important to be able to re-generate the hash once it's generated? Currently only do this for testing. I wonder about things like the order of nodes when it's serialized to/from json strings, also would have to remove the actual _etag jsonNode from the doc if we were going to try to re-gen the hash.
  • Speed of hashing is a factor because of cascading updates. Not sure if there's a significant difference between hashing the entire document vs the single property.

@bradbanister
Copy link
Contributor

Definitely could be done. Couple thoughts

  • Is it important to be able to re-generate the hash once it's generated? Currently only do this for testing. I wonder about things like the order of nodes when it's serialized to/from json strings, also would have to remove the actual _etag jsonNode from the doc if we were going to try to re-gen the hash.
  • Speed of hashing is a factor because of cascading updates. Not sure if there's a significant difference between hashing the entire document vs the single property.

Good points. Let's just keep it in the back of our minds.

@bradbanister bradbanister merged commit 7759e8f into main Jan 9, 2025
11 checks passed
@bradbanister bradbanister deleted the DMS-92 branch January 9, 2025 18:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants