Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RCORE-2058 Adding clickbench importer and query tool for evaluating compression #7664

Merged
merged 62 commits into from May 21, 2024

Conversation

nicola-cab
Copy link
Member

What, How & Why?

In order to evaluate compression results, a new utility has been created for importing clickbench files.
Fixes: https://jira.mongodb.org/browse/RCORE-2076
Depends by: #7571

nicola-cab and others added 30 commits March 25, 2024 14:05
---------

Co-authored-by: Finn Schiermer Andersen <[email protected]>
@nicola-cab nicola-cab changed the base branch from next-major to nc/rcore-2076 May 1, 2024 16:08
@nicola-cab nicola-cab changed the title Adding clickbench importer and query tool for evaluating compression RCORE-2058 Adding clickbench importer and query tool for evaluating compression May 1, 2024
Copy link

Pull Request Test Coverage Report for Build nicola.cabiddu_1689

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 62 unchanged lines in 16 files lost coverage.
  • Overall coverage increased (+0.01%) to 90.521%

Files with Coverage Reduction New Missed Lines %
src/realm/array_mixed.cpp 1 92.47%
src/realm/array_string.cpp 1 87.23%
src/realm/index_string.hpp 1 93.48%
src/realm/query_engine.hpp 1 94.13%
src/realm/query_expression.hpp 1 93.84%
src/realm/util/file.cpp 1 79.15%
test/test_index_string.cpp 1 93.48%
src/realm/sync/client.cpp 3 89.73%
src/realm/sync/instruction_applier.cpp 3 68.98%
src/realm/sync/noinst/protocol_codec.hpp 3 73.5%
Totals Coverage Status
Change from base Build nicola.cabiddu_1688: 0.01%
Covered Lines: 213569
Relevant Lines: 235933

💛 - Coveralls

Base automatically changed from nc/rcore-2076 to next-major May 14, 2024 14:30
const int bufs_per_commit = 100;
while (auto buf = mbx.receive()) {
// t->create_objects(buf);
// for (auto& val : buf->values) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Too many comments to actually import anything.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ops my bad, it should now be OK, although the strings are not going to be compressed right now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to be sure: you want to keep the verification? It will not exactly boost performance.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think, at this stage I do want to verify that the columns are correct, we may disable it in the future.

@nicola-cab
Copy link
Member Author

@finnschiermer @jedelbo can we get this reviewed, thanks :-)

@nicola-cab nicola-cab requested a review from jedelbo May 20, 2024 09:47
Copy link
Contributor

@jedelbo jedelbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved in case you want to keep the verification stuff.

@nicola-cab nicola-cab merged commit a02798a into next-major May 21, 2024
34 of 37 checks passed
@nicola-cab nicola-cab deleted the nc/RCORE-2058 branch May 21, 2024 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants