best way to 'tap' into commited changes

[guykatz]

hi all, i think I already asked something like this or similar but i couldnt find it..
is there a way to 'tap' into the changes that are commited (succesfuly) to the storage?
some sort of a transaction log i can read, some sort of event API i can listen to?
thanks

[hg-ms]

Hello,
There is a small tool to convert the binary content from the transaction log files to text.
To build the TransactionFileConverter as standalone jar you can use the maven profile transaction-file-converter:

eclipse-store\storage\storage>mvn -Ptransaction-file-converter clean package

Now you should find the transaction-file-converter.jar in the target directory.

The transaction log files are the *.sft files you can find in each channel’s directory, to convert them simple pass a *sft file to the transaction-file-converter.jar:

java -jar transaction-file-converter.jar transactions_0.sft

The transaction log files contain a time stamp, the type of change and some file length information.

[guykatz]

thanks @hg-ms
what I am looking for is to know what actually changed in some granularity (prefferably smaller then bigger).
looking at the transaction log as text is see the following info via the header:
Type Timestamp Time Delta (ms) Timestamp(long) Resulting Length Length Change Current Head File Significant File Special Offset
but I dont undertsand how to know what exactly was created, for example, in the following log line:
CREATION 2023-10-24 13:30:33.501 4528 1698143433501000000 0 0 1 2
i understand something was created but how can i know what exactly?
thanks!

[hg-ms]

The log entries a described at the start of StorageTransactionsAnalysis.java
The Creation Entry means that:

• a file was created
• at 2023-10-24 13:30:33.501
• 4528ms after the last log entry
• 1698143433501000000 is the raw timestamp as long
• the new file has a length of 0 bytes
• there was a length change of 0 bytes
• the current had file (the file the channel append data) is file no 1
• the new create file is file no 2

But I guess this is not the kind of changes you are interested in.

There is a way to get all objects that will be persisted, but it is a bit hacky:
To do so you need to create a storer and cast it to BinaryStorer.Default.
Between the store and commit you can retrieve all objects that will be persisted.

BinaryStorer.Default storer = (Default) storage.createLazyStorer();
storer.store(root);
storer.iterateMergeableEntries((oid, instance) -> {System.out.println("oid: " + oid + " instance: " + instance);} );
storer.commit();

[guykatz]

thanks @hg-ms
do you think it might be of interest for you guys to incorporate some sort of a listener API for this inside the code?
I am thinking about potential use cases like outbox pattern, Change Data Capture integration etc...
this would be beneficial I think...

[hg-ms]

You are right, it can be useful to have some convenient option to get the persisted objects.