Version 9.0.1

• Dual locking: instead of using one global lock for a joedb file, this version locks head and tail separately. This allows a much nicer handling of concurrent access to files:

  • Journal construction locks the head only, so it does not block if a transaction is in progress or the file was opened in exclusive mode since both of those situations lock the tail only.
  • Concurrent reads use a shared lock on the head of the file to read the checkpoint, and can be blocked only during very short periods of time when a writable journal is constructed or when the checkpoint is modified.

• Write access to exclusive and shared files is completely unified, so Connection and File_Connection can now handle both exclusive and shared files.

• Explicit handling of pull-only connections. The server can now cleanly serve a read-only file.

• New CURL_File allows opening any URL as a read-only file. An http server supporting range access can serve a read-only database.

• SQL dump connection

• Generated code produces an error if compiled with a version of joedb different from the version that was used to generate it.

• Incompatibilities with previous version:

  • In Posix environments, locking changed from using flock to using fcntl. Those two locking mechanisms are not compatible in Linux, so it is important to avoid mixing joedb versions because they may not understand each-other's locks. Windows and MacOS do not have this problem.
  • The network protocol changed to indicate a pull-only connection during handshake, so it is not compatible with the previous version.
  • Local_Connection is removed since it can be replaced by a plain Connection.
  • Pullable_Database is removed and replaced by Readonly_Client.

v8.0.1

Fix missing test for fsync error

Version 8.0.0

• joedb now uses C++17, and is not compatible with C++11 any more.

• joedb_server now takes a client as parameter, instead of a file. This gives much more flexibility, and allows:

  • chaining servers for synchronous remote backup;
  • more generally, creating a tree of multiple database replicas;
  • local programs running on the same machine as the server can access the database file directly, without having to use a network connection to the server.

• Some changes to better handle very large databases:

  • The server does not buffer a whole push before writing it to disk any more. This saves memory and time in case of a large push. If a connection fails in the middle of a push, the written data is not erased. But it is not checkpointed, and may be overwritten by a subsequent push.
  • Better timeouts: the timeout is not for the whole transaction, but for any network activity during the transaction. So, a very long transaction because of a very large push or pull will not time out if there is continuous network activity. Also, previous versions did not check for timeouts in the middle of a push. This is now fixed.

• Performance optimizations:

  • File_Connection is now about 10 times faster.
  • Large string reads are also much faster.
  • Improved networking performance by fusing small consecutive socket writes. Previous version could be hit hard by Nagle's algorithm and delayed ACKs. This version uses ip::tcp::no_delay.

• joedbc produces a new Pullable_Database, similar to Readonly_Database, but the file is not closed at the end of the constructor, and it is possible to pull new data in case of a concurrent update.

• Classes that write a journal (joedb::Writable_Journal, Generic_File_Database, File_Database, Client) now have two extra parameters:

  • check indicates the behaviour in case the file contains an incomplete transaction. It should be equal to either:

    • joedb::Readable_Journal::check::all (the default) fails if the file contains data after the checkpoint.
    • joedb::Readable_Journal::check::overwrite silently overwrite uncheckpointed data.

  • commit_level indicates the default commit level for checkpoints. Default is joedb::Commit_Level::no_commit. See Checkpoints for details.

• id_of_x is now a literal type. All its member functions are constexpr.

• Minor fixes and improvements.

Version 7.1.1: bugfix

Bug fix of previous version: large pulls (>256kb) from a Readonly_File_Connection to an SFTP_File could fail.

Version 7.1.0

New SFTP_File, and Readonly_File_Connection. Combining these classes allows read-only connection to a remote file without running a joedb server on the remote machine. SFTP does not support file locking in practice, so writing via SFTP is not implemented.

Version 7.0.0

• Blobs

• Deep reorganization of concurrency.

  • Ability to safely share a single local replica of a remote server among multiple clients.
  • Tools for efficient remote asynchronous backup.
  • Ability to serve a file read-only.
  • New joedb_push tool, with new interesting features such as the ability to follow a file or push to a remote backup server.

• Removed some useless ssh code that was left over from the old serverless sftp connection. ssh::Thread_Safe_Sesion is renamed to ssh::Session, and its constructor is faster than before because there is no sftp any more.

• No more implicit conversion from compiled row id to integer or boolean. With this new version, explicit methods must be used: is_null() or is_not_null() to test if a reference is null or not, and get_id() to convert to an integer. This ensures stronger typing, and prevents bug-prone implicit conversions.

• Type-safe vector update of references. A range of id_of_x is passed instead of the generic Record_Id.

• No more endianness conversions: on big-endian machines joedb reads and writes data in big-endian format.

• Minor fixes and improvements

Version 6.0.0

• new Local_Connection class for local serverless concurrent access to a file.

• SHA-256 is used to compare the client database with the server database at connection time. This prevents pulling into the wrong file. This also allows making offline modifications to a local database, and pushing them later to a remote server.

• Re-organize file classes:

  • File_Slice is removed. All file classes can be sliced now.
  • Stream_File uses a std::streambuf instead of std::iostream.
  • new Interpreted_File can read joedbi commands directly.

• Exception-safe transactions: if any exception is thrown by a client while writing, then none of what was written since the previous lock-pull will be pushed to the server. The previous approach, based on a Lock object, was defective and is not available any more.

• New approach to vector updates that allows testing for write errors (previous version wrote data in a destructor, which does not allow testing for errors).

• Databases must be explictly checkpointed before destruction. The destructor won't checkpoint any more, because this would risk checkpointing a failed buffer flush, and because write errors cannot be handled properly in destructors.

• Minor fixes and improvements.

Version 5.0.1

Fix compilation with clang++

Version 5.0

• Big improvements to concurrency: joedb now has a network protocol, and a server for efficient and reliable communication.
• The old serverless SSH_Connection was removed completely. It was inefficient and unreliable. Using the server is much better.
• Performance improvements of operations on large vectors.
• vim syntax highlighting for joedbi and joedbc files.
• Minor fixes and improvements.

Version 4.0

• Concurrency: a new mechanism to allow multiple distributed processes to access the same remote database.
• File sharing now works in Windows (one process can read a file while another is writing it, but two processes cannot open the same file for writing).
• joedb_embed compiles the content of a database into a C++ string literal.
• support for generating code inside a nested namespace (namespace deeply::nested::tutorial in the .joedbc file).
• .deb packages are provided for easy installation.
• Minor fixes and improvements

Documentation: https://www.remi-coulom.fr/joedb/index.html