Add generator metadata to cdxj files produces by ipwb replay

[machawk1]

@ibnesayeed Suggested generator as the cdxj key. This will allow us to migrate the format as we evolve how ipwb interacts with archival indexes. For example, when we change the key per #41, old versions of ipwb will not be able to read cdxj files generated by newer versions and newer versions will need to migrate old versions' cdxj files to work with whatever scheme we use in the future.

A first step in accomplishing this would be to provide the ipwb indexer version as the value of a "generator" key in produce cdxj's metadata.

@ibnesayeed I recall there being discussion of using a leading character other than "@" in cdxj metadata. What is the current one? Where was that discussion? What is the current "standard" was of conveying cdxj metadata?

[ibnesayeed]

The current metadata indicator prefix character is ! as noted in oduwsdl/ORS#6.

There is another specification draft for the CDXJ, especially in context to Open Wayback. This is trying to standardizing various keys. This is still far from done. To track variations, it uses a version number specific to the CDXJ itself, independent of how it was generated.

[machawk1]

@ibnesayeed I had not even considered needing to specify the version of CDXJ that the document represents. Is there an example of how to define this in a reference CDXJ? As mentioned in other tickets, it would be good to have a CDXJ spec/EBNF that we could programmatically reference for validation (like XSLT?).

[ibnesayeed]

That document is still very controversial and version number is one of those things that is not agreed upon yet. The biggest complaint about that was the format as it does not conforms to the CDXJ/ORS line grammar and the other issue was about merging multiple documents with different minor version numbers.

For now, I would suggest you use something like this:

!context ["http://oduwsdl.github.io/contexts/cdxj"]
!meta {"generator": "IPWB v0.1-alpha", "created_at": "2017-01-15T13:15:52Z"}