v0.13.0

Changes

• New format decoders midi, negentropy, tap and txz, see below for details.
• Add byte_array bits format. #992

  # by default binary values are UTF-8 strings
  $ fq -V -n '[1,2,3] | tobytes'
  # with -o bits_format=byte_array they are byte arrays instead
  "\u0001\u0002\u0003"
  $ fq -cV -n -o bits_format=byte_array '[1,2,3] | tobytes'
  [1,2,3]
  
  # data is a string
  $ fq '.program_headers[] | select(.type=="interp")' /bin/ls
  {"align":1,"data":"/lib/ld-musl-x86_64.so.1\u0000", ...}
  # data is a byte array
  $ fq -cV -o bits_format=byte_array '.program_headers[] | select(.type=="interp")' /bin/ls
  {"align":1,"data":[47,108,105,98,47,108,100,45,109,117,115,108,45,120,56,54,95,54,52,46,115,111,46,49,0], ...}

• Go 1.22 or later is now required to build. This is to keep up with build requirements of some dependencies. #1011
• gojq changes from upstream: #1000
  • Implement add/1 function
  • Improve time functions to accept fewer element arrays
  • Fix uri and urid formats not to convert a space between a plus sign

Format changes

• matroska Updated to latest specification. #1010
• midi MIDI decoder added by @twystd. #1004
• negentropy Negentropy message decoder added by @fiatjaf. #1012
• tap and txz TAP and TXZ tape format for ZX Spectrum computers added by @mrcook. #975

Changelog

• 64b0523 Add PBS tidbit 8 episode
• 3105b7d Added to probe group
• d73c663 Update docker-golang to 1.22.6 from 1.22.5
• 790aa8a Update docker-golang to 1.23.0 from 1.22.6
• ea3294f Update docker-golang to 1.23.1 from 1.23.0
• 34b2cc3 Update github-go-version to 1.22.6 from 1.22.5
• 8bb841f Update github-go-version to 1.23.0 from 1.22.6
• 2c38cf1 Update github-golangci-lint to 1.60.1 from 1.59.1
• f9dff5c Update github-golangci-lint to 1.60.2 from 1.60.1
• ccf78c7 Update github-golangci-lint to 1.60.3 from 1.60.2
• 23df925 Update gomod-creasty-defaults to 1.8.0 from 1.7.0
• 9e29682 Update gomod-ergochat-readline to 0.1.3 from 0.1.2
• 01b292d Update gomod-golang-x-crypto to 0.26.0 from 0.25.0
• da6dd9e Update gomod-golang-x-net to 0.28.0 from 0.27.0
• de3fa19 Update make-golangci-lint to 1.60.1 from 1.59.1
• f6d0279 Update make-golangci-lint to 1.60.2 from 1.60.1
• e01cff0 Update make-golangci-lint to 1.60.3 from 1.60.2
• 4299883 Update make-golangci-lint to 1.61.0 from 1.60.3
• ee404b2 all: re-run the test --update
• a241774 all: update tests for tap support
• 2c0b528 ci: Add CGO_ENABLED=0 to make sure we dont need it
• 550bcc2 cli: Add go version to version string
• a195b3f doc: Regenerate
• be9a916 doc: Update formats.svg
• db0dfb1 doc: Update formats.svg
• 66345fe doc: regenerat formats.md for tzx/tap updates
• f16f6dc fq: Update golang.org/x/exp and github.com/gomarkdown/markdown
• 7b6d853 go,lint: Update to go 1.22 and fix some lint warnings
• 3626787 gojq: Update fq fork
• 824e51e gojq: Update fq fork
• fc74f68 intepr: Revert binary value each
• 3f3183a intepr: Support each for binary values
• 7bf1129 interp: Add bits_format=byte_array support
• 52e6e1b matroska: Update spec
• fdde568 midi: (partly) fixed SMPTEOffsetEvent gap
• a1385ca midi: Decoded EndOfTrack and ProgramChange events
• 3276641 midi: Decoded KeySignature, NoteOn and Controller events
• 109719f midi: Decoded MIDIChannelPrefix, MIDIPort and SequencerSpecific metaevents
• cbd4a8a midi: Decoded NoteOff event
• 0226fc6 midi: Decoded PolyphonicPressure, ChannelPressure and PitchBend MIDI events
• b2e71a3 midi: accomodated MIDI event running status
• 0915f75 midi: added 'data' field to EndOfTrack event
• 473394b midi: added event type to events to simplify query by event
• 91fa547 midi: added example queries to the test data
• ce02d6e midi: added length field to TimeSignature struct
• b5f2bda midi: added localised Makefile (cf. #1004 (comment))
• 7950dd6 midi: added midi to the TestFormats all.fqtest list (cf. #1004 (comment))
• 80b9343 midi: added new test and MIDI files
• 57adef4 midi: added sample queries to midi.md
• 0e0a669 midi: added tests for format 0, format 1 and format 2 MIDI files
• e99d9f6 midi: asserted bytes left for MIDI events
• d7ec38a midi: basic test set
• a41e123 midi: cleaned up SysExContinuation and SysExEscape logic
• a7d0cde midi: cleaned up and simplied event decoding logic
• 8995496 midi: cleaned up event decoding logic
• 0b4be89 midi: cleaned up unknown chunk logic
• 3809ddb midi: combined metaevent status + event bytes
• b433998 midi: decoded SysEx events
• 8b236a1 midi: decoded TimeSignature meta event
• f11dfaf midi: decoded TrackName meta event
• 9099a3e midi: decoded chunk tags as FieldUTF8
• 4fe27f5 midi: decoded remaining text metaevents
• 1bdeeb6 midi: decoded tempo metaevent
• 59b1faa midi: decorated 'delta' field with running tick
• 4fac4c6 midi: discarded unknown chunks
• 9dfcb96 midi: experimenting with struct metaevent data
• 2a65036 midi: fixed fuzzing errors
• f1b888b midi: fixed gap in SequencerSpecificEvent
• 4bb3292 midi: fixed key signature map
• f424936 midi: fixed lint warning (cf. #1004 (comment))
• 2f2070e midi: fixed lint warnings
• 87c80f5 midi: fixes for PR comments
• 9f057b6 midi: fixes for PR comments
• 9c7f7f9 midi: fixes for PR comments:
• ad0a06d midi: initial import
• 2970fb1 midi: lowercased event names
• 3ed9889 midi: mapped SMPTE framerates
• 54a0cf1 midi: mapped SysEx event types
• 34fca40 midi: mapped manufacturer IDs to strings
• 0ef3304 midi: mapped note numbers to note names
• 763d142 midi: moved 'dev only' Makefile to workdir and removed it from .gitignore (cf. #1004 (comment))
• 5c89d7d midi: moved MIDI running status and Casio flag to context struct
• c5637f0 midi: partly fixed gaps in SysExMessage
• 7fd9ad2 midi: removed debug leftovers (cf. #1004 (comment))
• 462ae15 midi: removed superfluous AssertLeastBytesLeft (cf. #1004 (comment))
• 17bac77 midi: removed superfluous uint64 cast (cf. #1004 (comment))
• dad4a91 midi: replace d.BytesLen(1) with d.U8()
• 578b7e7 midi: restructured event decoding to decode length and struct fields (cf. #1004 (comment))
• befdf1f midi: reworked SysEx events as struct with length field
• ea3e089 midi: reworked decoding to expect an MThd header as the first chunk (cf. #1004 (comment))
• e940f47 midi: reworked metaevent decoding for PR comments
• a337ff0 midi: reworked sysex and metaevent 'struct' events to decode as byte arrays
• cee51ec midi: reworked to include delta in event
• a3a0a06 midi: simplifed and cleaned up MIDI 'fq' tests:
• c8d9397 midi: snake-cased event types and event names (cf. #1004 (comment))
• 3966d5b midi: tightened up status/event decoding logic (cf. #1004 (comment))
• 2350afe midi: updated help text to use snake-case event names (cf. #1004 (comment))
• 49bd7c2 negentropy: add format.
• e3c7b1a negentropy: another test from a different source, another doc entry, comments on source of test samples.
• 3853387 negentropy: const modes, RFC3339 dates and synthetic timestamps.
• 8f9a005 negentropy: timestamp description as UTC.
• 113ca63 tap: add DefaultInArg
• e526caf tap: add README to tests directory
• 90de619 tap: add a read_one_block test
• e3c3d92 tap: add author to tap.md
• 633160b tap: handle unoffical block types and minor improvements
• e5be55c tap: remove Probe group
• 7816fe1 tap: remove format.TAP_In and update tzx
• e345528 tap: rename block_type to type
• d384930 tap: revert using an array of bytes
• c93dc8b tap: verify checksum values
• 81b2304 tzx: Add suport for ZX Spectrum TZX and TAP files
• 6ac69e2 tzx: add README to tests directory
• 88b6ded tzx: add author to tzx.md
• b27017c tzx: add readme instructions for bits_format=byte_array
• fc350f3 tzx: change data fields to array of bytes for easier JSON parsing
• d3a5b8d tzx: revert using an array of bytes
• ffb5eb3 tzx: use jq-ish values for the mapped symbols in block type 0x18