Support multiple bases

[JonZeolla]

Request: Please add support for Ubuntu 20.04 and CentOS 8, and ensure the docker images tags indicate the specific version of Zeek within the container.

User story: As a Zeek plugin/package developer, I would like to be able to run my end to end dockerized tests on top of images already created and maintained by another project, including situations where my plugin or package supports a matrix of base OSs and specific Zeek versions.

Context: Currently the zeek-kafka plugin package maintains our own Ubuntu 20.04 and CentOS 8 docker images here and only supports e2e testing on one version of Zeek at a time. Also, building zeek as a part of our e2e causes significant delays in developer feedback.

[dominiklohmann]

This is good feedback.

We are working on automating tagged pushes in #1, and what we currently aim for is the following:

• zeek:latest (latest Zeek)
• zeek:lts (latest Zeek LTS)
• zeek:major (latest Zeek major.x)
• zeek:major.minor (latest Zeek major.minor.x)
• zeek:major.minor.patch (latest Zeek major.minor.patch.x)

Because we use it ourselves internally, we also want to support zeek-3 for the near future. The same tags as above (except its) apply to that as well, except the image is named tenzir/zeek-3 rather than tenzir/zeek.

Supporting Ubuntu 20.04 and CentOS 8 is certainly interesting, but does not quite match our use cases currently. That does not, however, mean that we are unwilling to accept such a PR.

On a side note, expect #1 to maybe sit for a bit longer until we figure out how not to run into OOM issues building spicy via zkg in CI.

[mavam]

Supporting Ubuntu 20.04 and CentOS 8 is certainly interesting, but does not quite match our use cases currently. That does not, however, mean that we are unwilling to accept such a PR.

What's the common practice to implement multiple bases with a shared final stage? Putting multiple stages in a single Dockerfile, and then select based on a build arg? Or multiple Dockerfiles per setup, as you do @JonZeolla?

Since we're not CentOS experts, we could provide the scaffold for this probably without too much hassles, and with your help, get it running quickly.

[JonZeolla]

The approach I took was modeled after the official images on docker hub, but with less folders (for example, python). I know many projects compose updates to numerous Dockerfiles by taking a template generation approach, like this

[mavam]

Wow, that shell script seems hard to maintain, but certainly works. I think there might be a sweet spot in keeping multiple Dockerfiles, but then perhaps a set of composable shell scripts that perform the heavy lifting, like installing dependencies. The downside of this outsourcing is that we change things outside Dockerfiles, and then need to force rebuilds whenever the scripts change.

@dominiklohmann what do you think could be a maintainable approach here?

[dominiklohmann]

@dominiklohmann what do you think could be a maintainable approach here?

I think it'd be fair for us to maintain multiple Docker images with regards to future changes, but I would personally not want to be the person to write them in the first place. What will mostly need to be changed is the dependencies setup.

There will definitely be some issues, e.g., ipsumdump does not build on Ubuntu 20.04.