diff --git a/seps/sep-0002.md b/seps/sep-0002.md new file mode 100644 index 0000000..49877ef --- /dev/null +++ b/seps/sep-0002.md @@ -0,0 +1,114 @@ +# Use-variants: for packages with conflicting libraries + +## The problem + +The `intel-oneapi-mkl` package is installed with many libraries. Some +of these libraries define the same symbols but differ in their +implementation. For example there is a set of libraries built with +threading support, and another set which offers the same +functionality but are built without threading support. + +Right now, the only mechanism Spack offers to choose between these +libraries is a variant: you could add a `threading` variant and then +choose which libraries to supply to dependents based on the value +of that variant. The problem with a variant is that different +variant values generate distinct installations of a package: we would +install `intel-oneapi-mkl` once for each choice of `threading`, even +though the installations would be the same. + +See also: https://github.com/spack/spack/discussions/22749 + +## Proposed Changes + +* Modify variant declarations with a flag to indicate that the variant + determines how the package is used rather than how it is installed. +* During concretization, this variant is treated like any other + variant (no changes to the concretization algorithm are required). +* After concretization, the variant is stripped from the node and + stored on the edge connecting the node to its parent. + +## Example Use Cases + +### Intel oneAPI MKL + +Described in the introduction. This would be resolved by adding a +`threading` variant: + +``` +variant('threading', values=['openmp', 'sequential'], + interface=True) +``` + +The new logic is the specification of `interface=True`: this +indicates that this variant describes how the package is used rather +than how it is installed. + +Dependents do not have to specify a value for `threading`: if they +don't choose a value, then all dependents still need to "agree" on +using the same libraries. Both the new and old concretizer already +enforce consistency of variants amongst dependents. + +### Boost + +If users want to create a boost installation with many components, +then dependents may only need a subset of the Boost libraries; this +is a bigger issue if the unneeded Boost libraries define conflicting +symbols (not with one another, but with symbols defined in the +dependent or some sibling dependency of Boost). + +Because Boost component libraries don't conflict with each other, +they do not need to be constrained like the dependents of +`intel-oneapi-mkl`. However, when we call +`root_spec['boost'].libs`, we want to know which libraries the +dependent actually needs: to handle this, the dependent can +recompute its dependencies based on its concrete spec, then pass the +dependency variants it requires as parameters to the dependency when +requesting `.libs`. + +We could avoid doing this universally by marking variants where this +is a concern: + +``` +variant('regex', request=True) +``` + +This is not essential though: the dependent could universally +recompute dependency information without any hints from dependents. +For this reason, the Boost use case can be handled completely +separately, since any variant modifications would only be used +to speed up computations and are not required to achieve the +semantics desired for the use case. + +### Intel Parallel Studio + +This provides multiple virtual packages: `mkl`, `blas`, `lapack`, +`mpi`, and more. Each implementation has multiple sets of conflicting +libraries. If a dependent is only using `intel-parallel-studio` for +`mpi`, then we don't want use variants related to `mkl` to be +recorded. + +This is murky pending a completed implementation of finer selection +of virtual providers: Currently we cannot say that we depend on +`intel-parallel-studio` *only* for `mpi` - if you ask for +`intel-parallel-studio` it is assumed that you are using it for +everything it provides (and there is no alternative `blas` +implementation for example). + +Once there is a means of recording which virtuals are being used +by a dependent, we will want to only propagate an interface variant +to the dependent edge if the dependent actually uses the virtual +associated with that variant. For that purpose, I think it would +be suitable to name the associated virtual in the variant +declaration: + +``` +variant('threading', interface=True, provider='mkl') +``` + +In other words, this has the same concretization semantics discussed +for the Intel oneAPI MKL use case (i.e. that this is treated +exactly the same as other variants during concretization), but +adds controls on when the variant influences the hash of dependents: +it only affects the dependent's hash when the dependent is actually +using the package for the virtual named in the variant's `provider=` +option.