satisfies("MIT", "MIT AND Apache-2.0) passes, but is expected to fail

[elrayle]

Description

For function satisfies(first, second), my reading of the parameters is that the check is determining if the first expression satisfies the second. If this is correct, then I expect...

satisfies("MIT", "MIT AND Apache-2.0")
    EXPECT: false
    ACTUAL: true

I expect it to return false since MIT AND Apache-2.0 requires both licenses to be present.

Similar results:

satisfies("MIT AND ICU", "(MIT AND GPL-2.0) OR (ISC AND (Apache-2.0 OR ICU))")
    EXPECT: false
    ACTUAL: true

I expect it to return false since MIT AND ICU does not satisfy any conditions in the second expression.

Analysis

function satisfies (first, second) {
  var one = expand(normalizeGPLIdentifiers(parse(first)))
  var two = flatten(normalizeGPLIdentifiers(parse(second)))
  return one.some(function (o) { return isANDCompatible(o, two) })
}

*** Entering satisfies(first:  MIT , second:  MIT AND Apache-2.0
firstExpanded:  [ [ { license: 'MIT' } ] ]
secondFlattened:  [ { license: 'Apache-2.0' }, { license: 'MIT' } ]
EXPECT:  false , ACTUAL: true 

With the second expression flattened, it appears to act as Apache-2.0 OR MIT and MIT does satisfy that expression resulting in a return value of true.

Similar result:

*** Entering satisfies(first:  MIT AND ICU , second:  (MIT AND GPL-2.0) OR (ISC AND (Apache-2.0 OR ICU))
firstExpanded:  [ [ { license: 'ICU' }, { license: 'MIT' } ] ]
secondFlattened:  [
  { license: 'Apache-2.0' },
  { license: 'GPL-2.0' },
  { license: 'ICU' },
  { license: 'ISC' },
  { license: 'MIT' }
]
EXPECT:  false , ACTUAL: true 

Since ICU and MIT are both in the flattened list, satisfies returns true. But MIT AND ICU do not match any of the conditions in the second expression, so I would expect false.

Exploration

I tried flipping the processing such that the second expression is expanded and the first is flattened. With that, satisfies("MIT", "MIT AND Apache-2.0") returns false as expected. This leads to new failures in the set of assertions in the readme.

Example:

assert(!satisfies('(MIT AND GPL-2.0)', '(ISC OR GPL-2.0)'))

The assertion is written to expect satisfies to return false, but I would expect this to return true since GPL-2.0 in the first expression satisfies ISC OR GPL-2.0.

This leads me to wonder if I am misinterpreting the meaning of the two parameters in the context of which parameter is the test and which is the expression to be satisfied.

[kemitchell]

I think of the first argument as the license expression for an artifact and the second argument as the license expression for a policy about which terms are acceptable.

The algorithm is essentially defined by the examples in README.md.

[dangoor]

@kemitchell in the first example:

satisfies("MIT", "MIT AND Apache-2.0")
    EXPECT: false
    ACTUAL: true

I think it's confusing to use AND to express a list of acceptable policies instead of OR, given the definitions of those in the spec. satisfies("MIT", "MIT OR Apache-2.0") would seem to be a more natural way to express that.

At the very least, if the order of the arguments is relevant then the argument names themselves shouldn't be first and second. Maybe something more like artifact and acceptable or test and policy.

[febuiles]

@kemitchell Would you be open to re-evaluating the behavior described in the title?

While trying to address this feature request we started using spdx-satisfies and quickly ran into the same issue (satisfies("MIT", "MIT AND Apache-2.0") returns true).

This behavior contradicts our expectations given the spec descriptions (linked by @dangoor above).

[kemitchell]

satisfies("MIT", "MIT AND Apache-2.0") should indeed return false.

[cnagadya]

@kemitchell, I think the error might be in the flatten function. Could you please clarify what it is supposed to do?
For example, when for this check satisfies ("MIT" , "(MIT OR ISC) AND GPL-3.0"), flatten is called with the expression param set to:

{
  left: {
    left: { license: 'MIT' },
    conjunction: 'or',
    right: { license: 'ISC' }
  },
  conjunction: 'and',
  right: { license: 'GPL-3.0' }
}

expanded is evaluated to be

[
  { MIT: { license: 'MIT' }, 'GPL-3.0': { license: 'GPL-3.0' } },
  { ISC: { license: 'ISC' }, 'GPL-3.0': { license: 'GPL-3.0' } }
]

which to me seems to be making sense so I think the issue is with this reduce since the key GPL-3.0 is duplicated in both array elements, it is essentially overridden so the output is

{
  MIT: { license: 'MIT' },
  'GPL-3.0': { license: 'GPL-3.0' },
  ISC: { license: 'ISC' }
}

I would have expected it though to be something like

{
  'MIT and GPL-3.0': { license: 'MIT and GPL-3.0' },
  'GPL-3.0 and ISC': { license: 'GPL-3.0 and ISC' }
}

Could you please clarify if my understanding is correct? I can create a PR for this

[courtneycl]

👋 Hi @kemitchell, just a bump on @cnagadya's question for clarification -- we're happy to submit a PR to resolve this but want to make sure we're doing it right 🙏

[kemitchell]

I'd be happy to see a PR, but my personal view on this package at the moment is that use of the SPDX expression syntax for the second argument was very likely a mistake.

A very small number of users likely want to allow specific licenses with exceptions, but mostly what people want is to call satisfies(expression, allowList), where allowList is a simple list (array) of license identifiers.

[neodmy]

Hi all👋. I've also come across unexpected behaviour with this package recently, and was hoping to get some help understand.

According to my probably mistaken interpretation, based on the examples provided in the README, both arguments should be strings with a valid SPDX expression. On the other hand, what would make sense to me when executing satisfies(first, second) is for the function to return true if first satisfies second, and false if first does not satisfy second. However, in the README, I see some somewhat contradictory cases, as in my opinion, they should be the opposite:

• assert(satisfies('MIT AND ISC', '(MIT AND GPL-2.0) OR ISC')) -> In this case, second would reduce to [[MIT, GPL-2.0], [ISC]] so first [[MIT, ISC]] would not satisfy any alternative.

• assert(satisfies('(MIT OR Apache-2.0) AND (ISC OR GPL-2.0)', 'Apache-2.0 OR ISC')) -> [[MIT, ISC], [MIT, GPL-2.0], [Apache-2.0, ISC], [Apache-2.0, GPL-2.0]] none of them would satisfy second [[Apache-2.0],[ISC]].

The same goes for assert(satisfies('(MIT AND GPL-2.0)', '(MIT OR GPL-2.0)')) and assert(satisfies('MIT AND ICU', '(MIT AND GPL-2.0) OR (ISC AND (Apache-2.0 OR ICU))')).

@kemitchell is my interpretation mistaken?

[kemitchell]

See #17.

The decision to take two SPDX expressions as arguments has caused far too much confusion.

[neodmy]

See #17.

The decision to take two SPDX expressions as arguments has caused far too much confusion.

Thanks for your fast reply.

I have a PR ready in case you'd like to keep the argument comparison working with 2 strings, which fits my use case. Actually, I believe both approaches (second as a string with a SPDX expression or array of allowed licenses) may be compatible.

[neodmy]

@kemitchell Please, see #18. Thanks in advance.

[kemitchell]

What's the use case for expression-versus-expression, specifically?

[neodmy]

What's the use case for expression-versus-expression, specifically?

Sorry, but isn't that the original purpose of this package? I mean, judging by the examples on the README I would assume it allows comparing expressions. For instance:

satisfies('MIT AND ISC', '(MIT OR GPL-2.0) AND ISC')

In our case, we use it to compare the licenses of every package on the package.json versus any arbitrary expression the user provides as input. second as an array of simple licenses won't work. Switching arguments won't work either as there may be packages with license1 AND license2 for example.

[kemitchell]

I believe the original issue here has now been avoided. Per #17, we're no longer accepting satisfies(expression, expression) but instead satisfies(expression, array). For most users, trying to explain what AND and OR meant in the second argument caused endless, justified confusion.

[neodmy]

I believe the original issue here has now been avoided. Per #17, we're no longer accepting satisfies(expression, expression) but instead satisfies(expression, array). For most users, trying to explain what AND and OR meant in the second argument caused endless, justified confusion.

Sorry again, but I don't understand why AND and OR would be a problem in this context. Aren't those operators supposed to work like any other logical operator in a logical expression? AND meaning both, and OR meaning one of.