Lifetime of qualified tokens

[ruv]

In some cases, the result of recognizing has a limited lifetime, after which it becomes invalid.

For example, if the result of recognizing the lexeme "foobar" from the input buffer is a string, the lifetime of this string is limited by the lifetime of the input buffer contents. As an opposite example, the lifetime of integer numbers is not limited.

Some plausible user-defined recognizer may return a token whose lifetime ends on the next time that recognizer is called or the next time it successfully recognizes a lexeme. This should be documented.

Thus, if the lifetime of a sub-type members of the qualified token type is limited, this must be specified for that sub-type.