Repeated headers are concatenated with ", ", which doesn't always work

[WGH-]

When libhtp encounters headers with repeated name, it unconditionally concatenates them with ", " into a single header. Unfortunately, it seems to do more harm than good.

For example, consider Set-Cookie. When multiple cookies are to be set, they're sent in repeated Set-Cookie headers:

HTTP/1.0 200 OK
Content-type: text/html
Set-Cookie: theme=light
Set-Cookie: sessionToken=abc123; Expires=Wed, 09 Jun 2021 10:18:14 GMT

If they're concatenated the way libhtp does, it becomes rather difficult to tell them apart (since they contain embedded "," symbols themselves).

I've tried to remove this logic, so repeated headers are always left as separate entries in the table (htp_table_t is a multimap), and apart the few failing tests, which were rather trivial to fix, nothing seemed to break.

However, I'm aware that libhtp clients may probably assume that header map never contains multiple headers, and if fix was implemented this way, the clients would miss some headers.

The code that does this is here:

libhtp/htp/htp_request_generic.c

Line 68 in 7b0cfef

// Do we already have a header with the same name?

libhtp/htp/htp_response_generic.c

Line 254 in 7b0cfef

// Do we already have a header with the same name?

[catenacyber]

Related to https://redmine.openinfosecfoundation.org/issues/1275