210 mapping updates

[pindec]

Fixes #210, Closes #216

General:

• update the 'Mapping from OCDS' column to reflect the logic implemented in convert-to-oc4ids.
• update cost/index.md to separate the OC4IDS and OCDS mappings, whilst maintaining the OCDS-OC4IDS-IDS relationship
• use italics to indicate which parts of the mapping are not implemented in convert-to-oc4ids.
• remove references to the PPP profile, reference individual extensions instead.

Specific mappings:

• Project owner: remove reference to 'publicAuthority' code from OCDS mapping per Restore buyer and suppliers fields open-contracting-extensions/public-private-partnerships#236. Add reference to publicAuthority field to OC4IDS mapping.
• Sector: mapping updated to reflect https://github.com/open-contracting/ocdskit/issues/163 (since sector is an array of strings, but projectSector is a single code from an open codelist, and additionalClassifications has a structure to accommodate other codelists (and in OCDS for PPPs, planning/project/sector is a Classification object as well). Also updates example.json with example additionalClassification.

Notes:

• For brevity, I haven't explicitly stated whether the referenced fields are in OCDS and OC4IDS, except where the field paths in the two standards are identical. I think it is clear enough from the use of 'copy from X to Y' and 'set Y to the value of X' which standard is being referenced. However, we can add those back in if you think it would be clearer. Alternatively, we could colour code the formatting - but I'm not sure how much work that would be.
• For clarity, I've used object names from the schema rather than titles, e.g. 'If there is more than one Contract object in contracts' rather than 'If there is more than one contract'. However, we can use the shorter form if you think it is clear enough.

Discussion:

• Is there a use case to also have a column that simply lists the relevant OCDS field paths without the logic documented in the mapping column?

TO-DO:

• Do not render the 'Mapping from OCDS' column in the 'IDS to OC4IDS Mapping' section
• Do not render the 'Mapping to OC4IDS' in the 'OCDS to OC4IDS Mapping' section
• Decide on a better approach for indicating which mappings aren't implemented in convert-to-oc4ids (the combination of italics and code formatting in the CSV files doesn't render correctly)
• Add a column that simply lists the relevant OCDS field paths without the mapping logic

[pindec]

Note: need to update mapping, rename Budgets and projects extension to "Project extension".

[duncandewhurst]

@pindec I've added my changes for #216 to this PR and updated the PR description with a summary.

Please could you review?

I have a couple of outstanding things to do per the summary, but they are just presentation, so I would like to get your feedback on the substantive changes.

To give a bit of context: the logic in convert-to-oc4ids isn't otherwise documented in detail (you have to read the python code to understand the transforms). So my vision here is to document the mapping to a similar level of detail to the EU profile, or merging routine documentation, with convert-to-oc4ids as the reference implementation. Based on which an implementer could develop their own tool.

Thanks!

[duncandewhurst]

@jpmckinney do you have any ideas for:

• Do not render the 'Mapping from OCDS' column in the 'IDS to OC4IDS Mapping' section
• Do not render the 'Mapping to OC4IDS' in the 'OCDS to OC4IDS Mapping' section
• Decide on a better approach for indicating which mappings aren't implemented in convert-to-oc4ids (the combination of italics and code formatting in the CSV files doesn't render correctly)

[jpmckinney]

I think we'll have to add a feature to the csv-table-no-translate directive to accept a list of columns to include or exclude.

[pindec]

Thanks @duncandewhurst, looks good and I'm happy with the substantive changes.

On the two notes, I think that it's clear from the context which source of fields you are referring to, and object names help.

On the display issue, we could just add a * for the items that are covered by convert-to-oc4ids, rather than use italics, but I realise that's more problematic for parts where some logic is implemented and some not (e.g. Project Budget). Ditto the latter issue for adding another column. Maybe using colour here might be best.

[pindec]

Should we clarify here OCDS Kit doesn't do currency conversions (I can see it's clear in the mapping)?

[duncandewhurst]

Fixed in edfdeae

[jpmckinney]

Would the sentence about currency conversion added under "Command-line tool and reference implementation" be more relevant here?

[pindec]

On the "use case to also have a column that simply lists the relevant OCDS field paths without the mapping logic", I think this would be useful as a quick reference.

Depending on how the csv-tables render when columns can be excluded, it might be clearer as additional separate tables with just 'CoST IDS element' and 'OCDS field paths' columns, to avoid users having to scroll down to the horizontal scroll bar to then scroll right to see the simplified mapping.

[duncandewhurst]

@jpmckinney although there's a couple of outstanding to-do's, I'd like to get your review. Please see the PR description and #294 (comment) for context.

• Do not render the 'Mapping from OCDS' column in the 'IDS to OC4IDS Mapping' section

• Do not render the 'Mapping to OC4IDS' in the 'OCDS to OC4IDS Mapping' section

I think we'll have to add a feature to the csv-table-no-translate directive to accept a list of columns to include or exclude.

It looks like https://github.com/crate/sphinx_csv_filter :included_cols: option gives us what we need. Could we use that or would it cause problems with translation?

• Decide on a better approach for indicating which mappings aren't implemented in convert-to-oc4ids (the combination of italics and code formatting in the CSV files doesn't render correctly)

Absent a better idea, I went for italicizing the text between the code literals.

• Add a column that simply lists the relevant OCDS field paths without the mapping logic

I'll do this once the mapping is finalized.

[jpmckinney]

I'll review, but let's create a new issue about Sphinx extensions, since it's just a UI improvement and not fundamental to the changes in the PR.

[jpmckinney]

I only skimmed the CSVs, but I assume they have been sufficiently reviewed.

[jpmckinney]

Would the sentence about currency conversion added under "Command-line tool and reference implementation" be more relevant here?

[duncandewhurst]

@jpmckinney for some reason I'm unable to reply to #294 (comment):

Would the sentence about currency conversion added under "Command-line tool and reference implementation" be more relevant here?

I thought it was better to keep all the OCDS Kit-related content under one heading, as it is only relevant to anyone interested in using the tool (and also easier to find and update if, for example, we add currency conversion to convert-to-oc4ids)

[jpmckinney]

OK for sentence placement.

[duncandewhurst]

Thanks!

I'll merge this now, we can address the outstanding TO-DOs once #296 is done:

* [ ]  Do not render the 'Mapping from OCDS' column in the 'IDS to OC4IDS Mapping' section

* [ ]  Do not render the 'Mapping to OC4IDS' in the 'OCDS to OC4IDS Mapping' section

* [ ]  Add a column that simply lists the relevant OCDS field paths without the mapping logic