Gokb integration

[AmandineMroad]

This patch will allow you to search resources on GOKb and import them automatically.

Structural changes:

• Multiple approach for resources (duplicate if parent is different of the existing one)
• Modification of database: IsbnOrIssn table was dropped and replaced by Identifier table. Now we can stock other types of identifier (GokbID, provider, ...); a IdentifierType table was added and provide an ID for the type of identifier.
• Add two classes : GOKbTool and ImportTool; Import tool is now common to csv import and gokb import. Use import tool need a pre-treatment to format datas.
• Package personalization: when your import a package you can customize its content, you can also customize your old package content.

New Features:

GOKb search tool:

This tool allows to search titles and packages on GOKb from their name, or provider (or IsXn for titles) and consult their details.
For the moment, search requests are proceed on a test database, GOKb staff is working on it; at the end of this year, the DB should become a "prod" one.

Step 1: fill the search fields (at least one)

Step 2: displaying a part of results

Step 3: "All results"

Step 4.1 : Click on details button -> Details tab

Step 4.2 : Click on details button -> "TIPPs" tab
Display titles included in if it is a package, availability if it is a title; Content is paginated.

Step 5: Click on select button => Import of the resource

Resource Import from GOKb:

It is now possible to import resource directly from GOKb.

Import results:

When a package is imported, user can customize its content:

Test plan :

• Change your database structure :

DROP TABLE IF EXISTS `_DATABASE_NAME_`.`Identifier`;
CREATE TABLE `_DATABASE_NAME_`.`Identifier` (
  `identifierID` int(11) NOT NULL auto_increment,
  `resourceID` int(11) NOT NULL,
  `identifierTypeID` int(11) default NULL,
  `identifier` varchar(45) NOT NULL,
  PRIMARY KEY (`identifierID`),
  UNIQUE KEY `identifierID` (`identifierID`),
  KEY `resourceID` (`resourceID`),
  KEY `identifierTypeID` (`identifierTypeID`)
)ENGINE=MyISAM AUTO_INCREMENT=1 DEFAULT CHARSET=latin1;

INSERT INTO `Identifier` (`resourceID`, `identifier`)
SELECT `resourceID`, `isbnOrIssn` FROM `IsbnOrIssn`;
UPDATE `Identifier` SET `identifierTypeID`=1;

DROP TABLE IF EXISTS `_DATABASE_NAME_`.`IsbnOrIssn`;

DROP TABLE IF EXISTS `_DATABASE_NAME_`.`IdentifierType`;
CREATE TABLE `_DATABASE_NAME_`.`IdentifierType` (
  `identifierTypeID` int(11) NOT NULL auto_increment,
  `identifierName` varchar(45) NOT NULL,
  PRIMARY KEY (`identifierTypeID`),
  UNIQUE KEY `identifierTypeID` (`identifierTypeID`)
) ENGINE=MyISAM AUTO_INCREMENT=1 DEFAULT CHARSET=latin1;

INSERT INTO `_DATABASE_NAME_`.`IdentifierType` (identifierName) values('Isxn');
INSERT INTO `_DATABASE_NAME_`.`IdentifierType` (identifierName) values('Issn');
INSERT INTO `_DATABASE_NAME_`.`IdentifierType` (identifierName) values('Isbn');
INSERT INTO `_DATABASE_NAME_`.`IdentifierType` (identifierName) values('eIssn');
INSERT INTO `_DATABASE_NAME_`.`IdentifierType` (identifierName) values('eIsbn');
INSERT INTO `_DATABASE_NAME_`.`IdentifierType` (identifierName) values('Gokb');

Install on your server php oai-pmh:
https://github.com/caseyamcl/phpoaipmh

Install history.js on your browser :
https://github.com/browserstate/history.js
(history.js is supported by default on Firefox, no installation required)

Example of a search + import of a package from a title :

1. click on "New resource" on home page to display the form.
2. type "package" in name field and click on "Search on GOKb" or on magnifying glass icon.
3. results must contains at least a title ( American Political Science Association Package): click on 'Details' button.
4. Resource's informations must be displayed (Details tab); click on 'Available in' tab, it must display a clickable link which redirect to parent resource, click on it.
5. A loading bar has to appear while details screen is not displayed. When it does, click on 'Select' button [This will import package and titles included in].
6. Results screen will appear with import stats (number of resources treated, inserted, etc. ).
7. Personalize package content by clicking on 'Custom' button.
8. Check/uncheck resources you want to keep/remove from package; you can use the filter field to group titles. Then, click on 'Confirm customization': it will display removed resources1;
9. click OK, your package and the wanted titles are now in your database.

[benheet]

@AmandineMroad Thank you for this work. It is an exciting development and you have performed some excellent UI work to incorporate the feature. This feature was discussed at the last Steering Committee call and we do have some questions for discussion before this can be incorporated and tested. The discussion may need to move to the [email protected] list.

A few initial questions. The identifier type that you have added does not display in the UI anywhere. If retained it would make sense to show that in the UI. Also values for any field are usually allowed to be customized in the Admin tab of the UI so that each site can set their own field values. We might want to consider doing that for this field as well.

I'm also curious to know how/if this feature uses components in common with the import tool.

We'd also like to see a chart that shows the data elements that are being pulled in from GOKb along with the new or existing fields in the coral database where that data is placed.

Another issue which may result in a request for change is the issue of the multiple versus single record approach in the import tool. A discussion at the last steering committee on this issue has us leaning towards a required multiple record approach and not making single an option. If that change is made in the import tool we want to make sure it won't negatively impact what you've built for GOKb integration.

Sorry for the long post. We'll allow some time for others to comment on this new feature and then bring discussion back to the coral-admin list.

Thank you, and well done. This is an exciting development.

[veggiematts]

"I'm also curious to know how/if this feature uses components in common with the import tool."

The import tool code has been refactored in the admin/classes/domain/ImportTool.php class, so both CSV and Gokb import tools use the same codebase.

"A discussion at the last steering committee on this issue has us leaning towards a required multiple record approach and not making single an option. "

The Gokb import tool has been designed with multiple record approach in mind, so it won't be an issue.

[PaulPoulain]

I won't move this PR to https://github.com/Coral-erm/Coral, things have to be cleaned, validated in Steering Committe,...