pdf-processing-1 example updated

[sujee]

Why are these changes needed?

Update examples/notebook/intro example with

• using newer / simpler APIs
• using v1.0.0
• updated workflow (pdf --> pq --> docid --> ededupe --> fuzzy dedupe --> doc quality)
• renamed the example as pdf-processing-1

Related issue number (if any).

#997

[sujee]

BTW, this will be a 2 stage merge.
Once this is merged, I will update the URLs to reflect the main repo so it works on Google colab ..etc

[shahrokhDaijavad]

@sujee An update: I have tested both Python and Ray notebooks on the local machine successfully. I have also tested the Python notebook on Google Colab successfully, but the ray version of pdf2parquet fails on Google Colab with the usual message of:

ERROR - Exception during execution out of 2 created actors only 0 alive: None

Don't know if you got the Ray version to work on Colab or not.

One thing that we did this week, and I would like you to do for this example, is to move all the files in your input folder to a new place where we want to keep all the input data files for all examples. The new folder is here:
https://github.com/IBM/data-prep-kit/tree/dev/examples/data-files/ and you should create a folder named pdf-processing-1 in this directory and bring the input data files you need for this example (and delete the other input folder). Does it make sense?

[sujee]

no luck running RAY on Colab either. So for now, we will just recommend python version for colab.

I will move the input files to that location and will notify you when ready.

[sujee]

@shahrokhDaijavad input files are moved to : examples/data-files/pdf-processing-1

ready for review and merge. thanks

[shahrokhDaijavad]

LGTM. Thanks, @sujee.