Handwritten Sentence Prediction

This project is a comprehensive solution for recognizing handwritten digits and text from images, with functionalities for training, testing, and usage, making it suitable for tasks like cheque amount verification and other handwritten text recognition applications.

This project provides solution for extracting the letters and then predicting the sentence. For this, few assumptions are made:

1. The handwriting is not cursive.
2. The text is in english language only.
3. Written text is human-readable.

Procedure

1. Training model for digit and character prediction using EMNIST and MNIST dataset available on Kaggle.
2. Histogram based segmentation to find out the characters and space between characters and words.
3. Prediction to decode the sentence

Histogram based Segmentation

1. The original image is converted into binary image.
2. Column wise sum is calculated to figure out the coordinates where the main contents are present and where space is present.
3. Two type of spaces are detected: space between subsequent letters and space between words. These are termed as local space and global space respectively. A threshold value is set to distinguish between local and global space.
4. Row wise sum is calculated to remove any top or bottom space available.
5. The extracted contents are resized for prediction.
6. Model prediction is performed.

Example

1. Binary Image

   

2. Column wise sum - Histogram: frequency vs total number of columns

   

3. Extracted Characters

   

4. Row wise content extraction

   

5. Image resizing

   

6. Prediction

   

   Predicted Sentence: ONE CAXM FOATY THOOIANO TNENTY SIX

   Actual Text: ONE LAKH FORTY THOUSAND TWENTY SIX

   Note: The accuracy of prediction can be increased by increasing accuracy of trained model.

Application - Cheque amount verification

Decoding amount from extracted text

Connect