Image or Object classification based digital aid is quite helpful for visually impaired people to get description of nearby objects. With the advancement in computer vision computing technologies we can afford to develop a system for visually impaired people, which can give audio feedback of surrounding images, objects and context. This project explores image classification methods to assist blind & visually impaired people. I have proposed an Image Classification App for Visually Impaired as an assistive system which is very useful for their safety, quality of life and freedom from other people all the time.
The objective of the project is:
- To develop an image classification app by training pretrained models using an online available dataset through a teachable machine.
- To increase the accuracy by using clear & precise images in the dataset & also increase the number of classes.
- To convert text to speech using python, so that description of classified text can be heard by the visually impaired.
- To build an app with all these features, the Android development environment used is Android Studio.
- To change the layout of the app using Android Studio.
- Teachable Machine (https://teachablemachine.withgoogle.com/)
- TensorFlow Lite
- Android Studio (Version used 4.0.0)
- Python (To convert text to speech)
Using Teachable Machine
- Data Collection : Gather and group the examples into classes, or categories, that we want the computer to learn.
- Training : Train the model, then instantly test it out to see whether it can correctly classify new examples.
- Export the model : Export your model for your projects: sites, apps, and more. Download the model or host it online for free.
Voice to the app
Convert text into natural sounding speech using Python libraries. The process of translating text input into audio is called synthesis and the output is called synthetic speech. The Voice will be added to the App now along with classification of the objects one can hear about the objects too.
Building the App
We will be using TensorFlow Lite Image classification that will classify the objects in the frames seen by the device’s back camera. The environment used for Android Development or building of the App is Android Studio.
-
Play with more machine learning methods to get better accuracy.
-
And classifying a wide variety of classes that can help the visually impaired.
-
Also more work can be done on the layout of the App
-
More models related to Pose detection, Sound Detection & Colour Detection can be added to the App.
-
Use of Gyroscope & Accelerometer.
