- The application uses a machine learning model trained on datasets sourced from kaggle. The model is trained and excelled in recognizing various fruits and vegetables from the given image
- Using the Spoonacular API available to the public, we integrate the findByIngredients api call. With the predictions made by the model from the images, the list of ingredients is sent over to the API, the response is then gathered from the call made and displayed onto the application. Spoonacular API also has the feature of returning the "missedIngredients" and "usedIngredients" from the recipes displayed, which helps the user to understand what is required for the recipe and what's not.
- The application runs on a minimal design for now, with the ability of the user to upload all the images of the ingredients he has at hand to the application. After prediction, the user is able to view the possible recipes on the application with the details of the recipe ingredients required possibly if missing any.
- The model, for fruits and for vegetables classification was custom trained on the dataset sourced from kaggle. The model was then saved on to the Azure Storage for the application to access the model during the period of classification analysis
- Next, a flask application was developed using Visual Studio Code and the Python 3.8.x interpreter.
- The Spoonacular API was then integrated along with the Azure Storage Client Integration with the application.
- The first page is the display of the feature of uploading the user's image on to the application, the user selects multiple various ingredients onto the application. The application then sends these images to the Azure Storage for access during the prediction analysis period.
- Once the images have been finished uploading, the models saved in the Azure Storage are then accessed by the application to predict the user's images submitted, and a display of the images submitted and the prediction of the image along with it is displayed for the user's confirmation.
- Upon confirming to produce recipes, the Spoonacular API is called to gather recipes based on the ingredient predictions made in the previous step. The Spoonacular API call has been tuned to proudce 10 recipes and also to only display recipes that are missing a maximum of 5 ingredients from the given ingredient lists (this is expandable in the future)
- The response from the API is then displayed on the results page of the application.
- With the name of the recipe, the user is able to search online to find out the cooking method for the same.
This application relies on several key dependencies, including:
- Flask: The web framework used for building the application.
- TensorFlow: For running machine learning models.
- Pillow: For image processing and handling.
- Azure Blob Storage SDK: If Azure Blob Storage is used for image storage.
- The application is only done as a MVP product, so the plan to develop more user interface will be done with the help of user experience comments in the future.
- The project as of now is able to only able to predict images of single ingredient, further scope includes the multi segement image classification to classify the image of a cluster of ingredients
- The trained models are for common vegetables and fruits available to the general, further improvement on more region specifc ingrredients such as cassava, various types of meat, etc,.
- The results as of now only display recipe names, maybe in future, we can define a model to generate recipe based on the recipe given to it.
- The usage of the web app might be inconvenient for a lot, but in future might move on to android and iOS platforms
- https://spoonacular.com/food-api/docs#Search-Recipes-by-Ingredients
- https://www.kaggle.com/datasets/sshikamaru/fruit-recognition
- https://www.kaggle.com/datasets/kritikseth/fruit-and-vegetable-image-recognition