Recipe Generation from Food Image Using Deep Learning

Abstract

Seeing a dish and wanting to cook it is a common experience, but a photograph alone does not tell us what went into the food or how it was made. This paper presents a web-based system that identifies a dish from a single food photograph and returns a ready-to-follow recipe. A deep convolutional network trained on the Food-101 dataset classifies the uploaded image into one of 101 dish categories and reports the five most likely candidates with confidence scores. The recognised label is then used to fetch a structured recipe from a curated recipe knowledge base, consisting of an ingredient checklist with quantities, ordered cooking instructions, cuisine, preparation time, servings and difficulty. The result is shown in a browser interface that also lets the user download the recipe as a text file or print it. On a test pizza image, the system identified the dish with a confidence of 99.7% and returned a nine-ingredient, seven-step recipe while running on a CPU-only machine. The paper describes the design, implementation and observed behaviour of the system, discusses its limitations, and outlines how it can be extended toward true image-conditioned recipe generation.

Introduction

The proposed system is an AI-powered food image-to-recipe application that identifies food items from uploaded photographs and provides users with the corresponding recipes. It uses deep learning and computer vision to recognise dishes and connects the predicted food category to a structured recipe database. Unlike complex recipe-generation models, this approach offers a lightweight and practical solution that can run on a CPU-based computer.

The system allows users to upload food images through a web interface and displays the predicted dish along with the top-five classification results and confidence scores. It also provides an ingredient checklist, step-by-step cooking instructions, and options to download or print recipes.

The model is trained using the Food-101 dataset, which contains 101 food categories with 1,000 images per category. The uploaded images are preprocessed and passed through a CNN-based classifier to identify the most likely dish. The predicted category is then used to retrieve the corresponding recipe from the knowledge base.

The literature review highlights previous research in food recognition, ingredient prediction, image-to-recipe retrieval, and recipe generation. Although advanced generative models can produce recipes directly from images, they require large datasets and considerable computational resources. The proposed system addresses this limitation by combining image classification with recipe retrieval, making it more efficient and accessible.

The main contributions of the project include:

1. Food Recognition: Identifies food dishes from uploaded images using a deep learning CNN model.

2. Recipe Retrieval: Retrieves relevant recipes from a structured knowledge base based on the predicted food category.

3. Top-5 Predictions: Displays five possible food classifications with their confidence scores.

4. User-Friendly Interface: Provides ingredients, cooking instructions, and recipe information through a browser-based application.

5. Recipe Management: Allows users to download and print the generated recipe details.

6. Lightweight Implementation: Demonstrates the complete system on a CPU-only machine without requiring expensive hardware.

Conclusion

This paper presented a web-based system that identifies a dish from a food photograph using a deep convolutional net-work trained on Food-101 and returns a structured recipe with ingredients, instructions and metadata. The implementation runs on a CPU-only machine, and a test with a pizza image produced a confident prediction and a complete, usable recipe. The design keeps recognition and recipe content separate, which makes the recipes easy to edit without retraining. The main limitations are the fixed set of 101 dishes and the fact that recipes are retrieved rather than generated from the image itself.

References

[1] A. Salvador et al., “Learning cross-modal embeddings for cooking recipes and food images,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Honolulu, HI, USA, 2017, pp. 3020–3028. [2] J. Marin et al., “Recipe1M+: A dataset for learning cross-modal embed-dings for cooking recipes and food images,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 1, pp. 187–203, Jan. 2021. [3] A. Salvador, M. Drozdzal, X. Giro-i-Nieto, and A. Romero, “Inverse cooking: Recipe generation from food images,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Long Beach, CA, USA, 2019, pp. 10453–10462. [4] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Las Vegas, NV, USA, 2016, pp. 770–778. [5] A. Vaswani et al., “Attention is all you need,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), Long Beach, CA, USA, 2017, pp. 5998–6008. [6] L. Bossard, M. Guillaumin, and L. Van Gool, “Food-101 – Mining discriminative components with random forests,” in Proc. Eur. Conf. Comput. Vis. (ECCV), Zurich, Switzerland, 2014, pp. 446–461. [7] J. Chen and C.-W. Ngo, “Deep-based ingredient recognition for cooking recipe retrieval,” in Proc. ACM Int. Conf. Multimedia, Amsterdam, The Netherlands, 2016, pp. 32–41. [8] W. Min, S. Jiang, L. Liu, Y. Rui, and R. Jain, “A survey on food computing,” ACM Comput. Surv., vol. 52, no. 5, pp. 1–36, Sep. 2019. [9] M. Tan and Q. V. Le, “EfficientNet: Rethinking model scaling for con-volutional neural networks,” in Proc. Int. Conf. Mach. Learn. (ICML), Long Beach, CA, USA, 2019, pp. 6105–6114. [10] A. Dosovitskiy et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2021. [11] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Comput., vol. 9, no. 8, pp. 1735–1780, Nov. 1997. [12] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in Proc. Int. Conf. Learn. Represent. (ICLR), San Diego, CA, USA, 2015. [13] K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “BLEU: A method for automatic evaluation of machine translation,” in Proc. 40th Annu. Meeting Assoc. Comput. Linguistics (ACL), Philadelphia, PA, USA, 2002, pp. 311–318.

Copyright

Copyright © 2026 Vijaya Durga H, Yerram Shiva Nagi Reddy, Shree Vishun J. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.