(P26-008-23) iDINE: A Database for Benchmarking Nutritional Dietary Assessment

Objectives: Accurate machine-learning tools offer promise to advance the field of dietary assessment but require large amounts of data for training and appropriate annotations for distinguishing and classifying food objects. The objective of this study was to create a new publicly available food database to fill this gap.

Methods: Food items from three quick service restaurants were chosen based on availability and popularity. Following a standardized protocol, researchers collected before/after images and videos using an iPhone S.E. camera. Foods were placed on a plate and 11x14 in. grid paper with/without a fiducial marker. For each item, staff varied leftover consumption scenarios using a modified Comstock scale and retook all images/videos. Image frames were extracted from videos at a rate of 10 frames/second. Each frame was annotated for segmentation and object detection. Annotations include segmentation polygons outlining the precise location of each food item in the images and corresponding labels for each polygon. Video data was used to construct three-dimensional models using an open-source structure-from-motion algorithm. Nutrient data in the form of weighed plate waste was obtained by trained research staff following standard protocols.

Results: The iDINE Food Database contains >1500 images and videos of food items from three major quick-service restaurants. This database includes a) images/videos of food items across varying quantities of leftovers, b) corresponding bounding polygons and segmentation masks, c) 3D models of food items, and d) weight (grams) of all foods. These annotations are useful to train machine learning models to distinguish and label food items and estimate volume using the corresponding 3D mesh models. Several state-of-the-art machine learning models were trained and tested using the semantic maps included in this dataset to establish a proof-of-concept benchmark for the further development of automated nutrient assessment systems. The best-performing benchmark model produced a 96.19% pixel accuracy.

Conclusions: The iDINE Food Database provides annotations for semantic segmentation, detection and localization, sample 3-D models, and ground truth for the weight and calories of each food item. iDINE represents the next major wave of AI-based food calorie estimation procedures.

Funding Sources: National Cancer Institute