Researchers tested whether artificial intelligence can accurately read and understand nutritional information from food labels in both English and Arabic. Currently, grocery stores and health organizations have to manually type out all the nutrition facts from product labels, which takes a lot of time and often leads to mistakes—especially when labels are in different languages. Scientists used advanced AI tools to see if they could automate this process. They tested the AI on 294 real food labels and found that while the AI worked better with English labels, special techniques could significantly improve its accuracy at reading both languages. This could help online grocery stores and health agencies get nutrition information faster and more accurately.

The Quick Take

  • What they studied: Can artificial intelligence accurately read and extract nutritional information from food labels that are written in both English and Arabic?
  • Who participated: The study analyzed 294 real food product labels with 588 pieces of nutritional information written in English and Arabic. These labels were carefully reviewed by humans first to create a reference standard for testing the AI.
  • Key finding: Advanced AI tools (especially one called GPT-4o) could successfully extract nutritional information from food labels, and special correction techniques made the AI significantly more accurate, particularly for Arabic labels where it initially struggled more.
  • What it means for you: In the future, online grocery stores and health organizations may be able to automatically pull nutrition facts from product labels instead of typing them manually. This could mean faster, more accurate nutrition information when you shop online or check food details, though the technology still needs improvement for non-English labels.

The Research Details

Researchers gathered 294 real food product labels from stores and took pictures of them. Each label contained nutritional information in both English and Arabic. They carefully typed out all the nutrition facts from these labels by hand to create a perfect reference guide—this is called “ground truth” in research. Then they fed these label images into three different artificial intelligence systems (GPT-4o, GPT-4V, and Gemini) and asked the AI to read and extract the nutritional information automatically. The researchers compared what the AI extracted to their hand-typed reference to see how accurate each AI system was.

The study focused specifically on how well the AI could handle bilingual labels, since many real-world food products sold internationally have labels in multiple languages. This is important because AI systems are often trained mostly on English text, so they may struggle with other languages. The researchers also tested special correction techniques—basically ways to fix the AI’s mistakes after it made its first attempt at reading the labels.

This type of study is called a technical evaluation or validation study. Instead of testing something on people, the researchers are testing whether a technology works correctly by comparing its output to known correct answers.

Right now, when online grocery stores or health agencies need nutritional information from product labels, someone has to manually type it all in. This is slow, expensive, and mistakes happen frequently—especially with labels in different languages. If AI could do this automatically and accurately, it would save time and money while reducing errors. This matters for food safety, helping people with allergies or dietary restrictions, and making sure health organizations have correct information.

This study has several strengths: it used real product labels (not fake ones), it tested multiple AI systems to compare them, and it created a carefully verified reference dataset. However, the study only tested 294 labels, which is a relatively small sample. The focus on just English and Arabic means results may not apply to other languages. The study doesn’t explain exactly how the AI systems were set up or what specific instructions were given to them, which could affect how well the results apply to real-world use. Additionally, the study is primarily a technical evaluation rather than testing whether this actually helps people in real grocery stores or health settings.

What the Results Show

The artificial intelligence systems showed different levels of success depending on the language. When reading English labels, the AI performed quite well at extracting nutritional information. However, when reading Arabic labels, the AI made more mistakes initially. The best-performing AI system was GPT-4o, which outperformed the other two systems tested (GPT-4V and Gemini).

When researchers applied special correction techniques after the AI made its first attempt, accuracy improved significantly for both languages. These correction techniques essentially allowed the AI to review and fix its own work. With these improvements, the AI became much better at reading Arabic labels, though English labels remained easier for the AI to process.

The study found that the main challenge was extracting information from Arabic text, suggesting that AI systems need more training on non-English languages to perform equally well. The researchers demonstrated that combining AI with post-processing techniques (correction methods) is more effective than relying on the AI alone.

The research revealed that different types of nutritional information had different accuracy rates. Some nutrition facts were easier for the AI to extract than others. The study also showed that the format and quality of the label image affected how well the AI could read it. Labels that were clear and well-photographed were easier for the AI to process than blurry or poorly angled images.

This research builds on earlier work showing that AI can help with reading and understanding text in images. Previous studies have shown that AI struggles more with non-English languages, and this study confirms that pattern with food labels. The finding that correction techniques improve accuracy aligns with other research showing that AI systems often benefit from multiple attempts or verification steps. This study is one of the first to specifically focus on extracting nutritional information from bilingual food labels.

The study only tested 294 labels, which is a small sample size for drawing broad conclusions. It only examined English and Arabic labels, so results may not apply to other languages like Spanish, Chinese, or French. The study didn’t test how well this would work in real grocery stores or with different types of label designs. The research also didn’t evaluate whether the AI could handle damaged, faded, or partially obscured labels—common problems in real life. Finally, the study doesn’t provide information about how much the correction techniques cost in terms of time or computing power, which matters for practical use.

The Bottom Line

Based on this research, AI shows promise for automatically reading food labels in multiple languages, but it’s not yet ready to replace human review entirely. For best results, organizations should use AI as a first step, then have humans verify the information—especially for non-English labels. The technology works better for English than other languages, so this should be considered when planning implementation. Confidence level: Moderate—the technology shows potential but needs more testing in real-world settings.

Online grocery stores, food manufacturers, health organizations, and food safety agencies should pay attention to this research. People with food allergies or specific dietary needs may eventually benefit from faster, more accurate nutrition information. However, individual consumers don’t need to change their behavior based on this study—it’s about behind-the-scenes technology that stores and health organizations use. People who work in food labeling, nutrition information management, or food safety should find this particularly relevant.

If this technology is adopted, improvements could happen gradually over the next 1-3 years as companies test and refine the systems. Don’t expect overnight changes, but within a few years, some online grocery stores may start using AI to help manage nutrition information more efficiently. Full automation without human verification is probably still several years away.

Want to Apply This Research?

  • Users could photograph food labels in their app, and the app could attempt to extract and display nutritional information automatically. Users should verify this information is correct before relying on it for dietary decisions. Track how often the app correctly identifies key nutrients like calories, protein, and allergens.
  • Implement a feature where users can photograph food labels and get instant nutrition facts displayed in the app. Include a verification step where users confirm the information is correct, which also helps train the system. This makes it easier for users to log meals and track nutrition without manually typing in numbers.
  • Monitor the accuracy of AI-extracted nutrition information by comparing it to user corrections over time. Track which types of labels and languages have the highest error rates. Use this data to improve the AI system and alert users when they’re photographing labels in languages where accuracy is lower, suggesting they double-check the information.

This research describes a technology tool for extracting information from food labels, not a medical treatment or dietary advice. The AI system discussed in this study is not yet approved for official food safety or regulatory use. If you have food allergies, dietary restrictions, or health conditions that require specific nutritional information, always verify nutrition facts from official sources or consult with a healthcare provider rather than relying solely on AI-extracted information. This study shows the technology’s potential but also its current limitations, particularly with non-English labels. Always read original product labels when making dietary decisions that affect your health.