Researchers tested whether AI chatbots like ChatGPT could answer nutrition questions as well as human nutrition experts. They gave four different AI systems the same nutrition knowledge test that dietetics students take. The results showed that some AI chatbots, especially ChatGPT-4, performed as well as or better than nutrition students on the test. However, the study also found that AI still has limitations and can’t replace real nutritionists who understand your personal health needs. This research suggests AI could be helpful for general nutrition information, but you should still talk to a real expert for personalized advice.

The Quick Take

  • What they studied: Can artificial intelligence chatbots answer nutrition questions correctly, and how do they compare to human nutrition experts and regular people?
  • Who participated: Four different AI chatbots (ChatGPT-3.5, ChatGPT-4, Google Gemini, and Microsoft Copilot) were tested and compared to nutrition students and English students taking the same nutrition knowledge test.
  • Key finding: ChatGPT-4 scored 82 out of 88 on a nutrition knowledge test, which was higher than nutrition students (79.3 out of 88) and much higher than non-experts (67.7 out of 88). On average, all AI systems scored 77.3 out of 88.
  • What it means for you: AI chatbots may be useful for getting general nutrition information and answering basic questions about healthy eating. However, they shouldn’t replace talking to a real nutritionist or doctor, especially if you have specific health conditions or need personalized meal plans.

The Research Details

Researchers used a standardized nutrition knowledge test called the “General Nutrition Knowledge Questionnaire-Revised” (GNKQ-R) to evaluate four different AI chatbots. This test has 88 questions covering topics like food groups, healthy eating choices, and how diet affects disease. Each AI system answered the same questions using simple prompts (just asking the question without special instructions). The researchers then scored the AI responses using the official test guidelines, just like they would score a human test-taker.

They compared the AI scores to scores from two groups of human test-takers: nutrition students (people studying to become dietitians) and English students (people without nutrition training). This comparison helped them understand whether the AI was performing at an expert level, an intermediate level, or a beginner level.

The study was straightforward and focused: give the same test to machines and humans, score them the same way, and compare the results. This approach allowed researchers to see exactly where AI excels and where it struggles with nutrition knowledge.

This research matters because more people are using AI chatbots to get health and nutrition advice. Understanding how reliable these AI systems are is important for public health. If AI can provide accurate nutrition information, it could help people in areas without access to nutritionists. However, if AI makes mistakes or oversimplifies complex nutrition advice, it could mislead people and harm their health. This study provides concrete evidence about AI’s strengths and weaknesses in nutrition knowledge.

The study used a validated, standardized test (GNKQ-R) that’s recognized in nutrition science, which strengthens the reliability of the results. The researchers tested multiple AI systems rather than just one, which helps show whether the findings apply broadly or just to specific chatbots. However, the study only tested AI systems at one point in time, so results may change as AI technology improves. The study also didn’t test whether AI could give personalized advice for individual health situations, which is an important limitation.

What the Results Show

ChatGPT-4 performed best among all AI systems, scoring 82 out of 88 on the nutrition test. This score was higher than nutrition students, who averaged 79.3 out of 88. ChatGPT-3.5 also performed well, scoring similarly to nutrition students. Google Gemini and Microsoft Copilot scored lower but still averaged around 77 out of 88 across all AI systems tested.

When researchers looked at specific nutrition topics, they found that different AI systems had different strengths. ChatGPT-4 was perfect at answering questions about healthy food choices and performed best on questions about food groups. ChatGPT-3.5 was particularly good at answering questions about diet, disease, and weight management. Both ChatGPT versions performed as well as nutrition students when answering questions about dietary recommendations.

The results show that modern AI chatbots have absorbed a lot of accurate nutrition knowledge and can answer many questions correctly. In some areas, like identifying healthy food choices, the AI performed even better than human experts. This suggests that AI has potential as a tool for providing general nutrition information to the public.

The study found important differences between AI systems. ChatGPT-4 consistently outperformed other AI models, suggesting that newer, more advanced AI systems may be more reliable for nutrition information. The study also revealed that AI systems had specific knowledge gaps in certain areas, meaning they weren’t equally good at all types of nutrition questions. This variation suggests that AI knowledge isn’t uniform and that some AI systems may give better advice on certain topics than others. The research also noted that while AI performed well on the test, it may struggle with the nuanced, personalized advice that real nutritionists provide based on understanding a person’s individual health situation.

This is one of the first studies to systematically compare AI chatbots’ nutrition knowledge to human experts using a standardized test. Previous research has raised concerns about AI reliability in health information, but this study provides concrete evidence that modern AI systems have substantial nutrition knowledge. The findings suggest that AI has advanced significantly in this domain and may be more reliable than previously thought. However, the study confirms what experts have long suspected: AI is better at general knowledge than at personalized, complex medical advice.

The study only tested four AI systems at one specific time, so results may change as AI technology improves or as these systems are updated. The test used (GNKQ-R) measures general nutrition knowledge but doesn’t test whether AI can provide safe, personalized advice for people with specific health conditions. The study didn’t evaluate whether AI could explain its answers well or whether it could adapt advice based on follow-up questions. Additionally, the study compared AI to nutrition students rather than experienced registered dietitians, so we don’t know how AI would compare to the most highly trained nutrition experts. The study also didn’t test whether people would understand or trust AI nutrition advice.

The Bottom Line

AI chatbots like ChatGPT-4 can be a helpful tool for learning general nutrition information and getting answers to basic questions about healthy eating (moderate confidence). They may be particularly useful for understanding food groups and identifying healthy food choices. However, for personalized nutrition advice, especially if you have health conditions like diabetes, heart disease, or food allergies, you should consult with a real nutritionist or doctor (high confidence). If you use AI for nutrition information, treat it as a starting point for learning, not as a replacement for professional medical advice.

General readers interested in nutrition information may find AI chatbots helpful for quick answers and learning basic nutrition facts. People without access to nutritionists might benefit from using AI as a starting point. However, people with chronic diseases, pregnant women, children with special dietary needs, and anyone taking medications that interact with food should always consult with a real healthcare provider. Athletes and people with specific fitness goals should also work with professionals rather than relying solely on AI.

If you use AI to learn general nutrition information, you might notice improvements in your nutrition knowledge within days or weeks. However, if you’re trying to change eating habits or manage a health condition, real results typically take 4-12 weeks to become noticeable. Remember that AI can provide information quickly, but making lasting dietary changes requires time, consistency, and often professional guidance.

Want to Apply This Research?

  • Track which nutrition questions you ask an AI chatbot and which ones you ask a real nutritionist or doctor. Note whether the answers were the same or different. This helps you understand when AI is reliable and when you need professional advice.
  • Use an AI chatbot to learn about nutrition topics and answer general questions, but set a reminder to schedule an appointment with a real nutritionist for personalized advice about your specific health goals or conditions. You could also use AI to prepare questions before seeing a professional.
  • Over the next month, compare nutrition advice you receive from AI with advice from healthcare professionals. Keep notes on areas where they agree and areas where they differ. This personal tracking helps you understand the strengths and limitations of AI for your specific situation.

This research shows that AI chatbots can provide accurate general nutrition information, but they should not replace professional medical or nutritional advice. If you have health conditions, take medications, are pregnant, or have specific dietary needs, please consult with a registered dietitian or doctor before making significant changes to your diet. AI chatbots cannot assess your individual health situation, medical history, or medication interactions the way a healthcare professional can. Always verify important health information with qualified professionals before making decisions that affect your health.