Skip to content

Barcode improvements, App AI Nutrition Label Scan#824

Merged
CodeWithCJ merged 18 commits intoCodeWithCJ:mainfrom
apedley:barcode-improvements
Mar 2, 2026
Merged

Barcode improvements, App AI Nutrition Label Scan#824
CodeWithCJ merged 18 commits intoCodeWithCJ:mainfrom
apedley:barcode-improvements

Conversation

@apedley
Copy link
Copy Markdown
Contributor

@apedley apedley commented Mar 2, 2026

Tip

Help us review and merge your PR faster!
Please ensure you have completed the Checklist below.
For Frontend changes, please run pnpm run validate to check for any errors.
PRs that include tests and clear screenshots are highly preferred!

Description

  1. Added the AI Nutrition Label Scan feature to the mobile app
  • Created new route on server just for label scans
  • Sets a base vision model for users that did not select a model
  • If a bar code is scanned and not found, the user is asked to scan a nutrition label. When the label is scanned the food is created in the database with the barcode information
  1. Significantly improved barcode scanning for the mobile app
  • Most of the work happens on the server
  • Now can use USDA external provider for barcode scans
  • Added a default barcode provider drop down to the web front end. Falls back to OFF data.
  • If all goes well when I'm testing this the next few days I will replace the web frontend with something similar using the new server code
  1. Fixed a small barcode scanning error on the web frontend unrelated to my changes
  2. HTTPS is now enforced for all mobile connections outside of developer mod
  3. Mobile app is now using pnpm. Updated associated workflows

Related Issue

PR type [x] Issue [x] New Feature [ ] Documentation
Linked Issue: #658

Checklist

Please check all that apply:

  • [MANDATORY for new feature] Alignment: I have raised a GitHub issue and it was reviewed/approved by maintainers
  • Tests: I have included automated tests for my changes.
  • [MANDATORY for UI changes] Screenshots: I have attached "Before" vs "After" screenshots below.
  • [MANDATORY for Frontend changes] Quality: I have run pnpm run validate (especially for Frontend).
  • Translations: I have only updated the English (en) translation file (if applicable).
  • Architecture: My code follows the existing architecture standards.
  • Database Security: I have updated rls_policies.sql for any new user-specific tables.
  • [MANDATORY - ALL] Integrity & License: I certify this is my own work, free of malicious code(phishing, malware, etc.) and I agree to the License terms.

Screenshots (if applicable)

Before

[Insert screenshot/GIF here]

After

[Insert screenshot/GIF here]

@apedley apedley changed the title Barcode improvements Barcode improvements, AI Nutrition Label Scan Mar 2, 2026
@apedley apedley changed the title Barcode improvements, AI Nutrition Label Scan Barcode improvements, App AI Nutrition Label Scan Mar 2, 2026
@apedley apedley marked this pull request as ready for review March 2, 2026 15:40
@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a suite of enhancements focused on food data entry and management, particularly for mobile users. It integrates advanced AI capabilities for scanning nutrition labels and significantly upgrades the barcode scanning system by incorporating multiple external data sources and user preferences. Additionally, it bolsters mobile app security by enforcing HTTPS and modernizes the build infrastructure by adopting pnpm, ensuring a more robust and efficient development and user experience.

Highlights

  • AI Nutrition Label Scan Feature: Introduced a new feature in the mobile app allowing users to scan nutrition labels using AI, which then populates food data into the database, including associated barcode information if available.
  • Enhanced Barcode Scanning: Significantly improved barcode scanning capabilities in the mobile app by integrating the USDA external food data provider and allowing users to set a default barcode provider via new web frontend settings. The system now intelligently falls back to Open Food Facts if the primary provider yields no results.
  • Mobile App Security & Performance: Enforced HTTPS for all mobile connections outside of developer mode to enhance security and migrated the mobile application's package management to pnpm, streamlining dependency management and build processes.
  • Frontend UI/UX Improvements: Updated the food entry screens in the mobile app for more intuitive quantity adjustments and added a dedicated UI for selecting default barcode providers in the web frontend settings.
  • Backend API & Database Updates: Added new API routes for nutrition label scanning and updated existing barcode lookup endpoints to support multiple external providers and user-defined preferences. Database migrations were included to store default barcode provider IDs and normalize barcode formats.
  • Bug Fix: Resolved a minor barcode scanning error on the web frontend, unrelated to the new features but included in this release.
Changelog
  • SparkyFitnessFrontend/src/api/Settings/preferences.ts
    • Added 'default_barcode_provider_id' to the UserPreferences interface.
  • SparkyFitnessFrontend/src/components/AboutDialog.tsx
    • Included 'USDA Food Database API' in the About dialog's list of data sources.
  • SparkyFitnessFrontend/src/components/FoodSearch/FoodSearch.tsx
    • Made the 'nutriments' property optional in the 'OpenFoodFactsProduct' interface.
  • SparkyFitnessFrontend/src/contexts/PreferencesContext.tsx
    • Integrated 'defaultBarcodeProviderId' into the preferences context, including its state and associated setter functions.
  • SparkyFitnessFrontend/src/pages/Settings/ExternalProviderList.tsx
    • Updated the component to manage and save 'defaultBarcodeProviderId' preferences, including support for USDA providers.
  • SparkyFitnessFrontend/src/pages/Settings/ExternalProviderSettings.tsx
    • Implemented a UI for selecting a default barcode provider, filtering for active Open Food Facts and USDA providers.
  • SparkyFitnessFrontend/src/pages/Settings/ProviderCard.tsx
    • Modified the ProviderCard to correctly handle 'defaultBarcodeProviderId' when external providers are activated or deactivated.
  • SparkyFitnessFrontend/src/services/preferenceService.ts
    • Added 'default_barcode_provider_id' to the UserPreferences interface.
  • SparkyFitnessFrontend/src/utils/foodSearch.ts
    • Adjusted 'convertOpenFoodFactsToFood' to gracefully handle optional 'nutriments' and scale nutrient values based on serving quantity.
  • SparkyFitnessMobile/tests/services/apiClient.test.ts
    • Added new test cases to verify HTTPS enforcement in the 'apiFetch' function.
  • SparkyFitnessMobile/tests/services/healthDataApi.test.ts
    • Extended test coverage to include HTTPS enforcement for health data synchronization and server connection checks.
  • SparkyFitnessMobile/tests/utils/rateLimiter.test.ts
    • Added a new test file for the 'RateLimiter' utility, covering its functionality and edge cases.
  • SparkyFitnessMobile/app.config.ts
    • Renamed the application to 'SparkyFitness' and configured iOS 'NSAppTransportSecurity' to enforce HTTPS.
  • SparkyFitnessMobile/docs/development.md
    • Updated development instructions to reflect the switch from 'npm install' to 'pnpm install'.
  • SparkyFitnessMobile/package.json
    • Updated several core dependencies, removed the 'engines' field, and adjusted 'transformIgnorePatterns' for pnpm compatibility.
  • SparkyFitnessMobile/pnpm-workspace.yaml
    • Added a new pnpm workspace configuration file.
  • SparkyFitnessMobile/src/components/FoodForm.tsx
    • Introduced an 'onServingChange' prop, added a 'useEffect' hook to track serving changes, and improved touch target area for a button.
  • SparkyFitnessMobile/src/hooks/useExternalFoodSearch.ts
    • Integrated a 'RateLimiter' to manage API call frequency for Open Food Facts searches.
  • SparkyFitnessMobile/src/screens/FoodEntryAddScreen.tsx
    • Refactored the serving size and quantity input logic for improved user experience and precision.
  • SparkyFitnessMobile/src/screens/FoodScanScreen.tsx
    • Implemented a new AI Nutrition Label Scan feature, added a segmented control for switching between barcode and label scanning, and introduced photo capture and processing workflows.
  • SparkyFitnessMobile/src/screens/ManualFoodEntryScreen.tsx
    • Updated the serving size and quantity input fields to use the new quantity adjustment logic and displayed a message when a barcode will be saved.
  • SparkyFitnessMobile/src/screens/SettingsScreen.tsx
    • Added validation to enforce HTTPS for server configurations when saving and activating them in production mode.
  • SparkyFitnessMobile/src/services/api/apiClient.ts
    • Implemented a check to enforce HTTPS for all API requests when the app is running in production mode.
  • SparkyFitnessMobile/src/services/api/externalFoodSearchApi.ts
    • Expanded the 'BarcodeFood' interface, updated the 'BarcodeLookupResult' type, and added a new function for scanning nutrition labels.
  • SparkyFitnessMobile/src/services/api/healthDataApi.ts
    • Added HTTPS enforcement for health data synchronization and server connection checks.
  • SparkyFitnessMobile/src/utils/rateLimiter.ts
    • Added a new utility class for implementing a sliding window rate limiter.
  • SparkyFitnessServer/ai/config.js
    • Introduced 'getDefaultVisionModel' to provide default AI models for vision-related tasks.
  • SparkyFitnessServer/db/migrations/20260302000000_add_default_barcode_provider_id.sql
    • Added a database migration to include 'default_barcode_provider_id' in 'user_preferences' and establish a foreign key constraint.
  • SparkyFitnessServer/db/migrations/20260302120000_normalize_barcodes.sql
    • Added a database migration to normalize 12-digit UPC-A barcodes by prepending a '0' to convert them to 13-digit EAN-13 format.
  • SparkyFitnessServer/docs/externalapis.md
    • Added new documentation detailing the API requests and responses for USDA and Open Food Facts, including internal barcode lookup mechanisms.
  • SparkyFitnessServer/integrations/openfoodfacts/openFoodFactsService.js
    • Added a 'User-Agent' header to Open Food Facts requests and introduced 'searchOpenFoodFactsByBarcodeFields' for more granular data retrieval.
  • SparkyFitnessServer/integrations/usda/usdaService.js
    • Implemented 'searchUsdaFoodsByBarcode' to facilitate barcode lookups using the USDA API.
  • SparkyFitnessServer/models/food.js
    • Integrated the 'normalizeBarcode' utility into food creation, retrieval, and update operations.
  • SparkyFitnessServer/models/preferenceRepository.js
    • Updated preference repository functions to support the new 'default_barcode_provider_id' field.
  • SparkyFitnessServer/routes/foodCrudRoutes.js
    • Added a new API route for AI nutrition label scanning and enhanced the barcode lookup route to accept a 'providerId' parameter and utilize the updated lookup logic.
  • SparkyFitnessServer/routes/foodIntegrationRoutes.js
    • Added a new route '/usda/barcode/:barcode' for direct USDA barcode lookups.
  • SparkyFitnessServer/services/foodCoreService.js
    • Refactored 'lookupBarcode' to support a multi-provider strategy (USDA first, then Open Food Facts) and user preferences. Also updated 'mapOpenFoodFactsProduct' and added 'mapUsdaBarcodeProduct' for accurate data mapping.
  • SparkyFitnessServer/services/labelScanService.js
    • Added a new service responsible for extracting nutrition information from images using various AI models.
  • SparkyFitnessServer/tests/barcodeLookup.test.js
    • Expanded unit tests for 'lookupBarcode' to cover USDA integration, provider preferences, and barcode normalization. New tests for 'normalizeBarcode' and 'mapUsdaBarcodeProduct' were also added.
  • SparkyFitnessServer/tests/barcodeLookupRoute.test.js
    • Updated tests for the barcode lookup route to include the 'providerId' query parameter.
  • SparkyFitnessServer/tests/foodCoreService.test.js
    • Added tests for 'createFood' to ensure proper barcode normalization and sanitization of custom nutrients.
  • SparkyFitnessServer/tests/labelScanRoute.test.js
    • Added a new test file for the '/food-crud/scan-label' API route.
  • SparkyFitnessServer/tests/labelScanService.test.js
    • Added a new test file for 'labelScanService', covering various AI providers, request formatting, and error handling.
  • SparkyFitnessServer/utils/foodUtils.js
    • Added utility functions 'normalizeServingUnit' and 'normalizeBarcode' for consistent data handling.
Ignored Files
  • Ignored by pattern: .github/workflows/** (2)
    • .github/workflows/android.yml
    • .github/workflows/ci-tests.yml
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces significant enhancements to barcode scanning and nutrition label processing across the SparkyFitness frontend and backend, alongside general dependency updates and mobile app improvements. Key changes include adding a default_barcode_provider_id preference to user settings, allowing users to select a preferred external service (like USDA or Open Food Facts) for barcode lookups. The mobile app now supports scanning both barcodes and nutrition labels, with a new UI for switching between modes and handling cases where barcodes are not found. The backend is updated to support USDA barcode lookups, normalize barcode formats, and integrate with AI services for nutrition label extraction from images. Additionally, the mobile app enforces HTTPS for API connections in production environments, updates several React Native dependencies, and refactors food entry quantity controls for better user experience. Review comments suggest improving error messages for HTTPS enforcement, ensuring nutriments are checked before destructuring, and considering schema validation for AI-parsed nutrition data.

Comment thread SparkyFitnessFrontend/src/utils/foodSearch.ts
Comment thread SparkyFitnessMobile/__tests__/services/apiClient.test.ts
Comment thread SparkyFitnessMobile/__tests__/services/healthDataApi.test.ts
Comment thread SparkyFitnessMobile/src/screens/SettingsScreen.tsx
Comment thread SparkyFitnessMobile/src/screens/SettingsScreen.tsx
Comment thread SparkyFitnessMobile/src/services/api/apiClient.ts
Comment thread SparkyFitnessMobile/src/services/api/healthDataApi.ts
Comment thread SparkyFitnessServer/models/food.js
.replace(/^```(?:json)?\n?/, "")
.replace(/\n?```$/, "");

const nutrition = JSON.parse(content);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Consider adding a validation step to ensure that the parsed JSON object conforms to the expected schema (e.g., using a schema validation library) before returning it. This can help prevent unexpected errors due to malformed or incomplete data.

Suggested change
const nutrition = JSON.parse(content);
const nutrition = JSON.parse(content); // Consider validating the schema here
return { success: true, nutrition };

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gemini's solution to a nonproblem is to post a todo comment lol

@CodeWithCJ
Copy link
Copy Markdown
Owner

@apedley This file has some conflicts. Could you check it out.

SparkyFitnessFrontend/src/pages/Settings/ExternalProviderList.tsx

@apedley
Copy link
Copy Markdown
Contributor Author

apedley commented Mar 2, 2026

@apedley This file has some conflicts. Could you check it out.

SparkyFitnessFrontend/src/pages/Settings/ExternalProviderList.tsx

Got it. Was one of the lint comments simsat cleaned up

@CodeWithCJ CodeWithCJ merged commit 44105c1 into CodeWithCJ:main Mar 2, 2026
6 checks passed
@apedley apedley deleted the barcode-improvements branch March 26, 2026 00:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants