⭐ DroneAI Sense

An AI-powered web application that analyzes audio recordings from drone motors to detect motor type and potential faults.

📖 About the Project

DroneAI Sense is a fault detection system developed to streamline drone maintenance processes. The system converts audio recordings from drone motors into spectrogram images and analyzes them using a Vision Transformer (ViT) model. This enables:

• Motor type automatic identification (Helicopter, Duocopter, Tricopter, Quadcopter)
• Fault type detection (Normal, Balance, Magnet, Propeller, Bearing)
• Confidence score calculation as percentage
• Historical analyses storage and viewing capabilities

🖼️ Website UI

The application offers a user-friendly interface for audio file upload, analysis result viewing, and access to historical analyses.

✨ Features

• 🎯 High Accuracy: 90%+ accuracy rate with Vision Transformer model
• 🚀 Fast Analysis: Results in seconds
• 💾 Historical Records: All analyses stored in database
• 🎨 Modern Interface: User-friendly design with React and TailwindCSS
• 🐳 Easy Setup: Get running with a single Docker command
• 📊 Detailed Reporting: Date and confidence score information for each analysis

🏗️ Architecture

The project consists of three main components in a microservice architecture:

┌─────────────────────┐
│   React Frontend    │  Port 3000
│   (Vite + Tailwind) │
└──────────┬──────────┘
           │
           ▼
┌─────────────────────┐
│   Express API       │  Port 3001
│   (Node.js)         │
└──────────┬──────────┘
           │
           ├──────────────┐
           ▼              ▼
┌─────────────────┐  ┌─────────────────┐
│  Python ML      │  │   RethinkDB     │
│  (FastAPI+PyTorch)  │  │   (NoSQL)       │
│  Port 5000      │  │   Port 28015    │
└─────────────────┘  └─────────────────┘

Technology Stack

Frontend

• React 18 - UI framework
• Vite - Build tool and dev server
• TailwindCSS - Utility-first CSS framework
• Axios - HTTP client
• Lucide React - Modern icon set

Backend API

• Node.js & Express.js - RESTful API
• Multer - File upload middleware
• RethinkDB - NoSQL database
• UUID - Unique ID generation

ML Service

• Python 3.8+ - Programming language
• FastAPI - Async web framework
• PyTorch - Deep learning framework
• timm - Vision Transformer models
• librosa - Audio processing

DevOps

• Docker & Docker Compose - Container management
• Nginx - Reverse proxy

📦 Installation

Prerequisites

• Docker (20.10+)
• Docker Compose (1.29+)

Quick Start

1. Clone the repository:

git clone https://github.com/yourusername/droneai-sense.git
cd droneai-sense

2. Start Docker containers:

For Windows:

start.bat

For Linux/Mac:

docker-compose up -d

3. Access the application:

• Frontend: http://localhost:3000
• API: http://localhost:3001
• ML Service: http://localhost:5000

Containers will load ML models on first startup, this process may take a few minutes.

Manual Installation (For Development)

Click to view details

1. Backend API Setup

cd drone_fault_detection_api
npm install
npm start

2. ML Service Setup

cd drone_fault_detection_model
pip install -r requirements.txt
python ml_service.py

3. Frontend Setup

cd drone_fault_detection_web
npm install
npm run dev

4. RethinkDB Setup

With Docker:

docker run -d -p 28015:28015 -p 8080:8080 --name rethinkdb rethinkdb

🎯 Usage

1. Open Main Page: Navigate to http://localhost:3000 in your web browser

2. Upload Audio File:

   • Click the "Upload Audio" button
   • Select drone motor audio recording (supported formats: .wav, .mp3, .ogg, .flac)
   • You can also use drag-and-drop feature

3. Review Analysis Results:

   • Motor type information
   • Fault status
   • Confidence score (%)
   • Analysis date

4. View Historical Analyses: You can see all your previous analyses at the bottom of the page

📂 Project Structure

droneai-sense/
├── drone_fault_detection_api/      # Backend API
│   ├── src/
│   │   ├── config/                 # Database configuration
│   │   ├── models/                 # Data models
│   │   ├── routes/                 # API endpoints
│   │   ├── services/               # Business logic services
│   │   └── server.js               # Main server file
│   ├── uploads/                    # Uploaded files
│   ├── Dockerfile
│   └── package.json
│
├── drone_fault_detection_model/    # ML Service
│   ├── dataset/                    # Training dataset
│   ├── models/                     # Trained model files
│   │   ├── drone_type/            # Drone type model
│   │   └── fault_type/            # Fault type model
│   ├── audio_to_spectrogram.py    # Audio → Spectrogram conversion
│   ├── drone_type.py              # Drone type model
│   ├── fault_type.py              # Fault type model
│   ├── main.py                     # Model training script
│   ├── ml_service.py              # FastAPI service
│   └── requirements.txt
│
├── drone_fault_detection_web/      # Frontend
│   ├── src/
│   │   ├── components/            # React components
│   │   │   ├── AnalysisCard.jsx  # Analysis card
│   │   │   ├── AnalysisResult.jsx # Result display
│   │   │   ├── FileUpload.jsx    # File upload
│   │   │   ├── Header.jsx        # Header
│   │   │   └── RecentAnalyses.jsx # Historical analyses
│   │   ├── services/
│   │   │   └── api.js             # API client
│   │   ├── App.jsx                # Main application
│   │   └── main.jsx               # Entry point
│   ├── Dockerfile
│   ├── nginx.conf                 # Nginx configuration
│   └── package.json
│
├── docker-compose.yml              # Production configuration
├── docker-compose.dev.yml          # Development configuration
└── README.md

🔧 API Endpoints

Analysis Endpoints

Method	Endpoint	Description
POST	/api/analysis/upload	Upload new audio file and analyze
GET	/api/analysis	List all analyses
GET	/api/analysis/:id	Get specific analysis
DELETE	/api/analysis/:id	Delete analysis

Health Check

Method	Endpoint	Description
GET	/api/health	API health status
GET	/api/health/ml	ML service health status

Sample Request

curl -X POST http://localhost:3001/api/analysis/upload \
  -F "audio=@drone_sound.wav"

Sample Response

{
  "success": true,
  "data": {
    "id": "abc123-def456-ghi789",
    "droneType": "Helicopter",
    "faultType": "Balance Fault Type",
    "droneTypeConfidence": 98.6,
    "faultTypeConfidence": 100.0,
    "timestamp": "2026-01-08T10:58:00.000Z",
    "filename": "drone_sound.wav"
  }
}

🧠 Model Details

Vision Transformer (ViT)

The project uses two separate ViT models:

1. Drone Type Classifier

   • Architecture: vit_base_patch16_224
   • Number of classes: 4 (Helicopter, Duocopter, Tricopter, Quadcopter)
   • Accuracy: ~92%

2. Fault Type Classifier

   • Architecture: vit_base_patch16_224
   • Number of classes: 5 (Normal, Balance, Magnet, Propeller, Bearing)
   • Accuracy: ~95%

Audio Processing Pipeline

Audio File (.wav)
      ↓
Spectrogram Conversion (librosa)
      ↓
Image Preprocessing (224x224, normalization)
      ↓
ViT Model Inference
      ↓
Classification Result + Confidence Score

🛠️ Development

Running in Development Mode

docker-compose -f docker-compose.dev.yml up

In development mode:

• Hot reload active
• Volume mounts enabled
• Debug logs active

Testing

# API tests
cd drone_fault_detection_api
npm test

# ML service tests
cd drone_fault_detection_model
pytest

Model Training

To train a new model:

cd drone_fault_detection_model
python main.py --mode train --epochs 50