Skip to content

zacheryvaughn/NextJS-TUS

Repository files navigation

NextJS TUS - Advanced Resumable File Upload System

A production-ready, high-performance file upload system built with Next.js 15 and the TUS (Tus Resumable Upload) protocol. This implementation provides enterprise-grade features including parallel multipart uploads, intelligent stream management, and robust error handling.

πŸš€ Key Features

Core Upload Capabilities

  • Resumable Uploads: Built on the TUS protocol for reliable, resumable file transfers
  • Parallel Processing: Intelligent multipart upload system with dynamic stream allocation
  • Large File Support: Handles files up to 20GB with configurable chunk sizes
  • Real-time Progress: Accurate progress tracking with byte-level precision
  • Queue Management: Advanced file queue with batch processing and stream optimization

Advanced Architecture

  • Dynamic Stream Management: Knapsack algorithm for optimal concurrent upload distribution
  • Multipart Assembly: Sophisticated file reconstruction from parallel upload parts
  • Extensible Filename Strategies: Pluggable system for custom filename handling
  • Duplicate Prevention: Configurable duplicate file handling with multiple strategies
  • Error Recovery: Automatic retry mechanisms with exponential backoff

User Experience

  • Drag & Drop Interface: Modern, responsive file upload UI
  • Live Upload Monitoring: Real-time status updates and progress visualization
  • Batch Operations: Upload multiple files simultaneously with intelligent batching
  • Interactive Controls: Start, pause, cancel, and manage uploads dynamically

πŸ“Έ Interface Preview

Upload Interface

Modern drag-and-drop interface with real-time progress tracking and queue management

πŸ—οΈ Architecture Overview

Frontend Components

  • TusFileUpload.tsx: Main upload interface with drag-drop support
  • useTusFileUpload.ts: Core upload logic and state management
  • UI Components: Modern interface built with Radix UI and Tailwind CSS

Backend Infrastructure

Intelligent Upload System

  • Stream Optimization: Dynamic allocation of concurrent upload streams based on file size
  • Batch Processing: Knapsack algorithm for optimal file batching within stream limits
  • Progress Aggregation: Accurate progress calculation across multiple parallel parts
  • Assembly Pipeline: Seamless reconstruction of multipart uploads

πŸ”§ Technical Implementation

TUS Protocol Compliance

Full implementation of the TUS resumable upload protocol with:

  • Creation Extension: File upload initialization with metadata
  • Core Protocol: Resumable upload with offset tracking
  • Termination Extension: Upload cancellation and cleanup

Multipart Upload Strategy

// Intelligent part calculation based on file size
const getPartCount = (fileSize: number): number => {
  if (fileSize <= 50 * 1024 * 1024) return 1;        // ≀50MB: single part
  if (fileSize <= 500 * 1024 * 1024) return 4;       // ≀500MB: 4 parts
  if (fileSize <= 2 * 1024 * 1024 * 1024) return 8;  // ≀2GB: 8 parts
  return 16;                                          // >2GB: 16 parts
};

Stream Management Algorithm

  • Knapsack Optimization: Maximizes concurrent streams while respecting limits
  • Dynamic Batching: Continuously processes new files during active uploads
  • Resource Allocation: Intelligent distribution of upload bandwidth

Filename Strategy System

Extensible filename handling with multiple strategies:

  • Default: Uses unique upload IDs
  • Original: Preserves original filenames with sanitization
  • Custom: Pluggable system for custom naming conventions

πŸ“ Project Structure

β”œβ”€β”€ app/
β”‚   β”œβ”€β”€ api/upload/[[...params]]/route.ts    # TUS protocol endpoints
β”‚   β”œβ”€β”€ layout.tsx                           # App layout with styling
β”‚   └── page.tsx                             # Main upload page
β”œβ”€β”€ components/
β”‚   β”œβ”€β”€ ui/                                  # Reusable UI components
β”‚   └── upload/TusFileUpload.tsx             # Main upload component
β”œβ”€β”€ hooks/
β”‚   └── upload/useTusFileUpload.ts           # Upload logic hook
β”œβ”€β”€ lib/
β”‚   └── upload/
β”‚       β”œβ”€β”€ config/tus-upload-config.ts      # Configuration settings
β”‚       β”œβ”€β”€ services/                        # Core upload services
β”‚       β”œβ”€β”€ types/upload-types.ts            # TypeScript definitions
β”‚       └── utils/                           # Utility functions
└── styles/globals.css                       # Global styling

βš™οΈ Configuration

Client Configuration

export const TUS_CLIENT_CONFIG = {
  maxFileSelection: 60,                      // Maximum files in queue
  endpoint: '/api/upload/',                  // Upload endpoint
  chunkSize: 8 * 1024 * 1024,              // 8MB chunks
  retryDelays: [0, 1000, 3000, 5000],       // Retry intervals
  maxStreamCount: 8,                        // Concurrent streams
  withFilename: "original",                 // Filename strategy
  onDuplicate: "prevent",                   // Duplicate handling
  destinationPath: ""                       // Upload destination
};

Server Configuration

export const TUS_SERVER_CONFIG = {
  stagingDir: './staging',                  // Temporary upload directory
  mountPath: './uploads',                   // Final upload destination
  maxFileSize: 20 * 1024 * 1024 * 1024,   // 20GB limit
  filenameSanitizeRegex: /[^a-zA-Z0-9._-]/g // Filename sanitization
};

🚦 Getting Started

Prerequisites

  • Node.js 18+
  • npm, yarn, pnpm, or bun

Installation & Setup

# Clone the repository
git clone <repository-url>
cd NextJS-TUS

# Install dependencies
npm install

# Start development server
npm run dev

Environment Variables (Optional)

STAGING_DIR=./staging          # Temporary upload directory
MOUNT_PATH=./uploads          # Final upload destination

Usage

  1. Open http://localhost:3000
  2. Drag and drop files or click to browse
  3. Click "Upload" to start the upload process
  4. Monitor progress and manage uploads in real-time

🎯 What Makes This Implementation Special

1. Production-Ready Architecture

  • Comprehensive error handling and recovery
  • Memory-efficient streaming for large files
  • Robust file system operations with fallbacks

2. Advanced Upload Intelligence

  • Dynamic stream allocation based on file characteristics
  • Optimal batching using knapsack algorithm
  • Real-time adaptation to changing upload conditions

3. Extensible Design

  • Pluggable filename and duplicate handling strategies
  • Modular service architecture for easy customization
  • Type-safe implementation with comprehensive TypeScript support

4. Enterprise Features

  • Configurable upload limits and restrictions
  • Detailed logging and monitoring capabilities
  • Scalable multipart upload system

5. Modern Development Stack

  • Next.js 15 with App Router
  • React 19 with modern hooks
  • Tailwind CSS with Radix UI components
  • Full TypeScript implementation

πŸ”„ Upload Flow

  1. File Selection: Users select files via drag-drop or file browser
  2. Queue Management: Files are added to an intelligent upload queue
  3. Batch Optimization: System calculates optimal upload batches
  4. Parallel Processing: Files are uploaded using multipart strategy when beneficial
  5. Progress Tracking: Real-time progress updates with byte-level accuracy
  6. Assembly: Multipart uploads are seamlessly reconstructed
  7. Finalization: Files are moved to final destination with proper naming

πŸ› οΈ Customization

Adding Custom Filename Strategies

registerFilenameStrategy('custom', (meta, uploadId) => {
  return `${Date.now()}_${meta.filename}`;
});

Adding Custom Duplicate Handlers

registerDuplicateStrategy('timestamp', (filename, directory) => {
  const ext = path.extname(filename);
  const base = path.basename(filename, ext);
  return `${base}_${Date.now()}${ext}`;
});

πŸ“Š Performance Characteristics

  • Throughput: Optimized for high-bandwidth scenarios with parallel uploads
  • Memory Usage: Streaming implementation minimizes memory footprint
  • Scalability: Handles concurrent uploads from multiple users
  • Reliability: Resumable uploads survive network interruptions

πŸ”’ Security Features

  • Filename Sanitization: Prevents directory traversal attacks
  • File Type Validation: Configurable file type restrictions
  • Size Limits: Enforced file size limitations
  • Path Validation: Secure destination path handling

πŸ“ˆ Future Enhancements

  • Cloud storage integration (AWS S3, Google Cloud, Azure)
  • Advanced file type validation and virus scanning
  • Upload analytics and monitoring dashboard
  • WebSocket-based real-time notifications
  • Distributed upload coordination for horizontal scaling

This implementation represents a comprehensive, production-ready file upload solution that combines the reliability of the TUS protocol with modern web development practices and intelligent upload optimization algorithms.

About

An example of uploading large files in NextJS using TUS. An additional file splitting layer allows for uploading in parallel with a knapsack queue to handle multiple multipart files at the same time.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors