Skip to content

STRATEGY_ANALYSIS.md

Latest commit

 

History

History
172 lines (126 loc) · 5.59 KB

STRATEGY_ANALYSIS.md

File metadata and controls

172 lines (126 loc) · 5.59 KB

Strategy System Analysis & Fixes

Summary

The 2048 AI strategy system has been thoroughly analyzed and fixed. The snake strategy is working correctly (identical to the legacy implementation), but there were several critical issues preventing proper strategy functionality:

Issues Found & Fixed

✅ Issue 1: Multi-line Array Parsing (CRITICAL)

Status: FIXED

Problem: The DSL parser couldn't handle multi-line array values. When a component defined arrays across multiple lines (like the corner.dsl weights grid), only the first line was parsed, causing JSON parse failures.

Example of broken parsing:

COMPONENT position_weights {
  weights: [
    [100, 50, 20, 10],
    [50, 10, 5, 2],
    ...
  ]
}

Would parse as: weights: [ (incomplete!)

Solution: Updated parseComponentBlock() and parseValue() methods in strategy-dsl.js to:

  1. Use regex to capture multi-line values properly
  2. Remove newlines from JSON before parsing

Files changed:

✅ Issue 2: API Contract Documentation (MISSING)

Problem: No clear documentation of the DSL API contract, making it difficult to:

  • Understand what components are available
  • Know which variables are valid in formulas
  • Know the scale/magnitude requirements for effective strategies
  • Create new strategies correctly

Solution: Created comprehensive STRATEGY_API.md with:

  • Detailed API contract specification
  • All 7 component types with examples
  • Formula expression reference
  • Key principles for effective strategies
  • Common mistakes and debugging tips
  • Complete example custom strategy

✅ Issue 3: Strategy Weakness (DESIGN ISSUE)

Status: IDENTIFIED, NOT FIXED (BY DESIGN)

Finding: The corner and merge_focused strategies are 1000x weaker than the snake strategy:

  • Snake: 4.30e+21 score
  • Corner: 4.78e+18 score
  • Merge-focused: 4.70e+15 score

Root Cause:

  1. Formula magnitude mismatch - position_weights uses 1e10 while snake uses 10^(15-index) (up to 1e15)
  2. Components need careful tuning and balancing
  3. These strategies would need game testing to verify if they're actually effective

Why not fixed: These are valid but underpowered strategies, not broken implementations. They parse correctly and execute correctly—they're just not competitive. Fixing requires domain knowledge about optimal weights, which should come from empirical testing.

Verification

All strategies now parse and execute correctly:

Strategy Status Score (test grid) Components
Snake ✅ Working 4.30e+21 monotonic_path, empty_cells, smoothness
Corner ✅ Working 4.78e+18 position_weights, empty_cells, smoothness
Merge-focused ✅ Working 4.70e+15 merge_potential, max_tile, empty_cells, corner_bonus

Test Results

  • ✅ Snake strategy identical to legacy implementation
  • ✅ All strategies parse without errors
  • ✅ Evaluators execute successfully
  • ✅ Monotonicity logic works correctly
  • ✅ Position weights parse correctly

Recommendations

For Better Performance

If the other strategies aren't working well in practice:

  1. Tune the corner strategy by increasing formula multipliers:

    formula: value^2 * weight * 1e12  // Was 1e10

  2. Rebalance merge_focused to be more competitive:

    formula: value * matches * 1e12  // Was 1e10
formula: count * 2e13            // Was 5e12

  3. Test empirically - Run actual games and track win rate, max tile, etc.

For Consistency

The snake strategy is the gold standard - it works well and matches the legacy implementation perfectly. Use it as a reference for:

  • Expected score magnitudes
  • Formula scaling patterns
  • Component interaction

For Future Development

  1. Create a strategy benchmarking tool that runs 100+ games and tracks:

    • Win rate (reach 2048)
    • Max tile achieved
    • Average final score
    • Games lost to timeout

  2. Add more diagnostic output to DSL parser for debugging

  3. Consider caching parsed/compiled strategies to avoid re-parsing

Technical Details

What the Snake Strategy Does

COMPONENT monotonic_path {
  path: [[0,0], [0,1], [0,2], [0,3], [1,3], [1,2], [1,1], [1,0], ...]
  position_score: value^2 * 10^(15-index)    // Position dominates scoring
  break_penalty: value_diff * 10^(16-index)  // Penalizes out-of-order tiles
}

COMPONENT empty_cells {
  formula: count * 1e12  // Critical for board mobility
}

COMPONENT smoothness {
  formula: smoothness * 1e4  // Tiebreaker for mergeable neighbors
}

Why it works:

  • The monotonic path drives the AI to arrange tiles in a specific order
  • The 10^(15-index) term is exponential, making position far more important than value
  • Empty cells prevent board lockup
  • Smoothness encourages natural merging

Expression Evaluation

Formulas support:

  • Standard operators: +, -, *, /, ^ (power)
  • Math functions: abs(), max(), min(), log2(), sqrt()
  • Variables: Component-specific (value, weight, count, etc.)
  • Scientific notation: 1e10, 5e3, etc.

Files Modified

  1. src/strategy-dsl.js - Fixed parser for multi-line arrays
  2. STRATEGY_API.md - New comprehensive API documentation

No Breaking Changes

  • All existing strategies continue to work
  • Snake strategy behavior unchanged
  • Backward compatible with existing code