Skip to content

Commit b7a7600

Browse files
jaberjaber23jaberjaber23
committed
add diagram
1 parent 0552242 commit b7a7600

2 files changed

Lines changed: 200 additions & 24 deletions

File tree

‎README.md‎

Lines changed: 4 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,15 @@
11
# TIDE -- Token-Informed Depth Execution
22

3+
<p align="center">
4+
<img src="assets/tide-diagram.svg" alt="TIDE: Per-token early exit for transformer inference" width="100%"/>
5+
</p>
6+
37
**Make any LLM faster by skipping layers tokens don't need.**
48

59
TIDE learns which tokens are "easy" (converge early) and which are "hard" (need all layers).
610
Easy tokens exit early. Hard tokens go deep. No model retraining. No architecture changes.
711
Drop it onto any HuggingFace model in 3 lines.
812

9-
```
10-
Standard LLM TIDE LLM
11-
========== ========
12-
13-
"The cat sat" "The cat sat"
14-
| | | | | |
15-
[ Layer 1 Layer 1 Layer 1 ] [ Layer 1 Layer 1 Layer 1 ]
16-
| | | | | |
17-
[ Layer 2 Layer 2 Layer 2 ] [ Layer 2 Layer 2 Layer 2 ]
18-
| | | | | |
19-
[ Layer 3 Layer 3 Layer 3 ] [ Layer 3 Layer 3 Layer 3 ]
20-
| | | | | |----> converged! exit.
21-
[ Layer 4 Layer 4 Layer 4 ] [ Layer 4 Layer 4 ] |
22-
| | | | | |
23-
... ... ... ... ... |
24-
| | | | | |
25-
[ Layer N Layer N Layer N ] [ Layer N Layer N ] |
26-
| | | | | |
27-
logits logits logits logits logits logits
28-
29-
Every token runs every layer. Easy tokens exit early.
30-
N layers x 3 tokens = 3N ops. Fewer ops. Same quality.
31-
```
32-
3313
## Install
3414

3515
```bash

‎assets/tide-diagram.svg‎

Lines changed: 196 additions & 0 deletions
Loading

0 commit comments

Comments
 (0)