Distributed semantic cache and stateful routing system that cuts LLM API costs by returning cached responses for semantically similar queries. Uses ANN vector search (cosine ≥ 0.8) and consistent hashing to pin requests to the same worker, achieving ~7× latency reduction on cache hits while scaling
| Тип | Репозиторий |
| Категория | GitHub-проекты / Из awesome-списка |
| Цена | открытый код |
| GitHub | redjackfred/distributed-semantic-cache-and-stateful-routing-system |