Skip to main content

Queue Service

Responsibilities​

  • In memory queue for processing tasks
  • Document processing
  • Audio processing
  • Allow Api / workers to consume queue tasks – requires secret headers
  • Allow Api / workers to manage queue – requires secret headers
  • Mark consumed tasks to prevent duplication
  • Queue cleanup

Health Check​

The /health endpoint provides information about:

  • Service status
  • System load
  • CPU
  • Memory
  • Uptime

Required Configurations​

  • See config.ts

Key Points​

  • Only 1 queue service should be up (Since in memory queue)
  • Can have multiple consumers
  • Runtime Deno / Node
  • Only Api / Workers allowed to manage tasks
  • Does not connect to any other services such as Db / LLM etc...

Future Improvements​

  • Consider replacing it with Kafka / RabbitMQ
  • Add persistence
  • After persistence we can add clustering capabilities
  • Add monitoring for vertical auto scale