Shipping an AI Chat Platform: Lessons from Building IntelAI
What I learned building IntelAI — from streaming LLM responses to managing conversation history at scale.
IntelAI started as a weekend experiment and turned into one of my most technically interesting projects. Here's what I'd do differently if I were starting today.
Streaming responses
Users hate waiting for a complete response. Streaming — where the model outputs tokens as they're generated — dramatically improves perceived performance. In Next.js, this is surprisingly clean: ReadableStream + Response with the stream option handles the heavy lifting. On the client, fetch with a reader lets you update the UI token by token.
Managing conversation history
LLMs are stateless. Every request needs to include the full conversation history to maintain context. This has cost implications — long conversations burn tokens fast. I implemented a sliding window that keeps the last N messages plus a persistent system prompt, trimming the middle when the context gets too long.
What I'd change
I'd reach for the Vercel AI SDK from the start. It handles streaming, tool calls, and provider abstraction out of the box. The primitives I built manually are all there, better tested and maintained. Save your energy for the product, not the plumbing.