ARX - AI Architect You Can Talk To
30 technologies · 12 highlights
At a glance
- Architected ARX (AI Architect) - a standalone, product-grade conversational AI application at /arx built on Next.js 15 (App Router), React 19, TypeScript 5, and Tailwind CSS 4 - giving direct chat access to 24+ frontier models across 11 providers including OpenAI (GPT-5.4 Pro, GPT-5.4 Mini, GPT-5.3 Codex), Anthropic (Claude Opus 4.8, Opus 4.7, Sonnet 4.6), Google (Gemini 3.1 Pro, Gemini 3 Flash, Gemma 3 27B), xAI (Grok 4.3, Grok 4.2 Reasoning), DeepSeek (V4 Pro, V4 Flash), Meta (LLaMA 4 Maverick), Mistral (Mistral Large 3), Moonshot (Kimi K2.6, K2.5), Qwen (Qwen3 VL), Nvidia (Nemotron Nano), and MiniMax (MiniMax M2) - all served through 30+ streaming endpoints with intelligent model routing and sub-300ms first-token latency.
- Engineered the full request pipeline surfaced to users as a 6-stage animated workflow: Your Query → JWT Auth & Rate-Limit → RAG Retrieval (ChromaDB vector search pulling full personal context in <1s) → Model Router (picks the best of 24+ frontier models) → AI Inference → Token Streaming - every stage backed by the real production backend: Passport.js JWT validation, Redis-backed rate limiting with structured 429 rateLimit envelopes (limit, remaining, resetInSeconds) that the frontend converts into live countdown timers without polling, and SSE token streaming rendered incrementally in the chat UI.
- Grounded every answer through a 3-Layer Hybrid RAG pipeline over 10M+ ChromaDB embeddings (Azure OpenAI text-embedding-3-large on self-hosted Azure VMs): Layer 1 - 13 regex intent-detection rules for zero-latency section routing; Layer 2 - 12 pre-cached semantic anchor embeddings compared via cosine similarity with no extra API calls; Layer 3 - full-corpus similarity search fallback; combined with entity-diverse re-ranking (guarantees ≥1 document per company/project) and keyword priority boosts (critical=+3, high=+1) - achieving ~99% RAG accuracy and 95% hallucination reduction so ARX answers about real experience, real architecture decisions, and real production numbers instead of generic LLM output.
In the full detail
- What I Built
- Demo Video
- Tech Stack
- Key Concepts
- Low-Level Architecture (LLD)
- High-Level Architecture (HLD)
- Impact
ARX - AI Architect You Can Talk To
30 technologies · 12 highlights
What I Built
- Architected ARX (AI Architect) - a standalone, product-grade conversational AI application at /arx built on Next.js 15 (App Router), React 19, TypeScript 5, and Tailwind CSS 4 - giving direct chat access to 24+ frontier models across 11 providers including OpenAI (GPT-5.4 Pro, GPT-5.4 Mini, GPT-5.3 Codex), Anthropic (Claude Opus 4.8, Opus 4.7, Sonnet 4.6), Google (Gemini 3.1 Pro, Gemini 3 Flash, Gemma 3 27B), xAI (Grok 4.3, Grok 4.2 Reasoning), DeepSeek (V4 Pro, V4 Flash), Meta (LLaMA 4 Maverick), Mistral (Mistral Large 3), Moonshot (Kimi K2.6, K2.5), Qwen (Qwen3 VL), Nvidia (Nemotron Nano), and MiniMax (MiniMax M2) - all served through 30+ streaming endpoints with intelligent model routing and sub-300ms first-token latency.
- Engineered the full request pipeline surfaced to users as a 6-stage animated workflow: Your Query → JWT Auth & Rate-Limit → RAG Retrieval (ChromaDB vector search pulling full personal context in <1s) → Model Router (picks the best of 24+ frontier models) → AI Inference → Token Streaming - every stage backed by the real production backend: Passport.js JWT validation, Redis-backed rate limiting with structured 429 rateLimit envelopes (limit, remaining, resetInSeconds) that the frontend converts into live countdown timers without polling, and SSE token streaming rendered incrementally in the chat UI.
- Grounded every answer through a 3-Layer Hybrid RAG pipeline over 10M+ ChromaDB embeddings (Azure OpenAI text-embedding-3-large on self-hosted Azure VMs): Layer 1 - 13 regex intent-detection rules for zero-latency section routing; Layer 2 - 12 pre-cached semantic anchor embeddings compared via cosine similarity with no extra API calls; Layer 3 - full-corpus similarity search fallback; combined with entity-diverse re-ranking (guarantees ≥1 document per company/project) and keyword priority boosts (critical=+3, high=+1) - achieving ~99% RAG accuracy and 95% hallucination reduction so ARX answers about real experience, real architecture decisions, and real production numbers instead of generic LLM output.
- Built a dual-channel model delivery layer: Channel 1 routes through Azure AI Foundry (Anthropic Foundry SDK) and direct provider APIs (Azure OpenAI, Google GenAI, Mistral, DeepSeek, Grok, Llama, Kimi clients); Channel 2 routes through AWS Bedrock (ConverseStreamCommand) for Claude, Qwen, Gemma, MiniMax, Nemotron, and Kimi - with a MongoDB ActiveAIModels registry so models can be added, reordered, or disabled at runtime with zero redeploys; the frontend model selector hydrates from this registry via a Zustand aiModelsStore, letting users switch between any of the 24+ models mid-session without losing conversation context.
- Delivered an adaptive token streaming architecture over Server-Sent Events with dynamic chunk sizing (2–8 characters per write) and ±3ms jitter timing to prevent synchronization artifacts across providers with wildly different native token cadences, sustaining <50ms chunk intervals uniformly across all 24+ models - complemented by a ChromaDB-backed response memory cache that re-streams previously computed answers chunk-by-chunk so cached responses feel identical to live inference, eliminating redundant LLM API calls for repeated questions and improving perceived latency by 60%.
- Developed an MCP-compliant prompt engineering layer with dynamic system/user role injection and adaptive tone control (professional, casual, friendly, technical) so the same grounded facts render in the register the user asked for, plus Redis-backed long-term session memory preserving conversational context across messages and reconnects; coupled with a daily cron-driven RAG re-ingestion pipeline that automatically clears and re-embeds resume/project data from MongoDB into ChromaDB across 12 structured sections (salary, experience, projects, education, GitHub, mentorship, DSA, skills, achievements, social media, personal, major contributions) - ARX never answers from stale context with zero manual content operations.
- Solved the cold-visitor trust problem by making the product demonstrate itself: a live HeroChat that streams a real grounded answer token-by-token before signup, an animated replay of the actual 6-stage request pipeline with live token counting across 3 real Q&A scenarios, and an 11-provider routing console mirroring real dispatch behavior - the architecture is the demo, with next/dynamic code-splitting keeping the animation-heavy page fast on first load.
- Built the model dispatch layer as a statically-typed streamHandlerMap - 27 dedicated streaming handler functions keyed by exact provider model ID: askGPT5_4ProChatStream for
gpt-5.4-pro, askClaudeOpus48ChatStream forclaude-opus-4-8, askClaudeOpus47ChatStream for the Bedrock-namespacedus.anthropic.claude-opus-4-7, askBedrockQwen3ChatStream forqwen.qwen3-vl-235b-a22b, askKimi25ChatStream formoonshotai.kimi-k2.5, askGrok4_2ReasoningChatStream forgrok-4-20-reasoning, and 21 more - each handler pre-bound to its correct transport (Azure OpenAI, Anthropic Foundry, Google GenAI, or AWS Bedrock) behind one uniform signature (message, chatHistory, onChunk, onStart, signal), so adding a new frontier model is a single map entry with zero conditional routing logic anywhere in the UI. - Engineered the askAI request lifecycle as a guarded state machine: 120-character input limit → auth check (unauthenticated users routed to /arx/chat/login) → isProcessingRef re-entrancy lock blocking double-submits → abortRef.current?.abort() cancels any in-flight stream before creating a fresh AbortController → optimistic user-message append → isThinking state held until the transport's onStart callback fires on the first received byte → a fullResponse accumulator mirrors every chunk into live streamingContent state → on completion the streamed buffer commits to the message list after a 40ms settle window that prevents double-render flicker at the stream/committed boundary - with AbortError explicitly distinguished from real failures (DOMException name check) so intentional cancels never surface an error bubble, and a finally block guaranteeing loading/processing flags reset on every exit path.
- Implemented two custom streaming transports (streamFetcher for raw token streams, sseStreamFetcher for typed JSON events): fetch POST with credentials:include → res.body.getReader() with incremental TextDecoder decoding (stream: true); the SSE variant maintains a partial-line buffer - splitting on newlines, re-buffering the trailing fragment, and parsing only
data:-prefixed lines - so JSON events survive network chunk boundaries that split mid-event; both transports check signal.aborted on every read-loop iteration and call reader.cancel() for leak-free teardown; HTTP 429 responses parse the structured rateLimit payload (success, message, rateLimit.resetInSeconds with a safe fallback shape when the body isn't JSON) and push resetInSeconds into the global navbar store - driving accurate countdown timers across the app with zero polling. - Unified cross-product chat history in a single Zustand chatHistoryStore holding three collections (chats, bestvsbest, voicechat) shared by ARX, Versus, and Voice - hydrated by exactly one getUserChatHistory call per authenticated session (an isFetched flag prevents refetch storms), with every completed answer optimistically prepended via prependChat carrying modelId ({name, model}), ragChunksUsed, fromMemory, and responseTime metadata before any server round-trip - history appears instantly in the sidebar and survives reloads.
- Engineered the chat UX mechanics: a stickRef tracks whether the user is parked within 80px of the scroll bottom via a passive scroll listener - auto-follow engages during token streaming only while the user hasn't scrolled up and re-engages on send; in-chat search with match-index navigation, a measured model-chip width injected as textarea padding so the placeholder never overlaps the selected-model chip, React Markdown + remark-gfm with syntax-highlighted code blocks, skeleton hydration (ArxChatSkeleton), isAuthChecked-gated route protection (redirect fires only after the auth check resolves - no login-page flash for valid sessions), a dedicated login page (email+OTP, Google OAuth, GitHub OAuth), and a wrong-domain guard with canonical redirect to www.princesinghai.com/arx - all dark/light themed and code-split via next/dynamic.
Demo Video
Tech Stack
Next.js 15 (App Router)React 19TypeScript 5Tailwind CSS 4Framer Motion 12Zustand 5 (authStore, aiModelsStore, chatHistoryStore - persisted)Server-Sent Events (SSE token streaming)React Markdown + remark-gfmreact-syntax-highlighterOpenAI (GPT-5.4 Pro, GPT-5.4 Mini, GPT-5.3 Codex)Anthropic (Claude Opus 4.8, Opus 4.7, Sonnet 4.6)Google AI (Gemini 3.1 Pro, Gemini 3 Flash, Gemma 3 27B)xAI (Grok 4.3, Grok 4.2 Reasoning)DeepSeek (V4 Pro, V4 Flash)Meta (LLaMA 4 Maverick)Mistral (Mistral Large 3)Moonshot (Kimi K2.6, K2.5)Qwen (Qwen3 VL)Nvidia (Nemotron Nano)MiniMax (MiniMax M2)Azure AI Foundry (Anthropic Foundry SDK)AWS Bedrock (ConverseStreamCommand)ChromaDB (10M+ embeddings, self-hosted Azure VM)Azure OpenAI Embeddings (text-embedding-3-large)MongoDB ActiveAIModels Registry (runtime model config)Redis (rate limiting with structured 429 envelopes)JWT Authentication (Passport.js)Google OAuth 2.0 / GitHub OAuthAWS CloudFront CDNIntersectionObserver (scroll-driven animation orchestration)
Key Concepts
- Product-Grade AI Chat App - 24+ Frontier Models, 11 Providers, 30+ Streaming Endpoints
- 6-Stage Pipeline: Query → Auth/Rate-Limit → RAG Retrieval → Model Router → Inference → Token Streaming
- 3-Layer Hybrid RAG (13 Regex Intent Rules → 12 Semantic Anchors → Full-Corpus Search)
- Entity-Diverse Re-Ranking + Keyword Priority Boosts - ~99% RAG Accuracy
- Dual-Channel Delivery: Azure AI Foundry / Direct APIs + AWS Bedrock
- MongoDB ActiveAIModels Registry - Zero-Redeploy Model Management
- Mid-Session Model Switching Without Context Loss
- Structured 429 rateLimit Envelopes → Live Countdown UI Without Polling
- Live Self-Playing Hero Chat Demo (Token-by-Token Streaming Simulation)
- Interactive Workflow Simulator - 3 Rotating Scenarios, Live Token Counter
- Live Model-Routing Console - 11-Provider Rail + Terminal Traffic Log
- IntersectionObserver-Driven Count-Up Stats (Re-Stream Every 5s In View)
- Persisted Chat History via Zustand + localStorage
- Wrong-Domain Guard with Canonical Redirect (Multi-Deployment Safety)
Low-Level Architecture (LLD)
High-Level Architecture (HLD)
Impact
- Served as the flagship chat surface of a platform tested with 1000+ active users, processing 18,000+ AI queries from 12,000+ unique visitors and gathering 800+ detailed user feedback that drove 3 major product iterations to a 99% user satisfaction rate - all with no API keys required from users.
- Achieved ~99% RAG accuracy and 95% hallucination reduction through the 3-Layer Hybrid RAG pipeline over 10M+ ChromaDB embeddings - Layers 1 and 2 (13 regex intent rules + 12 pre-cached semantic anchors) resolve most queries with zero additional embedding API calls, and full context retrieval completes in <1 second, so every answer is grounded in verifiable production facts instead of generic LLM filler.
- Sustained sub-300ms first-token latency across all 24+ models and 30+ streaming endpoints via dual-channel delivery - when a provider degrades, traffic reroutes between Azure AI Foundry and AWS Bedrock with zero downtime, so a single provider outage never takes ARX offline.
- Improved perceived response latency by 60% through adaptive SSE streaming (2–8 char chunks, ±3ms jitter, <50ms intervals) combined with the ChromaDB response memory cache - repeated questions replay instantly from cache while feeling identical to live inference, cutting redundant LLM token spend with zero buffering under high load.
- Eliminated model-release lag entirely: the MongoDB ActiveAIModels registry means new frontier models (GPT-5.4, Claude Opus 4.8, Grok 4.3) go live to users within minutes of provider release by flipping a database record - zero code changes, zero redeploys - keeping ARX permanently current in a market where model generations turn over monthly.
- Reduced API abuse by 99% on infrastructure handling 50K+ daily API calls: every chat request clears JWT auth and Redis-backed rate limiting before any model call is made, so not a single unauthorized request has ever triggered LLM spend - with structured 429 envelopes rendering accurate live countdown timers without any frontend polling.
- Kept answers permanently fresh with zero manual content operations: the daily cron re-ingestion pipeline re-embeds all 12 data sections from MongoDB into ChromaDB automatically, so profile, project, and experience updates propagate into RAG answers within 24 hours without anyone touching the pipeline.
- Built on the same production AI infrastructure powering systems that reached 600K+ users, drive 80%+ of platform traffic, and run a <40ms AI inference review engine - ARX is a customer-facing window into battle-tested architecture, not a demo assembled for screenshots.
- Generated direct inbound opportunity: multiple job calls and offers from AI companies, and 3–4 founders reached out to build similar multi-model chat products after using ARX and seeing the architecture respond in real time - the product itself became the strongest proof of engineering capability.
ARX - AI Architect You Can Talk To
30 technologies · 12 highlights
What I Built
- Architected ARX (AI Architect) - a standalone, product-grade conversational AI application at /arx built on Next.js 15 (App Router), React 19, TypeScript 5, and Tailwind CSS 4 - giving direct chat access to 24+ frontier models across 11 providers including OpenAI (GPT-5.4 Pro, GPT-5.4 Mini, GPT-5.3 Codex), Anthropic (Claude Opus 4.8, Opus 4.7, Sonnet 4.6), Google (Gemini 3.1 Pro, Gemini 3 Flash, Gemma 3 27B), xAI (Grok 4.3, Grok 4.2 Reasoning), DeepSeek (V4 Pro, V4 Flash), Meta (LLaMA 4 Maverick), Mistral (Mistral Large 3), Moonshot (Kimi K2.6, K2.5), Qwen (Qwen3 VL), Nvidia (Nemotron Nano), and MiniMax (MiniMax M2) - all served through 30+ streaming endpoints with intelligent model routing and sub-300ms first-token latency.
- Engineered the full request pipeline surfaced to users as a 6-stage animated workflow: Your Query → JWT Auth & Rate-Limit → RAG Retrieval (ChromaDB vector search pulling full personal context in <1s) → Model Router (picks the best of 24+ frontier models) → AI Inference → Token Streaming - every stage backed by the real production backend: Passport.js JWT validation, Redis-backed rate limiting with structured 429 rateLimit envelopes (limit, remaining, resetInSeconds) that the frontend converts into live countdown timers without polling, and SSE token streaming rendered incrementally in the chat UI.
- Grounded every answer through a 3-Layer Hybrid RAG pipeline over 10M+ ChromaDB embeddings (Azure OpenAI text-embedding-3-large on self-hosted Azure VMs): Layer 1 - 13 regex intent-detection rules for zero-latency section routing; Layer 2 - 12 pre-cached semantic anchor embeddings compared via cosine similarity with no extra API calls; Layer 3 - full-corpus similarity search fallback; combined with entity-diverse re-ranking (guarantees ≥1 document per company/project) and keyword priority boosts (critical=+3, high=+1) - achieving ~99% RAG accuracy and 95% hallucination reduction so ARX answers about real experience, real architecture decisions, and real production numbers instead of generic LLM output.
- Built a dual-channel model delivery layer: Channel 1 routes through Azure AI Foundry (Anthropic Foundry SDK) and direct provider APIs (Azure OpenAI, Google GenAI, Mistral, DeepSeek, Grok, Llama, Kimi clients); Channel 2 routes through AWS Bedrock (ConverseStreamCommand) for Claude, Qwen, Gemma, MiniMax, Nemotron, and Kimi - with a MongoDB ActiveAIModels registry so models can be added, reordered, or disabled at runtime with zero redeploys; the frontend model selector hydrates from this registry via a Zustand aiModelsStore, letting users switch between any of the 24+ models mid-session without losing conversation context.
- Delivered an adaptive token streaming architecture over Server-Sent Events with dynamic chunk sizing (2–8 characters per write) and ±3ms jitter timing to prevent synchronization artifacts across providers with wildly different native token cadences, sustaining <50ms chunk intervals uniformly across all 24+ models - complemented by a ChromaDB-backed response memory cache that re-streams previously computed answers chunk-by-chunk so cached responses feel identical to live inference, eliminating redundant LLM API calls for repeated questions and improving perceived latency by 60%.
- Developed an MCP-compliant prompt engineering layer with dynamic system/user role injection and adaptive tone control (professional, casual, friendly, technical) so the same grounded facts render in the register the user asked for, plus Redis-backed long-term session memory preserving conversational context across messages and reconnects; coupled with a daily cron-driven RAG re-ingestion pipeline that automatically clears and re-embeds resume/project data from MongoDB into ChromaDB across 12 structured sections (salary, experience, projects, education, GitHub, mentorship, DSA, skills, achievements, social media, personal, major contributions) - ARX never answers from stale context with zero manual content operations.
- Solved the cold-visitor trust problem by making the product demonstrate itself: a live HeroChat that streams a real grounded answer token-by-token before signup, an animated replay of the actual 6-stage request pipeline with live token counting across 3 real Q&A scenarios, and an 11-provider routing console mirroring real dispatch behavior - the architecture is the demo, with next/dynamic code-splitting keeping the animation-heavy page fast on first load.
- Built the model dispatch layer as a statically-typed streamHandlerMap - 27 dedicated streaming handler functions keyed by exact provider model ID: askGPT5_4ProChatStream for
gpt-5.4-pro, askClaudeOpus48ChatStream forclaude-opus-4-8, askClaudeOpus47ChatStream for the Bedrock-namespacedus.anthropic.claude-opus-4-7, askBedrockQwen3ChatStream forqwen.qwen3-vl-235b-a22b, askKimi25ChatStream formoonshotai.kimi-k2.5, askGrok4_2ReasoningChatStream forgrok-4-20-reasoning, and 21 more - each handler pre-bound to its correct transport (Azure OpenAI, Anthropic Foundry, Google GenAI, or AWS Bedrock) behind one uniform signature (message, chatHistory, onChunk, onStart, signal), so adding a new frontier model is a single map entry with zero conditional routing logic anywhere in the UI. - Engineered the askAI request lifecycle as a guarded state machine: 120-character input limit → auth check (unauthenticated users routed to /arx/chat/login) → isProcessingRef re-entrancy lock blocking double-submits → abortRef.current?.abort() cancels any in-flight stream before creating a fresh AbortController → optimistic user-message append → isThinking state held until the transport's onStart callback fires on the first received byte → a fullResponse accumulator mirrors every chunk into live streamingContent state → on completion the streamed buffer commits to the message list after a 40ms settle window that prevents double-render flicker at the stream/committed boundary - with AbortError explicitly distinguished from real failures (DOMException name check) so intentional cancels never surface an error bubble, and a finally block guaranteeing loading/processing flags reset on every exit path.
- Implemented two custom streaming transports (streamFetcher for raw token streams, sseStreamFetcher for typed JSON events): fetch POST with credentials:include → res.body.getReader() with incremental TextDecoder decoding (stream: true); the SSE variant maintains a partial-line buffer - splitting on newlines, re-buffering the trailing fragment, and parsing only
data:-prefixed lines - so JSON events survive network chunk boundaries that split mid-event; both transports check signal.aborted on every read-loop iteration and call reader.cancel() for leak-free teardown; HTTP 429 responses parse the structured rateLimit payload (success, message, rateLimit.resetInSeconds with a safe fallback shape when the body isn't JSON) and push resetInSeconds into the global navbar store - driving accurate countdown timers across the app with zero polling. - Unified cross-product chat history in a single Zustand chatHistoryStore holding three collections (chats, bestvsbest, voicechat) shared by ARX, Versus, and Voice - hydrated by exactly one getUserChatHistory call per authenticated session (an isFetched flag prevents refetch storms), with every completed answer optimistically prepended via prependChat carrying modelId ({name, model}), ragChunksUsed, fromMemory, and responseTime metadata before any server round-trip - history appears instantly in the sidebar and survives reloads.
- Engineered the chat UX mechanics: a stickRef tracks whether the user is parked within 80px of the scroll bottom via a passive scroll listener - auto-follow engages during token streaming only while the user hasn't scrolled up and re-engages on send; in-chat search with match-index navigation, a measured model-chip width injected as textarea padding so the placeholder never overlaps the selected-model chip, React Markdown + remark-gfm with syntax-highlighted code blocks, skeleton hydration (ArxChatSkeleton), isAuthChecked-gated route protection (redirect fires only after the auth check resolves - no login-page flash for valid sessions), a dedicated login page (email+OTP, Google OAuth, GitHub OAuth), and a wrong-domain guard with canonical redirect to www.princesinghai.com/arx - all dark/light themed and code-split via next/dynamic.
Demo Video
Tech Stack
Next.js 15 (App Router)React 19TypeScript 5Tailwind CSS 4Framer Motion 12Zustand 5 (authStore, aiModelsStore, chatHistoryStore - persisted)Server-Sent Events (SSE token streaming)React Markdown + remark-gfmreact-syntax-highlighterOpenAI (GPT-5.4 Pro, GPT-5.4 Mini, GPT-5.3 Codex)Anthropic (Claude Opus 4.8, Opus 4.7, Sonnet 4.6)Google AI (Gemini 3.1 Pro, Gemini 3 Flash, Gemma 3 27B)xAI (Grok 4.3, Grok 4.2 Reasoning)DeepSeek (V4 Pro, V4 Flash)Meta (LLaMA 4 Maverick)Mistral (Mistral Large 3)Moonshot (Kimi K2.6, K2.5)Qwen (Qwen3 VL)Nvidia (Nemotron Nano)MiniMax (MiniMax M2)Azure AI Foundry (Anthropic Foundry SDK)AWS Bedrock (ConverseStreamCommand)ChromaDB (10M+ embeddings, self-hosted Azure VM)Azure OpenAI Embeddings (text-embedding-3-large)MongoDB ActiveAIModels Registry (runtime model config)Redis (rate limiting with structured 429 envelopes)JWT Authentication (Passport.js)Google OAuth 2.0 / GitHub OAuthAWS CloudFront CDNIntersectionObserver (scroll-driven animation orchestration)
Key Concepts
- Product-Grade AI Chat App - 24+ Frontier Models, 11 Providers, 30+ Streaming Endpoints
- 6-Stage Pipeline: Query → Auth/Rate-Limit → RAG Retrieval → Model Router → Inference → Token Streaming
- 3-Layer Hybrid RAG (13 Regex Intent Rules → 12 Semantic Anchors → Full-Corpus Search)
- Entity-Diverse Re-Ranking + Keyword Priority Boosts - ~99% RAG Accuracy
- Dual-Channel Delivery: Azure AI Foundry / Direct APIs + AWS Bedrock
- MongoDB ActiveAIModels Registry - Zero-Redeploy Model Management
- Mid-Session Model Switching Without Context Loss
- Structured 429 rateLimit Envelopes → Live Countdown UI Without Polling
- Live Self-Playing Hero Chat Demo (Token-by-Token Streaming Simulation)
- Interactive Workflow Simulator - 3 Rotating Scenarios, Live Token Counter
- Live Model-Routing Console - 11-Provider Rail + Terminal Traffic Log
- IntersectionObserver-Driven Count-Up Stats (Re-Stream Every 5s In View)
- Persisted Chat History via Zustand + localStorage
- Wrong-Domain Guard with Canonical Redirect (Multi-Deployment Safety)
Low-Level Architecture (LLD)
High-Level Architecture (HLD)
Impact
- Served as the flagship chat surface of a platform tested with 1000+ active users, processing 18,000+ AI queries from 12,000+ unique visitors and gathering 800+ detailed user feedback that drove 3 major product iterations to a 99% user satisfaction rate - all with no API keys required from users.
- Achieved ~99% RAG accuracy and 95% hallucination reduction through the 3-Layer Hybrid RAG pipeline over 10M+ ChromaDB embeddings - Layers 1 and 2 (13 regex intent rules + 12 pre-cached semantic anchors) resolve most queries with zero additional embedding API calls, and full context retrieval completes in <1 second, so every answer is grounded in verifiable production facts instead of generic LLM filler.
- Sustained sub-300ms first-token latency across all 24+ models and 30+ streaming endpoints via dual-channel delivery - when a provider degrades, traffic reroutes between Azure AI Foundry and AWS Bedrock with zero downtime, so a single provider outage never takes ARX offline.
- Improved perceived response latency by 60% through adaptive SSE streaming (2–8 char chunks, ±3ms jitter, <50ms intervals) combined with the ChromaDB response memory cache - repeated questions replay instantly from cache while feeling identical to live inference, cutting redundant LLM token spend with zero buffering under high load.
- Eliminated model-release lag entirely: the MongoDB ActiveAIModels registry means new frontier models (GPT-5.4, Claude Opus 4.8, Grok 4.3) go live to users within minutes of provider release by flipping a database record - zero code changes, zero redeploys - keeping ARX permanently current in a market where model generations turn over monthly.
- Reduced API abuse by 99% on infrastructure handling 50K+ daily API calls: every chat request clears JWT auth and Redis-backed rate limiting before any model call is made, so not a single unauthorized request has ever triggered LLM spend - with structured 429 envelopes rendering accurate live countdown timers without any frontend polling.
- Kept answers permanently fresh with zero manual content operations: the daily cron re-ingestion pipeline re-embeds all 12 data sections from MongoDB into ChromaDB automatically, so profile, project, and experience updates propagate into RAG answers within 24 hours without anyone touching the pipeline.
- Built on the same production AI infrastructure powering systems that reached 600K+ users, drive 80%+ of platform traffic, and run a <40ms AI inference review engine - ARX is a customer-facing window into battle-tested architecture, not a demo assembled for screenshots.
- Generated direct inbound opportunity: multiple job calls and offers from AI companies, and 3–4 founders reached out to build similar multi-model chat products after using ARX and seeing the architecture respond in real time - the product itself became the strongest proof of engineering capability.