Projects/ARX - AI Architect You Can Talk To

ARX - AI Architect You Can Talk To

38 technologies · 25 highlights

Open Live Confidential Docs

What I Built

Architected ARX (AI Architect) - a standalone, product-grade conversational AI application at /arx built on Next.js 15 (App Router), React 19, TypeScript 5, and Tailwind CSS 4 - giving direct chat access to 24+ frontier models across 11 providers including OpenAI (GPT-5.4 Pro, GPT-5.4 Mini, GPT-5.3 Codex), Anthropic (Claude Opus 4.8, Opus 4.7, Sonnet 4.6), Google (Gemini 3.1 Pro, Gemini 3 Flash, Gemma 3 27B), xAI (Grok 4.3, Grok 4.2 Reasoning), DeepSeek (V4 Pro, V4 Flash), Meta (LLaMA 4 Maverick), Mistral (Mistral Large 3), Moonshot (Kimi K2.6, K2.5), Qwen (Qwen3 VL), Nvidia (Nemotron Nano), and MiniMax (MiniMax M2) - all served through 30+ streaming endpoints with intelligent model routing and sub-300ms first-token latency.
Engineered the full request pipeline surfaced to users as a 6-stage animated workflow: Your Query → JWT Auth & Rate-Limit → RAG Retrieval (ChromaDB vector search pulling full personal context in <1s) → Model Router (picks the best of 24+ frontier models) → AI Inference → Token Streaming - every stage backed by the real production backend: Passport.js JWT validation, Redis-backed rate limiting with structured 429 rateLimit envelopes (limit, remaining, resetInSeconds) that the frontend converts into live countdown timers without polling, and SSE token streaming rendered incrementally in the chat UI.
Grounded every answer through a 3-Layer Hybrid RAG pipeline over 10M+ ChromaDB embeddings (Azure OpenAI text-embedding-3-large on self-hosted Azure VMs): Layer 1 - 13 regex intent-detection rules for zero-latency section routing; Layer 2 - 12 pre-cached semantic anchor embeddings compared via cosine similarity with no extra API calls; Layer 3 - full-corpus similarity search fallback; combined with entity-diverse re-ranking (guarantees ≥1 document per company/project) and keyword priority boosts (critical=+3, high=+1) - achieving ~99% RAG accuracy and 95% hallucination reduction so ARX answers about real experience, real architecture decisions, and real production numbers instead of generic LLM output.
Built a dual-channel model delivery layer: Channel 1 routes through Azure AI Foundry (Anthropic Foundry SDK) and direct provider APIs (Azure OpenAI, Google GenAI, Mistral, DeepSeek, Grok, Llama, Kimi clients); Channel 2 routes through AWS Bedrock (ConverseStreamCommand) for Claude, Qwen, Gemma, MiniMax, Nemotron, and Kimi - with a MongoDB ActiveAIModels registry so models can be added, reordered, or disabled at runtime with zero redeploys; the frontend model selector hydrates from this registry via a Zustand aiModelsStore, letting users switch between any of the 24+ models mid-session without losing conversation context.
Delivered an adaptive token streaming architecture over Server-Sent Events with dynamic chunk sizing (2–8 characters per write) and ±3ms jitter timing to prevent synchronization artifacts across providers with wildly different native token cadences, sustaining <50ms chunk intervals uniformly across all 24+ models - complemented by a ChromaDB-backed response memory cache that re-streams previously computed answers chunk-by-chunk so cached responses feel identical to live inference, eliminating redundant LLM API calls for repeated questions and improving perceived latency by 60%.
Developed an MCP-compliant prompt engineering layer with dynamic system/user role injection and adaptive tone control (professional, casual, friendly, technical) so the same grounded facts render in the register the user asked for, plus Redis-backed long-term session memory preserving conversational context across messages and reconnects; coupled with a daily cron-driven RAG re-ingestion pipeline that automatically clears and re-embeds resume/project data from MongoDB into ChromaDB across 12 structured sections (salary, experience, projects, education, GitHub, mentorship, DSA, skills, achievements, social media, personal, major contributions) - ARX never answers from stale context with zero manual content operations.
Solved the cold-visitor trust problem by making the product demonstrate itself: a live HeroChat that streams a real grounded answer token-by-token before signup, an animated replay of the actual 6-stage request pipeline with live token counting across 3 real Q&A scenarios, and an 11-provider routing console mirroring real dispatch behavior - the architecture is the demo, with next/dynamic code-splitting keeping the animation-heavy page fast on first load.
Built the model dispatch layer as a statically-typed streamHandlerMap - 27 dedicated streaming handler functions keyed by exact provider model ID: askGPT5_4ProChatStream for gpt-5.4-pro, askClaudeOpus48ChatStream for claude-opus-4-8, askClaudeOpus47ChatStream for the Bedrock-namespaced us.anthropic.claude-opus-4-7, askBedrockQwen3ChatStream for qwen.qwen3-vl-235b-a22b, askKimi25ChatStream for moonshotai.kimi-k2.5, askGrok4_2ReasoningChatStream for grok-4-20-reasoning, and 21 more - each handler pre-bound to its correct transport (Azure OpenAI, Anthropic Foundry, Google GenAI, or AWS Bedrock) behind one uniform signature (message, chatHistory, onChunk, onStart, signal), so adding a new frontier model is a single map entry with zero conditional routing logic anywhere in the UI.
Engineered the askAI request lifecycle as a guarded state machine: 120-character input limit → auth check (unauthenticated users routed to /arx/chat/login) → isProcessingRef re-entrancy lock blocking double-submits → abortRef.current?.abort() cancels any in-flight stream before creating a fresh AbortController → optimistic user-message append → isThinking state held until the transport's onStart callback fires on the first received byte → a fullResponse accumulator mirrors every chunk into live streamingContent state → on completion the streamed buffer commits to the message list after a 40ms settle window that prevents double-render flicker at the stream/committed boundary - with AbortError explicitly distinguished from real failures (DOMException name check) so intentional cancels never surface an error bubble, and a finally block guaranteeing loading/processing flags reset on every exit path.
Implemented two custom streaming transports (streamFetcher for raw token streams, sseStreamFetcher for typed JSON events): fetch POST with credentials:include → res.body.getReader() with incremental TextDecoder decoding (stream: true); the SSE variant maintains a partial-line buffer - splitting on newlines, re-buffering the trailing fragment, and parsing only data: -prefixed lines - so JSON events survive network chunk boundaries that split mid-event; both transports check signal.aborted on every read-loop iteration and call reader.cancel() for leak-free teardown; HTTP 429 responses parse the structured rateLimit payload (success, message, rateLimit.resetInSeconds with a safe fallback shape when the body isn't JSON) and push resetInSeconds into the global navbar store - driving accurate countdown timers across the app with zero polling.
Unified cross-product chat history in a single Zustand chatHistoryStore holding three collections (chats, bestvsbest, voicechat) shared by ARX, Versus, and Voice - hydrated by exactly one getUserChatHistory call per authenticated session (an isFetched flag prevents refetch storms), with every completed answer optimistically prepended via prependChat carrying modelId ({name, model}), ragChunksUsed, fromMemory, and responseTime metadata before any server round-trip - history appears instantly in the sidebar and survives reloads.
Engineered the chat UX mechanics: a stickRef tracks whether the user is parked within 80px of the scroll bottom via a passive scroll listener - auto-follow engages during token streaming only while the user hasn't scrolled up and re-engages on send; in-chat search with match-index navigation, a measured model-chip width injected as textarea padding so the placeholder never overlaps the selected-model chip, React Markdown + remark-gfm with syntax-highlighted code blocks, skeleton hydration (ArxChatSkeleton), isAuthChecked-gated route protection (redirect fires only after the auth check resolves - no login-page flash for valid sessions), a dedicated login page (email+OTP, Google OAuth, GitHub OAuth), and a wrong-domain guard with canonical redirect to www.princesinghai.com/arx - all dark/light themed and code-split via next/dynamic.
Hardened the platform behind a 4-layer security perimeter where every layer catches exactly what the previous one structurally cannot: a CORS allowlist (browser-enforced only) → an Origin guard rejecting any request with a missing or non-allowlisted Origin header - the check that actually stops curl and Postman, since a direct HTTP client ignores CORS entirely → an internal-key guard requiring a shared secret held only by the server-side proxy, making the API unreachable except through the application itself → Passport JWT re-read against the database on every request, so a deactivated account's un-expired token stops working immediately.
Engineered the streaming layer as a two-clock decoupling that normalizes 5 structurally different provider stream shapes into one identical cadence: a provider-paced reader loop only fills a buffer, while an independent UI-paced writer loop drains it on a fixed frame using backlog-adaptive chunk sizing (2/4/6/8 characters at <20/<80/<200/≥200 buffered) - a self-balancing rate matcher that sizes up to drain a fast provider's backlog and sizes down to stretch a slow one smoothly, so every model types at the same rhythm and the delivery channel is invisible to the user.
Solved the three independent buffering layers that silently destroy paced streaming in production: Content-Encoding: identity to opt out of gzip, whose compression window would batch carefully-paced 2–8 character chunks back into bursts; X-Accel-Buffering: no to stop the reverse proxy doing the same thing a layer higher; and socket.setNoDelay(true) to disable Nagle's algorithm so small writes leave the kernel immediately instead of coalescing - without all three, the typewriter effect collapses into stutter no matter how well the writer loop is tuned.
Designed the RAG corpus as deterministically generated semantic chunks across 12 sections from a hand-written template generator rather than a recursive text splitter - because the source is structured data, not prose - and moved the expensive work to index time instead of query time: synonyms and alternate phrasings are embedded directly into chunks as SEARCH KEYWORDS, so retrieval costs zero LLM calls per query where a conventional pipeline spends multiple calls on query rewriting and HyDE; redundant confirmation chunks written as explicit question-and-answer pairs make common recruiter questions embed close to their answer form, and priority-weighted metadata guarantees curated summary chunks survive re-ranking instead of being crowded out by literal keyword matches.
Enforced grounding through a strict prompt contract rather than trusting the model to behave: retrieved context is injected as discretely numbered units so the anti-hallucination rule has something concrete to point at, followed by explicit constraints and a self-check validation list - and critically, any client-supplied system message is stripped server-side, so the system prompt cannot be injected or overridden from the request; when retrieval returns nothing the context block is simply empty, and the contract converts a would-be hallucination into a graceful in-persona admission instead.
Built a diagram delivery pipeline that stores raw architecture-diagram source inside the RAG corpus and hands the model a pre-repaired diagram to place verbatim - instead of asking an LLM to reproduce diagram syntax from memory, which reliably corrupts it - with a custom sanitizer that repairs malformed edge labels, unbalanced delimiters, incomplete subgraph blocks, reserved characters inside node labels and every arrow variant, wired to an instruction builder that switches between generating a diagram from scratch, placing the stored one, or showing it as reference only.
Implemented quota control as a structural cost guarantee: a Redis-backed limiter keyed on a composite user-ID + IP - neither alone is sufficient, since a user ID alone lets one person burn quota across devices while an IP alone punishes everyone behind a corporate NAT - deliberately made global across all models rather than per-model so quota cannot be multiplied by switching models, with a fallback to an in-process store when Redis is unavailable so limits degrade rather than disappear, and a resetInSeconds read from the actual Redis TTL rather than a recomputed estimate - making the client countdown accurate to the second with zero polling.
Made compensation privacy a property of the retrieval layer rather than a prompt instruction: the salary chunk is a single dense, numbers-heavy document, which makes it a strong semantic attractor for a wide range of unrelated career questions - so it is unconditionally excluded from every fallback retrieval path and reachable only when intent detection explicitly resolves a compensation question, guaranteeing that salary can never leak into an answer about architecture, projects or experience by construction instead of by hoping the model stays on topic.
Shipped the operational surface that makes runtime configuration real instead of aspirational: role-guarded admin endpoints providing live vector-store chunk CRUD - page through, read, update or delete an individual embedding chunk - so a single incorrect RAG chunk is hot-patched in place without a full re-ingest; and a model-registry cache refresh that completes the zero-redeploy loop, since flipping a model's active flag alone changes nothing while the registry cache is deliberately stored without expiry - the database flip plus the cache refresh is the actual operation.
Made resilience an explicit design property rather than an accident: chat persistence is deliberately fire-and-forget, so database latency can never block an answer that is already streaming, and every save writes through to the cache, turning a subsequent history load into one cache read instead of three database queries; on failure, cache loss drops the limiter to an in-process store while reads fall through to the database, an embedding failure yields an honest refusal instead of a fabrication, and a client disconnecting mid-stream still completes caching and persistence - so provider spend already incurred is never wasted.
Engineered the zero-latency intent router so that rule ordering is load-bearing rather than incidental: compensation is evaluated first, so "what package did you get at ProPeers?" routes to salary instead of employment; company-specific rules precede the generic experience rule so a named employer narrows to that employer rather than returning every job; named projects precede the generic "tell me about" rule so a product question never degrades into a biography; and the router carries fuzzy matching for commonly misspelled company names plus geographic inference - a question about the "US company" resolves to the correct employer because that fact is encoded into the routing layer - all resolved as pure metadata lookups with no embedding call and no LLM call.
Solved a retrieval-quality failure that pure relevance scoring cannot see: because one employer had roughly four times the indexed content of another, a question like "tell me about your work experience" returned four chunks about a single company and zero about the rest - each individually top-scoring, collectively a bad answer, since the user asked about a career, not a company. The fix reserves a guaranteed floor of result slots per distinct entity before filling the remainder by score, then re-sorts for presentation so the strongest chunk still leads - giving breadth across companies and projects without sacrificing the top result, and turning a relevance ranker into a coverage-aware one.
Absorbed real provider-level incompatibilities behind one uniform internal contract instead of leaking them upward: the same grounding prompt is delivered five structurally different ways - as a true system message, as a top-level system parameter, as an instructions field, as a single flattened prompt string, and - on the Bedrock channel, which accepts only user and assistant roles and rejects anything else - as a prepended user turn with every history role normalized to a permitted value; paired with a per-tier inference policy that assigns lower temperature to the models used for the most factual answers and looser sampling to the rest, so adding a provider never changes a single line of dispatch logic.

Demo Video

Tech Stack

Next.js 15 (App Router)React 19TypeScript 5Tailwind CSS 4Framer Motion 12Zustand 5 (authStore, aiModelsStore, chatHistoryStore - persisted)Server-Sent Events (SSE token streaming)React Markdown + remark-gfmreact-syntax-highlighterOpenAI (GPT-5.4 Pro, GPT-5.4 Mini, GPT-5.3 Codex)Anthropic (Claude Opus 4.8, Opus 4.7, Sonnet 4.6)Google AI (Gemini 3.1 Pro, Gemini 3 Flash, Gemma 3 27B)xAI (Grok 4.3, Grok 4.2 Reasoning)DeepSeek (V4 Pro, V4 Flash)Meta (LLaMA 4 Maverick)Mistral (Mistral Large 3)Moonshot (Kimi K2.6, K2.5)Qwen (Qwen3 VL)Nvidia (Nemotron Nano)MiniMax (MiniMax M2)Azure AI Foundry (Anthropic Foundry SDK)AWS Bedrock (ConverseStreamCommand)ChromaDB (10M+ embeddings, self-hosted Azure VM)Azure OpenAI Embeddings (text-embedding-3-large)MongoDB ActiveAIModels Registry (runtime model config)Redis (rate limiting with structured 429 envelopes)JWT Authentication (Passport.js)Google OAuth 2.0 / GitHub OAuthAWS CloudFront CDNIntersectionObserver (scroll-driven animation orchestration)Node.js + Express.js (streaming REST API)Passport.js (7 strategies - JWT + Google/GitHub per deployment domain)OpenAI Responses API (instructions + input, delta event stream)Google GenAI (thinkingConfig - low/high reasoning levels)AWS Bedrock ConverseCommand (non-streaming variant)ChromaDB Metadata Filtering ($eq / $in / $nin / $and)Async Reader/Writer Stream Pacing (Node.js event loop)HTTP Chunked Transfer - gzip opt-out, X-Accel-Buffering, Nagle disabled

Key Concepts

Product-Grade AI Chat App - 24+ Frontier Models, 11 Providers, 30+ Streaming Endpoints
6-Stage Pipeline: Query → Auth/Rate-Limit → RAG Retrieval → Model Router → Inference → Token Streaming
3-Layer Hybrid RAG (13 Regex Intent Rules → 12 Semantic Anchors → Full-Corpus Search)
Entity-Diverse Re-Ranking + Keyword Priority Boosts - ~99% RAG Accuracy
Dual-Channel Delivery: Azure AI Foundry / Direct APIs + AWS Bedrock
MongoDB ActiveAIModels Registry - Zero-Redeploy Model Management
Mid-Session Model Switching Without Context Loss
Structured 429 rateLimit Envelopes → Live Countdown UI Without Polling
Live Self-Playing Hero Chat Demo (Token-by-Token Streaming Simulation)
Interactive Workflow Simulator - 3 Rotating Scenarios, Live Token Counter
Live Model-Routing Console - 11-Provider Rail + Terminal Traffic Log
IntersectionObserver-Driven Count-Up Stats (Re-Stream Every 5s In View)
Persisted Chat History via Zustand + localStorage
Wrong-Domain Guard with Canonical Redirect (Multi-Deployment Safety)
4-Layer Security Perimeter: CORS → Origin Guard → Internal Key Guard → JWT
Origin-Header Enforcement - Blocks curl/Postman Where CORS Structurally Cannot
Auth + Quota Resolve Before Any Provider Call - Zero Unauthorized LLM Spend
Two-Clock Stream Decoupling: Provider-Paced Reader + UI-Paced Writer
Backlog-Adaptive Chunk Sizing as a Self-Balancing Rate Matcher
Triple Buffering Defeat: gzip Opt-Out + X-Accel-Buffering + Nagle Disabled
Index-Time Query Expansion - Zero LLM Calls Per Retrieval
Server-Side System-Prompt Stripping - Prompt Injection Resistance
Compensation Privacy Enforced at the Retrieval Layer, Not by Prompt
Live Vector-Store Chunk CRUD - Hot-Patch RAG Without Re-Ingest
Fire-and-Forget Persistence + Write-Through Cache (1 Read, Not 3 Queries)
Rule Ordering as Load-Bearing Design in the Intent Router
Coverage-Aware Re-Ranking - Guaranteed Entity Floor Before Score Fill
One Grounding Prompt, Five Provider-Specific Delivery Shapes
Per-Tier Inference Policy - Lower Temperature for Factual Answers

Low-Level Architecture (LLD)

High-Level Architecture (HLD)

Impact

Served as the flagship chat surface of a platform tested with 1000+ active users, processing 18,000+ AI queries from 12,000+ unique visitors and gathering 800+ detailed user feedback that drove 3 major product iterations to a 99% user satisfaction rate - all with no API keys required from users.
Achieved ~99% RAG accuracy and 95% hallucination reduction through the 3-Layer Hybrid RAG pipeline over 10M+ ChromaDB embeddings - Layers 1 and 2 (13 regex intent rules + 12 pre-cached semantic anchors) resolve most queries with zero additional embedding API calls, and full context retrieval completes in <1 second, so every answer is grounded in verifiable production facts instead of generic LLM filler.
Sustained sub-300ms first-token latency across all 24+ models and 30+ streaming endpoints via dual-channel delivery - when a provider degrades, traffic reroutes between Azure AI Foundry and AWS Bedrock with zero downtime, so a single provider outage never takes ARX offline.
Improved perceived response latency by 60% through adaptive SSE streaming (2–8 char chunks, ±3ms jitter, <50ms intervals) combined with the ChromaDB response memory cache - repeated questions replay instantly from cache while feeling identical to live inference, cutting redundant LLM token spend with zero buffering under high load.
Eliminated model-release lag entirely: the MongoDB ActiveAIModels registry means new frontier models (GPT-5.4, Claude Opus 4.8, Grok 4.3) go live to users within minutes of provider release by flipping a database record - zero code changes, zero redeploys - keeping ARX permanently current in a market where model generations turn over monthly.
Reduced API abuse by 99% on infrastructure handling 50K+ daily API calls: every chat request clears JWT auth and Redis-backed rate limiting before any model call is made, so not a single unauthorized request has ever triggered LLM spend - with structured 429 envelopes rendering accurate live countdown timers without any frontend polling.
Kept answers permanently fresh with zero manual content operations: the daily cron re-ingestion pipeline re-embeds all 12 data sections from MongoDB into ChromaDB automatically, so profile, project, and experience updates propagate into RAG answers within 24 hours without anyone touching the pipeline.
Built on the same production AI infrastructure powering systems that reached 600K+ users, drive 80%+ of platform traffic, and run a <40ms AI inference review engine - ARX is a customer-facing window into battle-tested architecture, not a demo assembled for screenshots.
Generated direct inbound opportunity: multiple job calls and offers from AI companies, and 3–4 founders reached out to build similar multi-model chat products after using ARX and seeing the architecture respond in real time - the product itself became the strongest proof of engineering capability.
Made unauthorized model spend structurally impossible rather than merely unlikely: because JWT verification and quota checks both resolve before the controller is entered, there is no code path in which an unauthenticated or over-quota request reaches a provider API - and the 4-layer perimeter rejects a direct curl for a missing Origin header before authentication is even attempted, closing the exact gap that CORS alone structurally cannot cover.
Cut retrieval cost to zero LLM calls per query by moving query expansion into the corpus at index time instead of paying for query rewriting on every request - where a conventional RAG pipeline spends multiple LLM calls per query on rewriting and HyDE, this one spends none, and intent routing resolves the most common questions with no embedding call at all, turning them into pure metadata lookups while the remaining layers absorb everything else.