Projects/ARX - AI Architect You Can Talk To
ARX - AI Architect You Can Talk To
42 technologies · 27 highlights
What I Built
- Architected ARX (AI Architect) - a standalone, product-grade conversational AI application at /arx built on Next.js 15 (App Router), React 19, TypeScript 5, and Tailwind CSS 4 - giving direct chat access to 35 frontier models across 11 providers including OpenAI (GPT 6.1 Sol, GPT 6 Luna, GPT 6 Astra, GPT-5.4 Pro, GPT-5.3 Codex, GPT-5.4 Mini), Anthropic (Claude Opus 5, Opus 4.8, Opus 4.7, Opus 4.6, Opus 4.5, Opus 4.1, Sonnet 4.6, Sonnet 4.5), Google (Gemini 3.8 Flash, Gemini 3.5 Flash, Gemini 3.1 Pro, Gemini 3 Flash, Gemma 3 27B), xAI (Grok 4.7, Grok 4.3, Grok 4.2 Reasoning, Grok 4.2 Non-Reasoning), DeepSeek (V4 Pro, V4 Flash), Meta (LLaMA 4 Maverick), Mistral (Mistral Large 3), Moonshot (Kimi K2.6, K2.5), Qwen (Qwen3 VL), Nvidia (Nemotron Nano), and MiniMax (MiniMax M2) - delivered over four provider lanes across 3 clouds - Azure Foundry, Azure Anthropic, Google Gemini and AWS Bedrock through one provider-aware streaming endpoint with sub-300ms first-token latency.
- Engineered the full request pipeline surfaced to users as a 6-stage animated workflow: Your Query → JWT Auth & Rate-Limit → RAG Retrieval (ChromaDB vector search pulling full personal context in <1s) → Model Router (picks the best of 35 frontier models) → AI Inference → Token Streaming - every stage backed by the real production backend: Passport.js JWT validation, Redis-backed rate limiting with structured 429 rateLimit envelopes (limit, remaining, resetInSeconds) that the frontend converts into live countdown timers without polling, and SSE token streaming rendered incrementally in the chat UI.
- Grounded every answer through a 3-Layer Hybrid RAG pipeline over 10M+ ChromaDB embeddings (Azure OpenAI text-embedding-3-large on self-hosted Azure VMs): Layer 1 - 13 regex intent-detection rules for zero-latency section routing; Layer 2 - 12 pre-cached semantic anchor embeddings compared via cosine similarity with no extra API calls; Layer 3 - full-corpus similarity search fallback; combined with entity-diverse re-ranking (guarantees ≥1 document per company/project) and keyword priority boosts (critical=+3, high=+1) - achieving ~99% RAG accuracy and 95% hallucination reduction so ARX answers about real experience, real architecture decisions, and real production numbers instead of generic LLM output.
- Built a four-lane model delivery layer: Azure Foundry serves GPT, Grok, DeepSeek, LLaMA, Mistral and Kimi through one OpenAI-compatible v1 client with the deployment chosen per request; Azure Anthropic serves Claude Opus 5, Opus 4.8/4.7/4.6/4.5 and Sonnet 4.6 through one shared Anthropic Foundry SDK client; Google Gemini serves Gemini 3.8 Flash, 3.5 Flash, 3.1 Pro and 3 Flash through Google GenAI; and AWS Bedrock (ConverseStreamCommand) serves the Bedrock-namespaced Claude models plus Qwen3 VL, Gemma 3, MiniMax M2, Nemotron Nano and Kimi K2.5 - with a MongoDB ActiveAIModels registry carrying a required provider field so a model can be enabled, reordered or disabled at runtime with zero redeploys; the frontend picker hydrates from this registry via a Zustand aiModelsStore, groups models by provider lane, and lets users switch between any of the 35 models mid-session without losing conversation context.
- Delivered an adaptive token streaming architecture over Server-Sent Events with dynamic chunk sizing (2–8 characters per write) and ±3ms jitter timing to prevent synchronization artifacts across providers with wildly different native token cadences, sustaining <50ms chunk intervals uniformly across all 35 models - complemented by a ChromaDB-backed response memory cache that re-streams previously computed answers chunk-by-chunk so cached responses feel identical to live inference, eliminating redundant LLM API calls for repeated questions and improving perceived latency by 60%.
- Developed an MCP-compliant prompt engineering layer with dynamic system/user role injection and adaptive tone control (professional, casual, friendly, technical) so the same grounded facts render in the register the user asked for, plus Redis-backed long-term session memory preserving conversational context across messages and reconnects; coupled with a daily cron-driven RAG re-ingestion pipeline that automatically clears and re-embeds resume/project data from MongoDB into ChromaDB across 12 structured sections (salary, experience, projects, education, GitHub, mentorship, DSA, skills, achievements, social media, personal, major contributions) - ARX never answers from stale context with zero manual content operations.
- Solved the cold-visitor trust problem by making the product demonstrate itself: a live HeroChat that streams a real grounded answer token-by-token before signup, an animated replay of the actual 6-stage request pipeline with live token counting across 3 real Q&A scenarios, and an 11-provider routing console mirroring real dispatch behavior - the architecture is the demo, with next/dynamic code-splitting keeping the animation-heavy page fast on first load.
- Rebuilt model dispatch around a declarative model catalog: every model is one spec - its catalog
modelid,provider(azure|google|aws), a humanproviderLabel, its stream protocol (openai-chat,openai-responses,anthropic,geminiorbedrock), prompt builder and RAG depth - and a single MODEL_CATALOG (model id → spec + handler) derived from those specs is the only source for chat, the Versus arena and voice. The client calls one unifiedaskModelStream(model, provider)transport againstPOST /cognitiveServices/stream, which resolves the spec and rejects an unknown model or a provider mismatch with a 400 before any provider call; legacy per-model URLs are generated from the same specs in a loop that throws at boot on a duplicate path. Adding a frontier model on an existing protocol is now one spec entry plus one seed entry - no new controller, no new route line, and zero conditional routing logic in the UI. - Engineered the askAI request lifecycle as a guarded state machine: 120-character input limit → auth check (unauthenticated users routed to /arx/chat/login) → isProcessingRef re-entrancy lock blocking double-submits → abortRef.current?.abort() cancels any in-flight stream before creating a fresh AbortController → optimistic user-message append → isThinking state held until the transport's onStart callback fires on the first received byte → a fullResponse accumulator mirrors every chunk into live streamingContent state → on completion the streamed buffer commits to the message list after a 40ms settle window that prevents double-render flicker at the stream/committed boundary - with AbortError explicitly distinguished from real failures (DOMException name check) so intentional cancels never surface an error bubble, and a finally block guaranteeing loading/processing flags reset on every exit path.
- Implemented two custom streaming transports (streamFetcher for raw token streams, sseStreamFetcher for typed JSON events): fetch POST with credentials:include → res.body.getReader() with incremental TextDecoder decoding (stream: true); the SSE variant maintains a partial-line buffer - splitting on newlines, re-buffering the trailing fragment, and parsing only
data:-prefixed lines - so JSON events survive network chunk boundaries that split mid-event; both transports check signal.aborted on every read-loop iteration and call reader.cancel() for leak-free teardown; HTTP 429 responses parse the structured rateLimit payload (success, message, rateLimit.resetInSeconds with a safe fallback shape when the body isn't JSON) and push resetInSeconds into the global navbar store - driving accurate countdown timers across the app with zero polling. - Unified cross-product chat history in a single Zustand chatHistoryStore holding three collections (chats, bestvsbest, voicechat) shared by ARX, Versus, and Voice - hydrated by exactly one getUserChatHistory call per authenticated session (an isFetched flag prevents refetch storms), with every completed answer optimistically prepended via prependChat carrying modelId ({name, model}), ragChunksUsed, fromMemory, and responseTime metadata before any server round-trip - history appears instantly in the sidebar and survives reloads.
- Engineered the chat UX mechanics: a stickRef tracks whether the user is parked within 80px of the scroll bottom via a passive scroll listener - auto-follow engages during token streaming only while the user hasn't scrolled up and re-engages on send; in-chat search with match-index navigation, a measured model-chip width injected as textarea padding so the placeholder never overlaps the selected-model chip, React Markdown + remark-gfm with syntax-highlighted code blocks, skeleton hydration (ArxChatSkeleton), isAuthChecked-gated route protection (redirect fires only after the auth check resolves - no login-page flash for valid sessions), a dedicated login page (email+OTP, Google OAuth, GitHub OAuth), and a wrong-domain guard with canonical redirect to www.princesinghai.com/arx - all dark/light themed and code-split via next/dynamic.
- Hardened the platform behind a 4-layer security perimeter where every layer catches exactly what the previous one structurally cannot: a CORS allowlist (browser-enforced only) → an Origin guard rejecting any request with a missing or non-allowlisted Origin header - the check that actually stops
curland Postman, since a direct HTTP client ignores CORS entirely → an internal-key guard requiring a shared secret held only by the server-side proxy, making the API unreachable except through the application itself → Passport JWT re-read against the database on every request, so a deactivated account's un-expired token stops working immediately. - Engineered the streaming layer as a two-clock decoupling that normalizes 5 structurally different provider stream shapes into one identical cadence: a provider-paced reader loop only fills a buffer, while an independent UI-paced writer loop drains it on a fixed frame using backlog-adaptive chunk sizing (2/4/6/8 characters at <20/<80/<200/≥200 buffered) - a self-balancing rate matcher that sizes up to drain a fast provider's backlog and sizes down to stretch a slow one smoothly, so every model types at the same rhythm and the delivery channel is invisible to the user.
- Solved the three independent buffering layers that silently destroy paced streaming in production:
Content-Encoding: identityto opt out of gzip, whose compression window would batch carefully-paced 2–8 character chunks back into bursts;X-Accel-Buffering: noto stop the reverse proxy doing the same thing a layer higher; andsocket.setNoDelay(true)to disable Nagle's algorithm so small writes leave the kernel immediately instead of coalescing - without all three, the typewriter effect collapses into stutter no matter how well the writer loop is tuned. - Designed the RAG corpus as deterministically generated semantic chunks across 12 sections from a hand-written template generator rather than a recursive text splitter - because the source is structured data, not prose - and moved the expensive work to index time instead of query time: synonyms and alternate phrasings are embedded directly into chunks as
SEARCH KEYWORDS, so retrieval costs zero LLM calls per query where a conventional pipeline spends multiple calls on query rewriting and HyDE; redundant confirmation chunks written as explicit question-and-answer pairs make common recruiter questions embed close to their answer form, and priority-weighted metadata guarantees curated summary chunks survive re-ranking instead of being crowded out by literal keyword matches. - Enforced grounding through a strict prompt contract rather than trusting the model to behave: retrieved context is injected as discretely numbered units so the anti-hallucination rule has something concrete to point at, followed by explicit constraints and a self-check validation list - and critically, any client-supplied system message is stripped server-side, so the system prompt cannot be injected or overridden from the request; when retrieval returns nothing the context block is simply empty, and the contract converts a would-be hallucination into a graceful in-persona admission instead.
- Built a diagram delivery pipeline that stores raw architecture-diagram source inside the RAG corpus and hands the model a pre-repaired diagram to place verbatim - instead of asking an LLM to reproduce diagram syntax from memory, which reliably corrupts it - with a custom sanitizer that repairs malformed edge labels, unbalanced delimiters, incomplete subgraph blocks, reserved characters inside node labels and every arrow variant, wired to an instruction builder that switches between generating a diagram from scratch, placing the stored one, or showing it as reference only.
- Implemented quota control as a structural cost guarantee: a Redis-backed limiter keyed on a composite user-ID + IP - neither alone is sufficient, since a user ID alone lets one person burn quota across devices while an IP alone punishes everyone behind a corporate NAT - deliberately made global across all models rather than per-model so quota cannot be multiplied by switching models, with a fallback to an in-process store when Redis is unavailable so limits degrade rather than disappear, and a
resetInSecondsread from the actual Redis TTL rather than a recomputed estimate - making the client countdown accurate to the second with zero polling. - Made compensation privacy a property of the retrieval layer rather than a prompt instruction: the salary chunk is a single dense, numbers-heavy document, which makes it a strong semantic attractor for a wide range of unrelated career questions - so it is unconditionally excluded from every fallback retrieval path and reachable only when intent detection explicitly resolves a compensation question, guaranteeing that salary can never leak into an answer about architecture, projects or experience by construction instead of by hoping the model stays on topic.
- Shipped the operational surface that makes runtime configuration real instead of aspirational: role-guarded admin endpoints providing live vector-store chunk CRUD - page through, read, update or delete an individual embedding chunk - so a single incorrect RAG chunk is hot-patched in place without a full re-ingest; and a model-registry cache refresh that completes the zero-redeploy loop, since flipping a model's active flag alone changes nothing while the registry cache is deliberately stored without expiry - the database flip plus the cache refresh is the actual operation.
- Made resilience an explicit design property rather than an accident: chat persistence is deliberately fire-and-forget, so database latency can never block an answer that is already streaming, and every save writes through to the cache, turning a subsequent history load into one cache read instead of three database queries; on failure, cache loss drops the limiter to an in-process store while reads fall through to the database, an embedding failure yields an honest refusal instead of a fabrication, and a client disconnecting mid-stream still completes caching and persistence - so provider spend already incurred is never wasted.
- Engineered the zero-latency intent router so that rule ordering is load-bearing rather than incidental: compensation is evaluated first, so "what package did you get at ProPeers?" routes to salary instead of employment; company-specific rules precede the generic experience rule so a named employer narrows to that employer rather than returning every job; named projects precede the generic "tell me about" rule so a product question never degrades into a biography; and the router carries fuzzy matching for commonly misspelled company names plus geographic inference - a question about the "US company" resolves to the correct employer because that fact is encoded into the routing layer - all resolved as pure metadata lookups with no embedding call and no LLM call.
- Solved a retrieval-quality failure that pure relevance scoring cannot see: because one employer had roughly four times the indexed content of another, a question like "tell me about your work experience" returned four chunks about a single company and zero about the rest - each individually top-scoring, collectively a bad answer, since the user asked about a career, not a company. The fix reserves a guaranteed floor of result slots per distinct entity before filling the remainder by score, then re-sorts for presentation so the strongest chunk still leads - giving breadth across companies and projects without sacrificing the top result, and turning a relevance ranker into a coverage-aware one.
- Absorbed real provider-level incompatibilities behind one uniform internal contract instead of leaking them upward: the same grounding prompt is delivered five structurally different ways - as a true system message, as a top-level system parameter, as an
instructionsfield, as a single flattened prompt string, and - on the Bedrock channel, which accepts only user and assistant roles and rejects anything else - as a prepended user turn with every history role normalized to a permitted value; paired with a per-tier inference policy that assigns lower temperature to the models used for the most factual answers and looser sampling to the rest, so adding a provider never changes a single line of dispatch logic. - Engineered provider-aware identity so the same model on two clouds is never confused: Claude Opus 4.6, Opus 4.5 and Sonnet 4.6 run on both Azure Anthropic and AWS Bedrock, so every request carries
{ model, provider }, the registry stores the provider on each record, the picker shows the lane next to the name, and every server log line is tagged with the lane and model (for example[Azure Anthropic · CLAUDE-OPUS-4-8]) - turning "which cloud actually served this answer?" from a guess into a grep, and making provider-level incidents attributable in seconds. - Ran every request through one orchestration flow instead of a controller per model: memory lookup → RAG retrieval → provider-shaped prompt → paced stream → memory commit → chat log, with the five provider protocols isolated as stream adapters behind a single paced writer - so the latest generation (GPT 6.1 Sol, GPT 6 Luna/Astra, Claude Opus 5, Gemini 3.8/3.5 Flash, Grok 4.7) went live by declaring specs, not writing code paths, and was verified end to end across chat, Versus and voice before release.
Demo Video
Tech Stack
Next.js 15 (App Router)React 19TypeScript 5Tailwind CSS 4Framer Motion 12Zustand 5 (authStore, aiModelsStore, chatHistoryStore - persisted)Server-Sent Events (SSE token streaming)React Markdown + remark-gfmreact-syntax-highlighterOpenAI (GPT 6.1 Sol, GPT 6 Luna, GPT 6 Astra, GPT-5.4 Pro, GPT-5.3 Codex, GPT-5.4 Mini)Anthropic (Claude Opus 5, Opus 4.8, Opus 4.7, Opus 4.6, Opus 4.5, Opus 4.1, Sonnet 4.6, Sonnet 4.5)Google AI (Gemini 3.8 Flash, Gemini 3.5 Flash, Gemini 3.1 Pro, Gemini 3 Flash, Gemma 3 27B)xAI (Grok 4.7, Grok 4.3, Grok 4.2 Reasoning, Grok 4.2 Non-Reasoning)DeepSeek (V4 Pro, V4 Flash)Meta (LLaMA 4 Maverick)Mistral (Mistral Large 3)Moonshot (Kimi K2.6, K2.5)Qwen (Qwen3 VL 235B)Nvidia (Nemotron Nano 3 30B)MiniMax (MiniMax M2)Azure AI Foundry (Anthropic Foundry SDK)AWS Bedrock (ConverseStreamCommand)ChromaDB (10M+ embeddings, self-hosted Azure VM)Azure OpenAI Embeddings (text-embedding-3-large)MongoDB ActiveAIModels Registry (runtime model config)Redis (rate limiting with structured 429 envelopes)JWT Authentication (Passport.js)Google OAuth 2.0 / GitHub OAuthAWS CloudFront CDNIntersectionObserver (scroll-driven animation orchestration)Node.js + Express.js (streaming REST API)Passport.js (7 strategies - JWT + Google/GitHub per deployment domain)OpenAI Responses API (instructions + input, delta event stream)Google GenAI (thinkingConfig - low/high reasoning levels)AWS Bedrock ConverseCommand (non-streaming variant)ChromaDB Metadata Filtering ($eq / $in / $nin / $and)Async Reader/Writer Stream Pacing (Node.js event loop)HTTP Chunked Transfer - gzip opt-out, X-Accel-Buffering, Nagle disabledAzure AI Foundry v1 (one OpenAI-compatible client - GPT, Grok, DeepSeek, LLaMA, Mistral, Kimi)Google GenAI (Gemini 3.8 Flash, 3.5 Flash, 3.1 Pro, 3 Flash)MODEL_CATALOG (model id → spec + handler, provider-aware)Unified POST /cognitiveServices/stream ({ model, provider })
Key Concepts
- Product-Grade AI Chat App - 35 Frontier Models, 11 Providers, 4 Provider Lanes, 3 Clouds
- 6-Stage Pipeline: Query → Auth/Rate-Limit → RAG Retrieval → Model Router → Inference → Token Streaming
- 3-Layer Hybrid RAG (13 Regex Intent Rules → 12 Semantic Anchors → Full-Corpus Search)
- Entity-Diverse Re-Ranking + Keyword Priority Boosts - ~99% RAG Accuracy
- Four Provider Lanes: Azure Foundry · Azure Anthropic · Google Gemini · AWS Bedrock
- MongoDB ActiveAIModels Registry - Zero-Redeploy Model Management
- Mid-Session Model Switching Without Context Loss
- Structured 429 rateLimit Envelopes → Live Countdown UI Without Polling
- Live Self-Playing Hero Chat Demo (Token-by-Token Streaming Simulation)
- Interactive Workflow Simulator - 3 Rotating Scenarios, Live Token Counter
- Live Model-Routing Console - 11-Provider Rail + Terminal Traffic Log
- IntersectionObserver-Driven Count-Up Stats (Re-Stream Every 5s In View)
- Persisted Chat History via Zustand + localStorage
- Wrong-Domain Guard with Canonical Redirect (Multi-Deployment Safety)
- 4-Layer Security Perimeter: CORS → Origin Guard → Internal Key Guard → JWT
- Origin-Header Enforcement - Blocks curl/Postman Where CORS Structurally Cannot
- Auth + Quota Resolve Before Any Provider Call - Zero Unauthorized LLM Spend
- Two-Clock Stream Decoupling: Provider-Paced Reader + UI-Paced Writer
- Backlog-Adaptive Chunk Sizing as a Self-Balancing Rate Matcher
- Triple Buffering Defeat: gzip Opt-Out + X-Accel-Buffering + Nagle Disabled
- Index-Time Query Expansion - Zero LLM Calls Per Retrieval
- Server-Side System-Prompt Stripping - Prompt Injection Resistance
- Compensation Privacy Enforced at the Retrieval Layer, Not by Prompt
- Live Vector-Store Chunk CRUD - Hot-Patch RAG Without Re-Ingest
- Fire-and-Forget Persistence + Write-Through Cache (1 Read, Not 3 Queries)
- Rule Ordering as Load-Bearing Design in the Intent Router
- Coverage-Aware Re-Ranking - Guaranteed Entity Floor Before Score Fill
- One Grounding Prompt, Five Provider-Specific Delivery Shapes
- Per-Tier Inference Policy - Lower Temperature for Factual Answers
- Declarative Model Catalog - One Spec Entry per Model, No New Controller or Route
- Provider-Aware Dispatch: { model, provider } → 400 on Unknown Model or Lane Mismatch
- Provider-Grouped Model Picker + Provider-Tagged Server Logs
- One Orchestration Flow: Memory → RAG → Prompt → Paced Stream → Commit → Log
Low-Level Architecture (LLD)
High-Level Architecture (HLD)
Impact
- Served as the flagship chat surface of a platform tested with 1000+ active users, processing 18,000+ AI queries from 12,000+ unique visitors and gathering 800+ detailed user feedback that drove 3 major product iterations to a 99% user satisfaction rate - all with no API keys required from users.
- Achieved ~99% RAG accuracy and 95% hallucination reduction through the 3-Layer Hybrid RAG pipeline over 10M+ ChromaDB embeddings - Layers 1 and 2 (13 regex intent rules + 12 pre-cached semantic anchors) resolve most queries with zero additional embedding API calls, and full context retrieval completes in <1 second, so every answer is grounded in verifiable production facts instead of generic LLM filler.
- Sustained sub-300ms first-token latency across all 35 models over four provider lanes on three clouds - because several frontier models are served on both Azure Anthropic and AWS Bedrock, a degraded lane is bypassed by picking the same model on the other cloud, so a single provider outage never takes ARX offline.
- Improved perceived response latency by 60% through adaptive SSE streaming (2–8 char chunks, ±3ms jitter, <50ms intervals) combined with the ChromaDB response memory cache - repeated questions replay instantly from cache while feeling identical to live inference, cutting redundant LLM token spend with zero buffering under high load.
- Eliminated model-release lag: with the declarative model catalog and the ActiveAIModels registry, new frontier models (GPT 6.1 Sol, GPT 6 Luna, GPT 6 Astra, Claude Opus 5, Gemini 3.8 Flash, Gemini 3.5 Flash, Grok 4.7) shipped as one spec plus one seed entry each and went live in chat, Versus and voice at once, while retiring a model is a registry flip with zero redeploys - keeping ARX permanently current in a market where model generations turn over monthly.
- Reduced API abuse by 99% on infrastructure handling 50K+ daily API calls: every chat request clears JWT auth and Redis-backed rate limiting before any model call is made, so not a single unauthorized request has ever triggered LLM spend - with structured 429 envelopes rendering accurate live countdown timers without any frontend polling.
- Kept answers permanently fresh with zero manual content operations: the daily cron re-ingestion pipeline re-embeds all 12 data sections from MongoDB into ChromaDB automatically, so profile, project, and experience updates propagate into RAG answers within 24 hours without anyone touching the pipeline.
- Built on the same production AI infrastructure powering systems that reached 600K+ users, drive 80%+ of platform traffic, and run a <40ms AI inference review engine - ARX is a customer-facing window into battle-tested architecture, not a demo assembled for screenshots.
- Generated direct inbound opportunity: multiple job calls and offers from AI companies, and 3–4 founders reached out to build similar multi-model chat products after using ARX and seeing the architecture respond in real time - the product itself became the strongest proof of engineering capability.
- Made unauthorized model spend structurally impossible rather than merely unlikely: because JWT verification and quota checks both resolve before the controller is entered, there is no code path in which an unauthenticated or over-quota request reaches a provider API - and the 4-layer perimeter rejects a direct
curlfor a missing Origin header before authentication is even attempted, closing the exact gap that CORS alone structurally cannot cover. - Cut retrieval cost to zero LLM calls per query by moving query expansion into the corpus at index time instead of paying for query rewriting on every request - where a conventional RAG pipeline spends multiple LLM calls per query on rewriting and HyDE, this one spends none, and intent routing resolves the most common questions with no embedding call at all, turning them into pure metadata lookups while the remaining layers absorb everything else.