AI Companies & Large Language Models

Trading firms are among the most aggressive adopters of AI, but they rarely release public models. Their systems are proprietary, trained on petabytes of market data, and deployed on custom silicon.

HRT AI Labs (HAIL)

• Foundation Models for Markets: Large-scale models trained on decades of market data. Dataset exceeds 100 TB.
• Approach: Treats market prediction as sequence modeling. Multi-task learning + LLMs.
• Scale: Every day, HRT models process terabytes of market data.
• LLM Team: Builds LLMs from scratch — pretraining, post-training, serving.
• RL Research: RL agents learn optimal trading policies (optimal execution).
• Compute: NVIDIA Rubin platform and Vera Rubin NVL72.

AI & LLM Usage

• LLM Feature Forecasting: Parses two decades of Federal Reserve speeches to predict rate changes.
• Internal Workbench: Secure internal environment for LLM-driven feature extraction.
• Scale: Thousands of independent models, each with its own market view.
• AI Framework: Advisory (insights), Oracle (predictions), Operational (workflow automation).

AI & Model Approach

• Custom Model Diversity: Massive architectural diversity vs "one model to rule them all."
• Data Center: Building own 100–200 MW data center for internal AI.
• Anthropic Investment: Major stake in Anthropic.
• Dormant LLM Puzzle (2026): Public challenge — hidden backdoor triggers in fine-tuned LLMs (Qwen2.5-7B-Instruct warmup, DeepSeek-V3 671B M1/M2/M3).

AI Trading System

• Scale: ~250 employees, processes 400+ PB of global financial market data.
• Cross-Asset Analysis: 50,000+ financial instruments simultaneously.
• Daily Volume: ~$370 trillion daily volume via AI-driven algorithms for bid-ask spreads.
• Wealth: Founder Alex Gerko — UK top-10 wealthiest, £16B (~$21B).

GPT-6 Astra Adoption

• First Adopter: One of the first to adopt GPT-6 Astra for quant research.
• Long-Horizon Research: Runs for days across many data sources.
• Autonomy: Lucas Baker (Head of LLM R&D) — "a new tier of autonomy for long-horizon tasks."
• Infrastructure: NVIDIA Rubin platform and Vera Rubin NVL72.

Agentic AI Research System

• 8 Weeks in 2 Hours: Reads academic papers, reproduces analysis, verifies, runs out-of-sample — 6–8 weeks → 2–3 hours.
• Labor Reduction: ~90% cost reduction for research reproduction.
• AI Assistant: Chatbot trained on licensed content highlights risks + generates research.

LLM Gateway & RAG Infrastructure

• LLM Gateway: Logs every call, strips PII, throttles per-desk budgets.
• DocLab: Adds audit hashes to retrievals.
• RAG at Scale: 400,000+ documents. Monthly bill ~$15,000.
• Quant Tools: Build tools in ~10 lines of code.

Medallion Fund & AI Approach

• Performance: Medallion Fund returned 66% annually (gross) since 1988 — the most successful hedge fund in history.
• ML Since Before It Was Cool: ~300 employees, ~100 PhDs in math/physics/CS.
• Model Approach: Thousands of features + mean-variance optimization + volatility contribution.
• Latency: <100 microseconds.
• Idea Generator: AI reads arXiv + SSRN nightly, extracts hypotheses, writes structured work orders.

AI Model Evaluation & Trading Tests

• AI Trading Test: Put LLMs through trading intern exam — median intern = 61%, AI still falls short.
• Model Evaluation: Claude Fable 5 showed improvement at max reasoning effort.
• Sentiment Analysis: Quantized Transformer (.onnx) for ticker sentiment.
• Market Data as Deep Learning: Order book data as deep learning challenge.

Foundation Model Training

• CoreWeave Partnership: Primary AI cloud for foundation model training.
• Dedicated Compute: Migrating to AI-native platforms for frontier-scale trading.
• Foundation Models for Trading: Domain-specific models for quantitative trading.

Open Source in Trading

• Kaggle Competition: Released crypto market prediction competition with proprietary trading signals (X1 to X890).
• Open Source Models Talk: Wachi Bandara — "Free Like a Puppy, Not Like a Beer."
• Belief: AI makes markets more efficient, drove down trading costs.

EmbeddingGemma 2 (Google)

• What it is: Open-weight multimodal embedding model under Apache 2.0. Maps text, code, images, video, and audio into a unified 768-dimensional space.
• Architecture: Modular encoders — 270M params for text/code, +170M for vision, +300M for audio, total 740M for full multimodal[reference:17].
• Memory: ~191MB active RAM for text-only, ~567MB for full multimodal on Google Pixel 11 Pro[reference:18].
• Use case: On-device RAG, local codebase indexing, private semantic search without cloud dependency.
• Links: Google Blog

1-bit Bonsai 2 (PrismML)

• What it is: 1-bit precision VLM series from PrismML. The 2B variant runs on Snapdragon AR1 Gen 1 Platform on smart glasses.
• Efficiency: Delivers 4-bit AI model equivalent inference with 4× lower memory and 2× faster token generation[reference:19].
• Ternary Bonsai 2 27B: Miniaturized Qwen3.8 27B to 5.9GB while maintaining 98.2% performance — runs on laptops[reference:20].
• Use case: Smart glasses, wearables, IoT, edge robotics.
• Links: PrismML

Muse Glimmer 30B (Meta)

• What it is: 30B dense multimodal model, Apache 2.0, distilled from Muse Spark 1.2. Built for autonomous agentic tasks on consumer hardware.
• Hardware: Runs on 18GB VRAM (4-bit quantized). Context >120K, 20K tok/s throughput on Blackwell Ultra[reference:21].
• Use case: Local agents, function calling, local coding, LLM-as-a-judge.
• Links: Hugging Face

Core Generation

Multimodal

Reasoning

Context & Memory

Cost & Performance

Safety & Control

🆕 Trading & Financial AI

Prompt Engineering

Cost Optimization

Security

Production

Choosing the Right Model

Flagship Model Comparison

SWE-bench Verified Leaderboard (Oct 2026)

Source: BenchLM SWE-bench Verified Leaderboard, October 7, 2026[reference:24]

Terminal-Bench-Science 0.1 Leaderboard

Source: BenchLM Terminal-Bench-Science 0.1 Leaderboard, October 2026[reference:25]

Open-Source LLM Leaderboard (Oct 2026)

Source: BenchLM Open-Source LLM Leaderboard, October 2026[reference:26]

Cost Leaderboard (input $/M)

Context Window Leaderboard

Open Weights vs Proprietary

1. Transformer Architecture (2017–2026)

• Exploration (2018–2019): Encoder-, decoder-, encoder-decoder designs. Decoder-only prevailed.
• Scaling (2020–2022): Capability bought with parameters and data.
• Efficiency (2023–2024): Inference cost → GQA, MLA, sliding windows, paging, MoE.
• Convergence (2025–2026): Full-attention + linear mixers + routed experts. Subquadratic attention (SSA) emerges as a new paradigm.

2. Mixture-of-Experts (MoE)

Sparse, token-level routed experts. Example: DeepSeek-V3 uses E=256, k=8, shared=1. Nemotron 3 Ultra = 550B/55B. Qwen3.8 2.4T A95B. GLM-5 = 744B. Kolibri = 78.1B/3.46B active.

3. Multi-Head Latent Attention (MLA)

DeepSeek-V2/V3 — compresses KV cache into low-rank latent space. Critical for long context.

4. Subquadratic Selective Attention (SSA)

New in 2026. Subquadratic's SSA architecture scales linearly with input length instead of quadratically. Selectively focuses only on token comparisons that matter. Reduces attention compute by nearly 1,000× at 12M tokens. Runs 52× faster than FlashAttention at 1M tokens[reference:27].

5. Tokenization: BPE & SentencePiece

• BPE: GPT-2, GPT-4, Claude, LLaMA, Mistral, Falcon.
• SentencePiece: Language-agnostic. Google's Gemma.

6. Training Pipeline

• Pretraining: Learn general linguistic understanding. GLM-5 = 28.5T tokens.
• SFT: Curated instruction-response pairs.
• Alignment (RLHF/DPO/RLAIF): Shape behavior. 2026 shift toward DPO and RLAIF.

7. Reasoning Models: Test-Time Compute

o-series, DeepSeek R1, Gemini Thinking, Claude Adaptive Thinking, Big Pickle, Gemini 4 Argon. Reasoning tokens bill as output.

8. KV Cache & Inference Optimization

• PagedAttention (vLLM): KV cache like virtual memory. Up to 24× throughput.
• MLA: Compresses KV into low-rank latent vectors.
• GQA: Shares K/V heads across query heads.
• DSA (DeepSeek Sparse Attention): 2026 — learns which tokens to attend. Integrated into GLM-5.
• SSA (Subquadratic Selective Attention): 2026 — fully subquadratic. Linear scaling. 52× faster at 1M tokens.

Safetensors · GGUF · MLX

• Safetensors: Hugging Face. Safe PyTorch weights. GPU inference, fine-tuning.
• GGUF: llama.cpp. Local CPU/consumer. Ollama, LM Studio, Jan.
• MLX: Apple Silicon. macOS with mlx-lm.

GGUF Quantization Levels

Q4_K_M recommended for most local use. VRAM rule: ~2 GB per 1B params at Q4.

GPU Quantization

• GPTQ: 4-bit weights, 16-bit activations (W4A16). Best accuracy for GPU.
• AWQ: Protects salient weights. Often slightly better quality.
• BitsAndBytes: On-the-fly at load.
• FP8: Native 8-bit on H100/B200.
• 1-bit / Ternary: PrismML Bonsai — 4× lower memory, 2× faster. 98.2% of full precision at 5.9GB for 27B model.

Decision Framework

• Cloud API: Prototyping, rare complex queries, frontier reasoning, zero infrastructure.
• Self-Hosted: Production above sustained-usage floor, data residency, fine-tuned checkpoints.
• Edge / On-Device: Offline, privacy, latency-critical, wearable/embedded. Muse Glimmer, Bonsai, EmbeddingGemma.

4-Layer Self-Hosting Stack (2026)

• Layer 1 — Inference Server: vLLM (highest throughput), SGLang (agentic), TGI (HF).
• Layer 2 — Orchestrator: KubeAI or Ray Serve.
• Layer 3 — Gateway: LiteLLM. D.E. Shaw's LLM Gateway = enterprise gold standard.
• Layer 4 — Deployment & Governance: Qovery, Terraform + CI.

Cost Analysis

Compute cost per M tokens = GPU hourly ÷ (output tok/s × 3600 ÷ 1e6) ÷ utilisation. Add 20–40% for platform engineering.

Inference Server Comparison

Ollama

• What: Model manager + API wrapping llama.cpp. OpenAI-compatible on localhost:11434.
• Install: ollama.com/download
• Usage: ollama pull qwen2.5:3b · ollama run qwen2.5:3b · ollama list · ollama serve
• Tags: qwen2.5:3b, llama3.2:3b, phi4-mini, gemma3:4b, deepseek-r1:7b, yi:34b, glm4:9b, muse-glimmer:30b
• Links: Tool Calling · CLI

llama.cpp · Unsloth Studio · LM Studio · Apple MLX

• llama.cpp: Raw C/C++ engine. GitHub
• Unsloth Studio: Desktop app for running + training locally. unsloth.ai
• LM Studio: GUI for local models. lmstudio.ai
• Apple MLX: uv tool install mlx-lm. GitHub

Hardware Sizing

Key Methods

• Full Fine-Tuning: All weights. Expensive. Rarely above 7B.
• LoRA: 99% fewer trainable params. Quality often indistinguishable from full fine-tuning.
• QLoRA: 4-bit base + LoRA. 60% less memory. Enables 70B on one 48GB GPU.

Fine-Tune vs RAG vs Prompt Engineering

Frameworks

• Unsloth: 2–5× faster, 80% less memory.
• HF PEFT: Standard library.
• ShadowLM Trainer: Any model, any method, any hardware.

RAG Pipeline

• Indexing: Chunk, embed, store in vector DB.
• Retrieval: Dense vector, BM25, or hybrid.
• Generation: LLM conditioned on retrieved context.

2026 trend: GraphRAG for multi-hop reasoning.

LangChain vs LlamaIndex

MCP (Model Context Protocol)

• Stateless core: No protocol-level sessions.
• Adoption: MCP tool calls from ChatGPT users increased 98× in 2026.
• SDKs: C# v2.0, Python, TypeScript, Go.

Agentic Frameworks (2026)

• LangGraph: Graph-based workflows. Docs
• CrewAI: Role-based multi-agent. crewai.com
• Dapr Agents v1.0: Durable workflows.
• Google ADK: Apache-2.0.
• Pydantic AI: ai.pydantic.dev
• NVIDIA NemoClaw: Open AI-agent stack.
• Amazon Bedrock AgentCore: Managed agent orchestration.
• OpenCode: CLI agent for Big Pickle, Claude, GPT.

• 🏆 Largest Context: SubQ 1M-Preview at 12M tokens, then Llama 4 Scout (10M).
• 💰 Cheapest Full 1M-Token: Big Pickle (Free), then DeepSeek V4 Flash ($0.14).
• 📤 Largest Max Output: Gemini 4 Argon at 1M output tokens, then DeepSeek V4 Pro/Flash (384K).
• ⚡ Fastest Inference: Groq's LPU at ~1,000 tok/s on GPT-OSS 20B.
• 🧠 Most Expensive Flagship: GPT-5 Pro at $15/$120 per 1M.
• 🔓 Open Weights Leaders: Xiaomi (MiMo-V2.6-Pro #1), Alibaba (Qwen3.8 Max #2), Zhipu (GLM-5.3 #3), Meta (Muse Glimmer 30B), Google (EmbeddingGemma 2).
• 🕵️ Stealth Model Leaders: Big Pickle (11+ months), Ox Alpha (GLM-5.3-Flash), Space Bunny, Pony Alpha (GLM-5), Union Alpha (Pareto).
• 🏦 Trading AI Leaders: HRT (foundation models), Two Sigma (LLM features), Jane Street (custom models), XTX ($370T daily), Jump (GPT-6 Astra), Citadel (agentic research), D.E. Shaw (LLM Gateway), Renaissance (66% annual since 1988).
• 🔌 Edge AI Breakthroughs: EmbeddingGemma 2 (740M, multimodal, ~191MB text-only), Bonsai 2 (1-bit, 4× less memory, 2× faster), Muse Glimmer 30B (18GB VRAM, Apache 2.0).
• 🏢 Enterprise Platforms: Azure AI, Bedrock, watsonx aggregate providers with governance.
• 💡 Cost Optimization: Batch 50% off, caching up to 90%, off-peak (DeepSeek), free stealth models.
• 🔬 Architecture Shift: MoE default (GLM-5 744B, DeepSeek V4, Qwen3.8 2.4T). MLA compresses KV. Subquadratic attention (SSA) emerges as a new paradigm.
• 📦 Local Inference: Ollama + llama.cpp + GGUF make 7–14B models a one-liner. Muse Glimmer 30B adds 30B capability.
• 🤖 Agentic Infrastructure: MCP is de facto standard. LangGraph & CrewAI dominate.
• 🔒 Security Alert: Jane Street's dormant LLM challenge — backdoors persist through fine-tuning.
• ⚖️ Governance: EU AI Act revision bans NCII/CSAM from Dec 2, 2026. Gartner: 17% deployed agents, 60% within 2 years.