We Couldn't Use Cloud LLMs. Here's How We Still Built an AI-Powered SDLC.

Air-gapped environments don't have to mean no AI. FlowSquad runs entirely on-premise with Ollama (Llama 3.1/Mistral) and ChromaDB — no data leaves your network.

What Is the Compliance Problem with Cloud LLMs?

For organisations operating under strict data classification requirements — government agencies, defence contractors, financial institutions, healthcare providers — the question of cloud LLM usage isn't a preference, it's a compliance boundary. The specific concerns vary by regulatory framework but cluster around several issues:

• Data residency: Source code, requirements documents, and design specifications often contain information subject to jurisdictional data residency requirements (e.g., EU GDPR, UK DPA 2018, US Executive Order 13556 for CUI). Sending code to an LLM API whose servers are in a different jurisdiction may constitute a violation.
• Training data exposure: Early versions of some commercial LLM APIs used user inputs for model training. While modern enterprise agreements typically include opt-outs, the risk of contractual and configuration gaps is unacceptable for many regulated entities.
• Network security boundary violations: In classified and IL4/IL5 environments, no data may traverse the internet regardless of encryption. This categorically prohibits any cloud API call.
• Vendor security audit requirements: Many regulated organisations require security audits of all third-party services before use. The timeline for auditing a major cloud LLM provider is 12–18 months — too slow for teams that need AI tooling now.

What Is FlowSquad's Air-Gapped Deployment Architecture?

FlowSquad's on-premise deployment replaces every cloud component with a self-hosted equivalent:

Every component in this stack runs within your network perimeter. There are no outbound connections — not for telemetry, not for model updates, not for license validation. FlowSquad's on-premise licence is a static key validated against your deployment's hardware fingerprint at startup.

Key components:

• Ollama — local model serving runtime that runs on Linux, macOS, or Windows with GPU acceleration via CUDA (NVIDIA) or Metal (Apple Silicon). Supports Llama 3.1, Mistral, CodeLlama, Mixtral, and any Ollama-compatible model.
• ChromaDB — open-source vector database that runs as a local service. All code embeddings are stored on-premise.
• Local embedding model — FlowSquad uses nomic-embed-text via Ollama for embedding generation, eliminating the need for any external embedding API calls.
• Field-level encryption — FlowSquad encrypts sensitive stored data (API keys, personal information in requirements) using Fernet (symmetric AES-128) encryption at rest.

How Do You Set Up Ollama + Local ChromaDB?

FlowSquad's 10-step setup wizard includes a dedicated air-gapped configuration path. Below is the core configuration for an on-premise deployment:

# FlowSquad on-premise / air-gapped configuration
# config/flowsquad.yaml

deployment:
  mode: "on_premise"          # disables all external API calls
  airgapped: true             # enforces no-egress mode; any outbound call raises error

llm:
  provider: "ollama"
  ollama:
    base_url: "http://localhost:11434"
    default_model: "llama3.1:70b"     # requires ~40GB VRAM
    code_model: "codellama:34b"        # optimised for code tasks
    fast_model: "mistral:7b"           # for quick scan / low-latency tasks
    timeout_seconds: 120
    num_gpu_layers: -1                 # use all available GPU layers

embeddings:
  provider: "ollama"
  model: "nomic-embed-text"            # local embedding model, no API cost
  dimensions: 768

vectordb:
  provider: "chromadb"
  chromadb:
    host: "localhost"
    port: 8000
    persist_directory: "/data/flowsquad/chromadb"
    collection_prefix: "fs_prod_"

security:
  encryption_at_rest: true
  encryption_provider: "fernet"        # AES-128 symmetric encryption
  key_path: "/secrets/flowsquad.key"   # store outside container

auth:
  mfa_required: true
  session_timeout_minutes: 30
  audit_log_enabled: true
  audit_log_path: "/var/log/flowsquad/audit.jsonl"

What Are the Performance Tradeoffs vs. Cloud LLMs?

Local inference is slower than cloud inference, primarily because commercial cloud LLM providers operate at scale with purpose-built inference hardware. The tradeoffs are manageable with appropriate server provisioning:

For most development workflows, 8–22 second response times are entirely acceptable — the latency is comparable to waiting for a CI pipeline to start. Teams that require lower latency for interactive use (e.g., real-time AskProduct in IDE plugins) should provision Mistral 7B alongside Llama 3.1 70B and use FlowSquad's model routing to assign fast, lightweight queries to the smaller model.

What Regulated Industries Use This Architecture?

FlowSquad's air-gapped deployment is actively used across several regulated sectors:

• Government / Defence: IL4 and IL5 environments where CUI (Controlled Unclassified Information) prohibits internet egress. FlowSquad runs on GovCloud-equivalent private infrastructure with full air-gap.
• Financial services: Investment banks and trading firms where source code contains proprietary algorithms and is classified as a competitive trade secret under firm security policy.
• Healthcare: Hospital systems and health tech companies where code may process or reference PHI (Protected Health Information), triggering HIPAA security rule requirements for all tools in the development toolchain.
• Critical infrastructure: Energy, water, and telecommunications operators subject to NERC CIP or equivalent critical infrastructure protection regulations.