Exposed AI Infrastructure: 1 Million LLM Services Found Misconfigured and Unprotected

When AI Speed Defeats AI Security

The explosion in enterprise AI deployment has outpaced security practice by a significant margin. A new analysis documented in The Hacker News scanned approximately 1 million internet-facing AI services, drawn from 2 million hosts identified through certificate transparency logs, and found endemic misconfiguration across AI orchestration tools, LLM wrappers, and agentic platforms.

The findings are specific and alarming. Of 5,200+ Ollama servers queried, 31% responded to authentication prompts without requiring credentials — meaning roughly 1,600 instances were open to anyone on the internet. Ollama is a local LLM hosting platform used by developers to run models like Llama 3, Mistral, and Qwen locally or on-server; an open Ollama instance exposes not just the LLM endpoint but any data and business logic accessible through it. Across the broader scan, 518 frontier model instances were identified across scanned servers — meaning production-grade GPT-4, Claude, and Gemini API proxies were accessible without authentication. Ninety-plus exposed instances were found across government, marketing, and finance sectors.

The OWASP GenAI Q1 2026 exploit round-up provides additional context: 12,000 to 15,000 Flowise instances were found vulnerable to active exploitation during the same period. Flowise is a no-code AI workflow builder used to create RAG pipelines, chatbot interfaces, and AI agent orchestration. An exposed Flowise instance gives attackers access to not just the LLM endpoint but the full business logic embedded in the workflow — which may include database connections, API credentials, customer data, and internal process definitions.

The March 2026 LiteLLM supply chain attack, documented by both HeroDevs and Zscaler, adds a third dimension: 36% of cloud environments contained LiteLLM, the Python library that unifies API access to multiple LLM providers. When attackers compromised the PyPI maintainer account and published malicious versions 1.82.7 and 1.82.8, they potentially had access to AI infrastructure in more than one-third of cloud environments. The malware specifically exfiltrated LLM API keys — OpenAI, Anthropic, and Azure OpenAI credentials — alongside cloud infrastructure credentials.

The Three Misconfiguration Patterns Enabling Mass Exposure

Understanding why AI infrastructure is so consistently insecure requires examining the structural patterns that enable these exposures at scale.

Pattern 1: Authentication Disabled on Fresh Install. Ollama, n8n, Flowise, and several other AI platforms do not enable authentication by default. Developers spin up instances following quickstart guides that emphasize getting the AI working, not securing it. The OWASP analysis found “a significant number of hosts had been deployed straight out of the box, with no authentication in place.” Because these platforms often require elevated permissions to interact with GPU resources, the default startup process typically uses root-level or high-privilege accounts — meaning an unauthenticated endpoint grants root-equivalent access. This is not a developer error — it is a platform design failure that shifts security responsibility onto users who are often AI engineers, not security engineers.

Pattern 2: Credentials Embedded in Infrastructure-as-Code. The scan found hardcoded and static credentials in setup examples and docker-compose files across multiple AI platforms. Docker Compose examples for Flowise, n8n, and similar tools frequently include API keys and database passwords as environment variables in plaintext. When these files are committed to public or semi-public repositories, they expose production credentials. The LiteLLM supply chain attack exploited this pattern: stage 2 of the malware specifically harvested .env files because environment variables are the canonical way AI frameworks receive API keys.

Pattern 3: Insecure Docker Configurations. AI services running in Docker containers often inherit overly permissive configurations: mounting the Docker socket (which grants container escape capabilities), running as root inside the container, disabling seccomp or AppArmor profiles, and exposing all ports to all interfaces. The scan found applications running as root as a common misconfiguration — a pattern that eliminates any privilege separation between the AI service and the underlying host.

What Enterprise Security Teams Should Do About AI Infrastructure

1. Run an Immediate Inventory of All AI Services in Your Environment

Before applying any configuration fix, you need to know what exists. Conduct an AI service discovery sweep: query your cloud provider’s service catalog for all compute instances tagged with AI-related labels; check your container orchestration platform (Kubernetes, ECS, Docker Swarm) for running images from common AI registries (Ollama, Flowise, n8n, LangChain-related images, Hugging Face Inference); review your CDN and reverse proxy access logs for traffic to ports 11434 (Ollama’s default), 3000 (n8n default), 3001 (Flowise default), and 8080 (common LLM API proxy port); and search your infrastructure-as-code repositories for ollama, flowise, n8n, litellm, langchain, and langflow references. This inventory becomes the input to every subsequent step — organizations that skip this step will spend months discovering shadow AI deployments reactively rather than proactively.

2. Enforce Authentication on Every AI Service Endpoint

Every AI service that is accessible from a network must require authentication. For Ollama, this means putting an authenticated reverse proxy (Nginx + basic auth, or OAuth2-Proxy with an identity provider) in front of the Ollama endpoint and binding Ollama to localhost (127.0.0.1) rather than all interfaces (0.0.0.0). For Flowise, enable the built-in authentication in the environment configuration (FLOWISE_USERNAME, FLOWRITE_PASSWORD) and, for production instances, add OAuth2 or SAML SSO. For n8n, enable the built-in basic auth or configure its enterprise SSO options. For any custom LLM API proxy or wrapper, implement API key authentication at minimum and rotate those keys quarterly. The OWASP finding that users “dropped into high-privilege accounts with full management access” on fresh installations means the default configuration should be treated as a broken configuration that security teams must remediate, not as a baseline to preserve.

3. Audit and Rotate All LLM API Keys in Your Environment

The LiteLLM supply chain attack specifically targeted LLM API keys stored in environment variables. The March 2026 supply chain wave affected 36% of cloud environments containing LiteLLM. Regardless of whether your environment was directly affected, this is the right moment to audit all LLM API credentials. For each AI service in your inventory from Step 1: identify the API keys used (OpenAI, Anthropic, Azure OpenAI, Google Gemini, Hugging Face tokens), rotate each key by generating a new one in the provider’s dashboard and updating your secret manager, revoke the old key, and confirm the service restarts successfully with the new key. Implement key rotation as a scheduled quarterly process, not a one-time event. Store all LLM API keys in your organization’s secrets manager (AWS Secrets Manager, HashiCorp Vault, Azure Key Vault), not in .env files, Docker Compose configurations, or application config files checked into version control.

The Bigger Picture: AI Infrastructure Joins the Attack Surface

The 2025-2026 period represents the first full cycle of AI infrastructure being present in enterprise environments at scale — and the security industry is discovering what the network security industry discovered with cloud infrastructure between 2012 and 2018: that rapid adoption creates an attack surface that security practice has not yet learned to govern.

The OWASP GenAI Q1 2026 report documented eight major AI security incidents in the January-April 2026 period, including approximately 150 GB of Mexican government data exposed through an AI agent misconfiguration, 513,000 lines of Anthropic source maps leaked via a misconfigured CDN, and multiple cloud agent platforms compromised through privilege escalation. The attack categories are novel — agentic autonomy failures, prompt injection in RAG pipelines, supply chain attacks on AI libraries — but the root causes are familiar: authentication not enforced, credentials stored insecurely, network exposure wider than intended.

For CISOs, the implication is clear: AI infrastructure must be incorporated into the standard vulnerability management lifecycle. That means quarterly audits of AI service authentication configurations, inclusion of AI platforms in penetration testing scope, mandatory secrets management for all LLM API keys, and a patch SLA for AI framework updates that matches the urgency applied to the LiteLLM incident — where 36% of cloud environments were potentially compromised through a library that 3.4 million daily downloads would not expect to be weaponized.

Frequently Asked Questions

Sources & Further Reading

• We Scanned 1 Million Exposed AI Services — The Hacker News
• The LiteLLM Supply Chain Attack — HeroDevs
• OWASP GenAI Exploit Round-Up Report Q1 2026
• Supply Chain Attacks Surge in March 2026 — Zscaler
• 2026: Year of AI-Assisted Attacks — The Hacker News