diff --git a/docs/source/deployment/security.md b/docs/source/deployment/security.md index e2ef8196c167..9c4d639c0b3d 100644 --- a/docs/source/deployment/security.md +++ b/docs/source/deployment/security.md @@ -53,6 +53,45 @@ Key points from the PyTorch security guide: - Implement proper authentication and authorization for management interfaces - Follow the principle of least privilege for all system components +## Security and Firewalls: Protecting Exposed vLLM Systems + +While vLLM is designed to allow unsafe network services to be isolated to +private networks, there are components—such as dependencies and underlying +frameworks—that may open insecure services listening on all network interfaces, +sometimes outside of vLLM's direct control. + +A major concern is the use of `torch.distributed`, which vLLM leverages for +distributed communication, including when using vLLM on a single host. When vLLM +uses TCP initialization (see [PyTorch TCP Initialization +documentation](https://docs.pytorch.org/docs/stable/distributed.html#tcp-initialization)), +PyTorch creates a `TCPStore` that, by default, listens on all network +interfaces. This means that unless additional protections are put in place, +these services may be accessible to any host that can reach your machine via any +network interface. + +**From a PyTorch perspective, any use of `torch.distributed` should be +considered insecure by default.** This is a known and intentional behavior from +the PyTorch team. + +### Firewall Configuration Guidance + +The best way to protect your vLLM system is to carefully configure a firewall to +expose only the minimum network surface area necessary. In most cases, this +means: + +- **Block all incoming connections except to the TCP port the API server is +listening on.** + +- Ensure that ports used for internal communication (such as those for +`torch.distributed` and KV cache transfer) are only accessible from trusted +hosts or networks. + +- Never expose these internal ports to the public internet or untrusted +networks. + +Consult your operating system or application platform documentation for specific +firewall configuration instructions. + ## Reporting Security Vulnerabilities If you believe you have found a security vulnerability in vLLM, please report it following the project's security policy. For more information on how to report security issues and the project's security policy, please see the [vLLM Security Policy](https://github.com/vllm-project/vllm/blob/main/SECURITY.md).