# Running UiPath on-premises with a local LLM: architecture, GPU sizing and model choice

> UiPath Automation Suite with a local LLM: OpenShift, vLLM, GPU memory sizing, model choice and server tiers. See how many GPUs your process needs.

- Source: https://snok.ai/en/news/blog/uipath-on-premise-local-llm-gpu-sizing/
- Author: Jacek Bugajski
- Published: 2026-10-15

---
UiPath on-premises can work with a local LLM running on GPUs in your own data centre. Agents arrived in UiPath Automation Suite with release 2.2510.2 on 15 April 2026, and one of the models UiPath recommends for them in a self-hosted installation is gpt-oss-120b: an OpenAI model under Apache 2.0 that sends no requests to OpenAI. Models outside that list can be attached too: the same connector accepts Qwen, Mistral, Bielik or PLLuM, provided the model passes UiPath's compatibility check.

Organisations that keep their data away from closed models run by US providers such as OpenAI, Anthropic or Google could, until recently, run UiPath agents only against a cloud model or not at all. A reference on-premises environment has five layers, and the number of GPUs comes from a memory calculation for the chosen model.

## Why the language model moves into your own data centre

**US legal reach.** The CLOUD Act (18 U.S.C. § 2713) requires a US provider to preserve or disclose data on request from US authorities regardless of whether that data is stored inside or outside the United States. A European region in the contract does not change this.

**An unsettled legal basis for transfers.** The European Commission's decision on the EU-US Data Privacy Framework stands, but the appeal against the judgment that upheld it is pending before the Court of Justice of the EU (case C-703/25 P).

**Segregated networks.** Public administration, defence and parts of critical infrastructure work in environments with no internet connection. A cloud model is not an option there.

**Cost at high volume.** Hardware you own can absorb growing request volumes up to its capacity, with no charge per token. Whether that pays off is a separate calculation for each process.

These are risks, and the legal assessment of any specific processing is for your lawyers. A local model means prompts and responses do not reach an external LLM provider. Whether the whole environment stays local, including the UiPath platform's own licensing traffic, we check component by component.

## The platform constraint: UiPath agents with a local LLM need OpenShift

Automation Suite comes in three deployment flavours: Automation Suite on Linux (UiPath's installer on Red Hat Enterprise Linux), managed Kubernetes in the cloud (EKS, AKS), and OpenShift. The 2.2510 documentation is explicit that LLM integration is supported on AKS, EKS and OpenShift. Automation Suite on Linux has no Agent Builder, no agents, no LLM Gateway, no Context Grounding, no GenAI Activities, no Healing Agent and no UiPath Maestro.

**Agents with a local model in your own data centre therefore need Automation Suite on OpenShift**, since EKS and AKS are public cloud. The Linux flavour covers classic automation, Document Understanding, AI Center and Action Center. We settle this before ordering hardware, because OpenShift brings its own licensing and its own operating model.

Bring-your-own-model support is switched on by the `enableByoAiGateway` parameter in `input.json`, which belongs in the installation design. We also read the release notes before installing. In 2.2510.3 UiPath lists an open authorisation error that stops agents from opening, and on 17 September 2026 it corrected an earlier note to say the fix is not in 2.2510.4 either. We choose the version to install from the known-issues list.

## UiPath on-premises architecture with a local LLM

![UiPath Automation Suite architecture with a local LLM in the client's data centre](https://snok.ai/images/blog/uipath-on-premise-llm-architecture-en.webp)

**The UiPath platform.** A high-availability Automation Suite deployment needs at least three machines. The minimum for the Complete profile on the Linux flavour of 2.2510 is 96 vCPUs and 192 GB of RAM in total, plus, on each server node, a 256 GB cluster disk (SSD, at least 1,100 IOPS), a 512 GB data disk and a 16 GB etcd disk. For OpenShift we derive the figures from the Automation Suite Install Sizing Calculator and capacity planning for your volumes; UiPath stresses that minimums do not guarantee production behaviour.

**Supporting services.** SQL Server 2019 or later (Insights needs 2019 or 2022), an external object store, which UiPath recommends over the in-cluster one (an S3-compatible store is one of the options), and a layer 4 load balancer with TLS pass-through.

**The inference layer.** Separate GPU servers running an engine that exposes an OpenAI V1-compatible API. UiPath names vLLM and SGLang as supported. Ollama and LM Studio work well in the lab, but Ollama does not support the `tool_choice` parameter, and Batch Transform in Context Grounding requires the model to accept a forced tool call. vLLM has handled that since version 0.8.3.

**AI Trust Layer and LLM Gateway.** Every model call passes through UiPath's governance layer and lands in the audit log. You attach your own model through the "OpenAI V1 Compliant LLM" connector in Integration Service, and it has to pass a probe test before the configuration can be saved. An Automation Ops policy that disables a UiPath-provided model does not automatically cover your own model of the same type, so its rules are configured separately and checked in our [AI Security](https://snok.ai/en/offer/ai-automation/ai-security/) reviews.

**Nodes for documents.** Document Understanding uses a GPU node of its own: 8 vCPUs with AVX2, 52 GB of RAM and a card with at least 11 GB of memory. Modern DU projects carry a separate minimum of 18 vCPUs and five GPUs (or two with partial CPU inference), with the NVIDIA T4 as the lowest recommended card. We keep these cards separate from the language model; the vendor does not require it, it is our design choice.

## What works with a local model and what stays in the cloud

![UiPath features available with a local model and features that need the cloud](https://snok.ai/images/blog/uipath-on-premise-llm-local-vs-cloud-en.webp)

According to UiPath's matrix for self-hosted models, a local model covers agent execution and evaluations, GenAI Activities, Healing Agent, and embeddings plus Batch Transform in Context Grounding. Without a cloud provider you lose DeepRAG and Advanced Extraction, Autopilot, agent simulations, Coded Agents that call a model, ScreenPlay and semantic selectors. IXP for unstructured documents is available only in Automation Cloud.

An agent with tools that reads documents, checks data in SAP and prepares a decision for a person runs entirely on the local model. Features on the second list are either replaced with your own components or left out of a segregated environment.

## Which models work with UiPath on-premises

UiPath distinguishes recommended models, whose compatibility it has tested, from any other model with an OpenAI V1-compatible API, which attaches through the same connector.

**Models recommended by UiPath (2.2510).** For agents and GenAI Activities: GLM 4.5, GLM 4.6, GLM 4.6 V and gpt-oss-120b. For Healing Agent: GLM 4.5, GLM 4.6 and Qwen3 235B.

**Other open models.** UiPath's guide to connecting open-source models to AI Trust Layer states that any OpenAI-compatible gateway works and contains no list of approved models. There are two conditions: the model must pass the probe, and it must meet the requirements of the feature it serves, such as forced tool calling for Batch Transform or dependable function calling for an agent. For a model outside the list, the delivery team confirms compatibility by testing.

**Qwen.** Qwen3 235B is on UiPath's list for Healing Agent. The newer Qwen3.5 models, from 27 billion parameters upwards, are Apache 2.0 and, according to the model card, cover more than 200 languages and dialects, which makes them candidates to test as an agent engine.

**Mistral.** Mistral Small 4 is Apache 2.0 and its card declares native function calling, although the card's language list does not include Polish. Mistral Medium 3.5 uses a modified MIT licence with carve-outs for high-revenue companies.

**Bielik.** Bielik 11B v3.0 from SpeakLeash is tuned for Polish and released under Apache 2.0. Its card does not declare native function calling, but in our own test Bielik produced correct tool calls through Chat Completions, and the `bielik-tools` repository provides a tool-call parser for vLLM.

**PLLuM.** The PLLuM 2512 family includes 4B and 12B models under Apache 2.0 (PLLuM-12B is built on Mistral Nemo) and Llama-PLLuM 8B and 70B under the Llama 3.1 licence. According to its authors, PLLuM is particularly strong on public administration and legal text. The card says nothing about function calling, so compatibility with UiPath agents has to be tested. We evaluated PLLuM at proof-of-concept stage in a public sector project.

For Polish organisations we propose a model from UiPath's list as the agent engine, with Bielik or PLLuM as the task model for Polish text, called from GenAI Activities or an API. A lawyer reads each licence before a model reaches production. The Llama 4 acceptable use policy, for instance, grants no rights to the multimodal Llama 4 models to individuals or companies domiciled in the EU, although the restriction does not apply to end users of a finished product.

## GPU sizing: weights, KV cache and concurrent sessions

A card's memory goes on three things.

**Model weights.** Parameter count times bytes per parameter: 2 bytes in BF16, 1 in FP8, roughly half a byte with 4-bit quantisation. Llama 3.3 70B takes about 141 GB in BF16, 71 GB in FP8 and 35 GB in INT4. Mixture-of-experts models must hold the weights of every expert in memory: Qwen3-30B-A3B activates about 3 billion parameters per token yet occupies 61 GB in BF16. Speed follows the active parameters; memory follows all of them.

**KV cache.** The KV cache is the store of attention keys and values that the model keeps for every token in every session's context. Per token it takes `2 × layers × KV heads × head dimension × bytes`. For Llama 3.3 70B that is 320 KiB per token in BF16, or about 10.5 GB for one session with a 32,000-token context. Sixteen such sessions need about 167.8 GB, more than twice the model's FP8 weights. So we size hardware for weight memory, context length and concurrent sessions, and for agents with long context the last two decide the configuration.

**Headroom.** Activations, CUDA graphs and memory fragmentation. Our calculations reserve 15 per cent of card capacity, a working assumption we confirm by measuring on the target hardware.

The model's architecture matters as well: gpt-oss-120b has 36 layers, but only 18 attend over the full context; the other 18 use a 128-token sliding window. Its KV cache therefore grows half as fast as the layer count suggests: about 4.72 GB per session at 128,000 tokens in BF16. The weights in native MXFP4 quantisation take about 65 GB (the model checkpoint on Hugging Face); OpenAI states that the model fits on a single 80 GB card.

![Concurrent gpt-oss-120b sessions on different GPU configurations](https://snok.ai/images/blog/uipath-on-premise-llm-gpu-sessions-en.webp)

Results for gpt-oss-120b with BF16 KV cache and 15 per cent headroom (SNOK calculation, not a benchmark). With two cards, a single model instance is split across them (tensor parallel).

| Configuration | GPU memory | Sessions at 32k | 128k |
|---|---|---|---|
| 1 × RTX PRO 6000 | 96 GB | 13 | 3 |
| 1 × H200 | 141 GB | 46 | 11 |
| 2 × RTX PRO 6000 | 192 GB | 82 | 20 |
| 2 × H200 | 282 GB | 147 | 36 |

An FP8 KV cache halves context memory, roughly doubling the session count, and we test the effect on quality against a sample of your own tasks.

The table shows memory ceilings. Tokens per second for a given number of users depends on prompt length, prefix sharing and vLLM settings, and we benchmark it on the target configuration.

A discrepancy in the documentation needs resolving before any order. The table in UiPath's documentation gives eight 80 GB cards as the minimum for models of 120B and above, while OpenAI states that gpt-oss-120b fits on a single 80 GB card. UiPath's table does not distinguish configurations by quantisation, and the reason for the gap needs confirming with the vendor. We therefore treat one- and two-card designs as proposals to be confirmed with UiPath before any purchase order.

## GPU servers for local LLMs in three tiers

![Three hardware tiers for UiPath with a local LLM: lab, production start and scale](https://snok.ai/images/blog/uipath-on-premise-llm-hardware-tiers-en.webp)

**Lab and proof of concept.** A workstation built on the NVIDIA GB10 chip (DGX Spark and its equivalents): 128 GB of unified memory and a 240 W power supply, which NVIDIA rates for inference on models of up to 200 billion parameters. The limit is memory bandwidth, 273 GB/s against 4.8 TB/s on an H200. In our lab gpt-oss-120b generates 56-60 tokens per second for a single user on GB10 (llama.cpp). It lets you evaluate the model, its prompts and tool calling; whether it can serve many concurrent users has to be measured for the chosen model and load. [We tested a GB10 workstation in our lab](https://snok.ai/en/news/blog/lenovo-thinkstation-pgx-gb10-hands-on/).

**Mac Studio.** Apple is moving steadily towards local model inference. The Mac Studio with M5 Ultra takes up to 512 GB of unified memory at 1.2 TB/s. Configurations up to 256 GB have shipped since 22 September 2026, and Apple has announced the 512 GB version for the end of October. Apple says four of them linked over Thunderbolt 5 with RDMA run inference up to three times faster than one. 256 GB holds the weights of gpt-oss-120b (about 65 GB) or Qwen3 235B in 4-bit quantisation (about 118 GB) with room for context. On a Mac we serve models through MLX or llama.cpp with an OpenAI-compatible API, while UiPath lists vLLM and SGLang as supported engines, so we treat the Mac Studio as a lab or a single-department environment, always subject to the probe test. In our view this is Apple's first step towards server-side AI; we compared the machine with GB10 in [a Mac Studio as a PoC machine](https://snok.ai/en/news/blog/mac-studio-512gb-vs-dgx-spark-poc-machine/).

**Production start.** One server with two cards: RTX PRO 6000 Blackwell (96 GB, up to 600 W each) or H200 NVL (141 GB, up to 600 W each), alongside an Automation Suite cluster on CPU servers. By the calculation above, this tier holds gpt-oss-120b for 82 (RTX PRO 6000) or 147 (H200) concurrent sessions at a 32,000-token context.

**Scale and high availability.** Servers with four to eight RTX PRO 6000, H200 or B200 cards. At this tier we plan a second inference node behind a load balancer, a second model (for instance Bielik or PLLuM for Polish text) and room to grow. With eight cards at 600-700 W each, power and cooling are agreed with the data centre team before the order.

## Lighter GPU load with decision models

Not every step an agent takes needs a model that generates text. Classifying a ticket, choosing a process path and checking an agent's action are decisions with a closed set of answers. Decision models such as TypeSafe's Jev return a typed decision with a calibrated confidence score instead of prose.

We measured Jev on 26 September 2026 on four tasks modelled on UiPath projects, using synthetic data. In email classification it reached 96 per cent accuracy against 90-92 per cent for an agent on Claude Sonnet 4.5, with a median of 0.42 s instead of about 5 s. Jev does not do arithmetic: it approved inflated invoices until code calculated the amounts for it. Jev runs as an external cloud service hosted by its vendor, so in an on-premises environment open equivalents take its place. We have run basal-1.0 locally on a Mac; it is a Polish decision model built on Bielik v3.0 in 4.5 and 1.5 billion parameter versions, released under Apache 2.0.

In an on-premises design, a decision model of a few billion parameters handles routing and action checks, and the large language model receives only the tasks that need generated text. Traffic to the large model falls, so the same number of cards can serve more processes. The measurement is written up in [Jev in UiPath](https://snok.ai/en/news/blog/jev-uipath-maestro-trust-layer-delegate/).

## Order of work

We run these projects from the task to the hardware:

1. Describe the process and pick two or three tasks for an agent.
2. Test candidate models in the lab on a sample of real documents with real tools.
3. Calculate memory for the measured context length and number of concurrent sessions.
4. Specify the servers and confirm the configuration with UiPath.
5. Install Automation Suite with bring-your-own-model enabled, attach the model through AI Trust Layer and set governance policies.

Agents with human approval points are designed in [UiPath Maestro](https://snok.ai/en/offer/ai-automation/uipath-maestro/), following the approach we set out in our piece on [HITL gates in AI Trust Layer](https://snok.ai/en/news/blog/hitl-gates-uipath-maestro-ai-trust-layer/).

## Frequently asked questions

**Which models does UiPath support on-premises?** For agents in Automation Suite 2.2510, UiPath recommends GLM 4.5, GLM 4.6, GLM 4.6 V and gpt-oss-120b; for Healing Agent, GLM 4.5, GLM 4.6 and Qwen3 235B. Other models with an OpenAI V1-compatible API can be attached if they pass the AI Trust Layer probe and meet the requirements of the feature in question.

**Does UiPath work with Bielik or PLLuM?** Technically yes, through the OpenAI V1 connector and a vLLM engine. Neither is on UiPath's recommended list, so compatibility, tool calling in particular, has to be confirmed by testing on your own tasks.

**How many GPUs does gpt-oss-120b need?** The model fits on a single 80 GB card. The number of cards depends on concurrent sessions and context length: at 32,000 tokens one H200 holds about 46 sessions and two hold about 147 (a memory calculation, not a benchmark). UiPath's documentation lists eight 80 GB cards as the minimum for 120B+ models, a discrepancy we resolve with the vendor before ordering.

**Is Automation Suite on Linux enough for agents with a local model?** No. The Linux flavour has no agents and no LLM Gateway. UiPath supports LLM integration on OpenShift, EKS and AKS.

## What we have already delivered

In the public sector we deployed UiPath Automation Suite on-premises in a network separated from public networks, integrated the platform with a local language model, and built case-registration automation on it with Document Understanding. In central government we installed and tuned UiPath in a closed on-premises environment. We benchmark local models on our own GB10 hardware, and our Automation Suite installations are run by engineers certified as UiPath Infrastructure Engineers. We do not name these clients because the projects are confidential.

SNOK is a [UiPath Platinum partner](https://snok.ai/en/uipath-snok/) and a [Lenovo Platinum partner](https://snok.ai/en/lenovo-snok/), so we specify the platform, the model and the servers in a single project. If you are planning UiPath agents without passing data to an external model, start with the numbers for your own process. The [on-premises LLM](https://snok.ai/en/offer/ai-automation/llm-on-premise/) service page sets out the scope, and you can arrange a conversation through our [contact form](https://snok.ai/en/contact/).

## Sources

- [UiPath - About LLM integration, Automation Suite 2.2510](https://docs.uipath.com/automation-suite/automation-suite/2.2510/admin-guide/about-llm-integration) (accessed 14.10.2026)
- [UiPath - Connecting OSS models to AI Trust Layer](https://docs.uipath.com/automation-suite/automation-suite/2.2510/admin-guide/connecting-oss-models-aitl) (accessed 14.10.2026)
- [UiPath - Cross-deployment model feature comparison, 2.2510](https://docs.uipath.com/automation-suite/automation-suite/2.2510/installation-guide/cross-deployment-model-feature-comparison) (accessed 14.10.2026)
- [UiPath - Hardware and software requirements, 2.2510](https://docs.uipath.com/automation-suite/automation-suite/2.2510/installation-guide/hardware-and-software-requirements) (accessed 14.10.2026)
- [UiPath - Context Grounding LLM requirements](https://docs.uipath.com/automation-suite/automation-suite/2.2510/admin-guide/context-grounding-llm-requirements) (accessed 14.10.2026)
- [UiPath - Kubernetes cluster and nodes, OpenShift](https://docs.uipath.com/automation-suite/automation-suite/2.2510/installation-guide-openshift/kubernetes-cluster-and-nodes) (accessed 14.10.2026)
- [UiPath - Agents release notes 2.2510.2](https://docs.uipath.com/agents/automation-suite/2.2510/release-notes/2-2510-2) (accessed 14.10.2026)
- [vLLM - Tool calling](https://github.com/vllm-project/vllm/blob/main/docs/features/tool_calling.md) and [Ollama - OpenAI compatibility](https://docs.ollama.com/api/openai-compatibility) (accessed 14.10.2026)
- [NVIDIA - Mastering LLM Techniques: Inference Optimization](https://developer.nvidia.com/blog/mastering-llm-techniques-inference-optimization/) (17.11.2023)
- Model cards on Hugging Face: [gpt-oss-120b](https://huggingface.co/openai/gpt-oss-120b) (with [config.json](https://huggingface.co/openai/gpt-oss-120b/blob/main/config.json)), [Qwen3-30B-A3B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-30B-A3B-Instruct-2507), [Qwen3.5-35B-A3B](https://huggingface.co/Qwen/Qwen3.5-35B-A3B), [Bielik 11B v3.0](https://huggingface.co/speakleash/Bielik-11B-v3.0-Instruct), [PLLuM-12B-instruct-2512](https://huggingface.co/CYFRAGOVPL/PLLuM-12B-instruct-2512) (accessed 14.10.2026)
- [Meta - Llama 4 Acceptable Use Policy](https://dev.meta.ai/llama/llama4/use-policy/) (accessed 14.10.2026)
- NVIDIA specifications: [H200](https://www.nvidia.com/en-us/data-center/h200/), [RTX PRO 6000 Blackwell Server Edition](https://www.nvidia.com/en-us/data-center/rtx-pro-6000-blackwell-server-edition/), [DGX Spark](https://www.nvidia.com/en-us/products/workstations/dgx-spark/) (accessed 14.10.2026)
- [Apple Newsroom - Mac Studio with M5 Ultra](https://www.apple.com/newsroom/) (25.08.2026)
- [TypeSafe - Jev](https://typesafe.ai/) and [basal-1.0](https://github.com/rkinas/basal) (accessed 14.10.2026)
- [CLOUD Act, 18 U.S.C. § 2713](https://www.law.cornell.edu/uscode/text/18/2713) and [appeal C-703/25 P, OJ C/2025/6610](https://eur-lex.europa.eu/eli/C/2025/6610/oj)

Memory calculations were carried out on 14.10.2026 using parameters from each model's `config.json`, for contexts of 32,000 and 128,000 tokens.