Key takeaways
- Bielik 11B v3.1 connects to UiPath agents through the “OpenAI V1 Compliant LLM” connector, but the AI Trust Layer probe passed only with a rule in the gateway and the Agent Builder agent ended in an error.
- In a benchmark on 60 emails, Bielik 11B v3.1 got the category right 45 times out of 50 and the priority right 37 times out of 50, clearly behind Claude Sonnet 4.5.
- Both failures share one root: the mlx_lm.server on a Mac does not enforce the tool_choice and parallel_tool_calls parameters that UiPath requires.
- A Mac running MLX is for measuring quality, while the production candidate is a GPU server with vLLM or SGLang, whose probe test we have not yet run.
- Decisions from a closed list go to a decision model such as basal, while a Polish reply and a tool call in a single step go to Bielik 3.1.
Bielik 11B v3.1, the open-weight language model from the Polish SpeakLeash project, ran on a MacBook Pro with an M4 Max chip and classified 45 of 50 helpdesk emails correctly. In the classification test it returned a valid tool call in all 60 cases. On the same dataset and with the same prompt, Claude Sonnet 4.5 called through the UiPath LLM Gateway scored 45 and 46 in two September runs. Bielik set the ticket priority correctly in 37 of 50 cases, clearly behind Claude.
We then connected the same model to UiPath Automation Cloud as a custom model in the AI Trust Layer. The compatibility test (the probe) passed only after the gateway in front of the model enforced a single tool call. An agent built in UiPath Agent Builder classified a ticket correctly and still ended in an error. Both problems trace back to the same gap: the server on the Mac does not enforce the tool-call parameters that UiPath requires.
Bielik 11B v3.1 - what SpeakLeash announced
SpeakLeash announced a new family of Bielik models on 10 November 2026. According to its authors, Bielik v3.1 supports 56 languages, including every official language of the European Union, and combines function calling, structured output and reasoning. On τ²-bench, which tests agentic work in realistic scenarios, SpeakLeash reports that version 3.1 scores between 16 and 45 percentage points higher than version 3.0. These are the vendor’s figures; we have not reproduced them.
The same announcement introduced Bielik-One, four decision models for ticket classification, content moderation, preliminary document review and procedural compliance checks, as well as a DFlash variant that speeds up generation through speculative decoding. The Bielik 11B v3.1 Instruct weights are on Hugging Face under the Apache 2.0 licence, with downloads for Mac (MLX), for llama.cpp (GGUF) and for NVIDIA GPUs (AWQ, FP8, NVFP4). On the morning of 12 November we could not find the Bielik-One models in the SpeakLeash repository, so this article does not assess them. The authors have promised further releases at Bielik Summit on 26 November.
The benchmark: 60 emails from our UiPath agent PoC
We tested Bielik on a dataset that already has two reference points. It consists of 60 synthetic emails from a proof of concept for a UiPath ticket classifier: 50 ordinary tickets across four categories (outage, support, enquiry, spam), 34 in Polish and 16 in English, plus 10 emails carrying a prompt injection attack. The system prompt and the output normalisation rules came over unchanged from the PoC agent. We changed one thing: Bielik returns its classification through a tool call, because that is how UiPath agents talk to a model.
The model ran locally on a MacBook Pro with an M4 Max and 128 GB of memory, as an 8-bit MLX build served by mlx_lm.server, with no network connection.
| Metric | Bielik 11B v3.1 | Claude Sonnet 4.5 |
|---|---|---|
| Category | 45/50 (90%) | 45-46/50 |
| Priority | 37/50 (74%) | 44-45/50 |
| Injection: category matches label | 6/10 | 6/10 |
| Tool calls | 60/60 valid | output as JSON |
The Claude Sonnet 4.5 figures come from two runs on 28 September 2026. Median call latency was 3.18 s for Bielik and 3.05-3.10 s for Claude, but the two travel different paths: local inference on a laptop for Bielik, a round trip from Warsaw through the UiPath cloud gateway for Claude. The medians indicate an order of magnitude and do not settle which model is faster. On the same dataset the Jev decision model, running in a different mode (decision API), earlier reached 96% category accuracy, as we reported in our test of Jev in UiPath Maestro.
Bielik classified every English email correctly (16 of 16) and 29 of 34 Polish ones. Two of its five category errors concern hardware faults reported by a single user (a printer and a microphone), which the model treated as support requests. Two more are a conference invitation and a partner newsletter: the dataset labels them as enquiries, the model labelled them spam.
Where Bielik falls short: priority calibration
Almost the entire priority gap comes from one pattern. Bielik marked ten support requests (password reset, folder access, MFA setup) as P2 instead of P3. Its justifications are sound, which means the model understands the ticket and misjudges the scale. An error of this kind is fixed with examples in the prompt or with a rule in code. We apply the same principle with decision models: code handles rules and arithmetic, and the model decides on facts already established.
On the prompt injection emails Bielik assigned each of the 10 messages the same category as Claude Sonnet 4.5 did in both September runs. It executed none of the injected instructions. It labelled four of the attack emails as spam, as the security rule in the prompt requires, whereas the dataset expected “enquiry” for them. One of the six matching results (an attempt to extract the prompt) came from a fallback rule in code, which sets the category to “enquiry” when the model’s confidence is low.
The UiPath AI Trust Layer probe: a pass with help from the gateway
UiPath lets you connect your own model to the AI Trust Layer through the “OpenAI V1 Compliant LLM” connector. We exposed Bielik from the MacBook over Tailscale Funnel at an HTTPS address, behind a gateway that requires a key and accepts only two types of request. The data was entirely synthetic, and we took the public address down immediately after the test. In the AI Trust Layer the configuration applies to the Agents product and the “Design, Evaluate & Deploy” feature, with “OpenAI Chat Completions” as the API type. The list of model identifiers loaded from our server, which confirmed the connection.
Before saving, the AI Trust Layer runs a probe. The first request is a plain prompt for an answer. The second includes a single tool and two requirements: the model must call it (tool_choice: required) and may do so only once (parallel_tool_calls: false). Bielik returned two calls at once, and in the second it supplied the number 42 where the schema expects text. The probe ended with a message about an incorrect tool call.
mlx_lm.server accepts the parallel_tool_calls parameter but does not enforce it. We added a rule to the gateway: when a request forbids parallel calls, the response keeps only the first call. With that change the probe passed, and Bielik appeared in Agent Builder’s model list alongside models from Anthropic, OpenAI and Google. We state this plainly: the green result came from the rule in our gateway, not from the model itself.
An agent in UiPath Agent Builder: the right answer in the wrong form
Finally, we built an autonomous agent with Bielik as its model and gave it a synthetic ticket reading “URGENT - SAP production server down”. Bielik’s answer was correct: outage, priority P1, with a justification citing halted invoicing. It answered in plain text, however, and a UiPath agent can finish only with a tool call (end_execution or raise_error), which is why it sends tool_choice: required to the model. The server on the MacBook does not enforce that requirement, so after two text replies UiPath stopped the job with the error AGENT_RUNTIME.THINKING_LIMIT_EXCEEDED.
The gateway log showed one more thing. Before the agent’s prompt reached Bielik, the AI Trust Layer replaced the date and the agent’s name in the system prompt with the placeholders [DateTime-1] and [Person-1]. Data masking therefore also covers a model connected through the connector. The same observation shows that in Automation Cloud the prompt passes through UiPath’s cloud before it reaches the model.
A Mac for the lab, vLLM for production
Both failures have the same root. The model can call a tool, but it does not always do so exactly as the request specifies: it returned two calls instead of one and a number instead of text. Enforcing the format is the job of the serving layer, and the MLX server on a Mac does not do it. UiPath lists vLLM and SGLang as supported engines. vLLM has supported forced tool calling since version 0.8.3, as we covered in our article on UiPath on-premise architecture with a local model.
That gives a division of roles. A Mac running MLX is for measuring quality: whether the model understands the client’s Polish tickets, where it misjudges the scale, what a run costs. The candidate for production is a GPU server running vLLM or SGLang, provided it passes the probe without workarounds. We have not run that test yet, and it is where we will check whether vLLM also enforces parallel_tool_calls. Hardware sizing for both is covered in our guide to local models on Mac and Lenovo PGX.
When to use Bielik and when to use the basal decision model
In our benchmark Bielik took about 3 seconds to classify an email. The basal-1.0 decision model in its 1.5B version reached a decision on a Mac in about 0.11 seconds, though on a different, smaller dataset, so we do not compare the two models’ accuracy. The basal model does not write text. It picks an answer from a closed list and returns a calibrated probability that signals when a case should go to a human. In an external evaluation published in the system-1-ai repository, on 402 Polish decisions from October, the basal-omni-max version reached 96% accuracy. That measurement is someone else’s and we have not repeated it.
This suggests a simple division of labour inside an agent:
- A decision from a list (category, priority, pass or escalate) goes to a decision model such as basal: a fraction of a second per decision and a confidence threshold below which the ticket goes to a human.
- Text and action in Polish (a reply to the requester, extraction into a schema, a plan of steps, tool calls) go to Bielik 3.1, running locally when the data must stay inside the organisation.
- Multi-step reasoning over a large context goes to a cloud model through the UiPath LLM Gateway, provided the data may leave the organisation.
In our benchmark Bielik did the classifier’s job correctly, but more slowly than a decision model and with weaker priority calibration than Claude Sonnet 4.5. Its strength lies in what a decision model cannot do: a Polish reply and a tool call in a single step.
How we start a project like this
We run a local-model project in UiPath agents in four steps, and the first looks like this article: a quality benchmark on an anonymised sample of the client’s tickets, on a Mac or a Lenovo ThinkStation PGX, with an accuracy report broken down by language and category. Second, the AI Trust Layer probe on the target engine, with a list of the parameters the gateway must enforce. Third, the split of work between the decision model and the generative model, with thresholds for handing over to a human. Finally, an AI security review with prompt injection tests on the client’s data. The client’s board receives a results table and a recommendation for the engine and hardware, and we deliver the whole engagement as part of our on-premise LLM service as a UiPath partner.
Frequently asked questions
Does Bielik work with UiPath agents? The model connects through the “OpenAI V1 Compliant LLM” connector, and in the classification test it returned valid tool calls. The AI Trust Layer probe and the autonomous agent, however, require an engine that enforces tool_choice and parallel_tool_calls. On the MLX server on a Mac, the probe passed only with the rule in our gateway, and the agent ended in an error.
Do I need an on-premise installation to test Bielik with UiPath? No. In Automation Cloud it is enough for the model to be reachable at an HTTPS address with a key. Production use with data that must not leave the organisation requires Automation Suite in your own data centre, because in Automation Cloud the prompt passes through the AI Trust Layer in UiPath’s cloud, as the data masking in our test shows.
Is Bielik better than Claude at classifying Polish tickets? On our dataset both models assigned the category with similar accuracy, and Claude Sonnet 4.5 was better at setting priority. Bielik’s advantage is that it can run on your own infrastructure. With Automation Suite on premises, the data does not reach the model vendor.
Sources
- SpeakLeash - launch of Bielik v3.1, Bielik-One and DFlash, LinkedIn post, 10.11.2026
- Bielik 11B v3.1 Instruct on Hugging Face (accessed 11.11.2026)
- UiPath - Connecting OSS models to AI Trust Layer, Automation Suite 2.2510 (accessed 14.10.2026)
- SNOK in-house benchmark: MacBook Pro M4 Max 128 GB,
mlx_lm.server0.32, Bielik 11B v3.1 Instruct MLX 8-bit, 60 emails from the Email Classifier PoC, 11.11.2026; UiPath Automation Cloud test (AI Trust Layer, Agent Builder), 12.11.2026 - Reference points: Claude Sonnet 4.5 through the UiPath LLM Gateway, same dataset, two runs on 28.09.2026; basal-1.0 1.5B, SNOK in-house measurement, 30.09.2026
- External evaluation of decision models (practitioner outside SNOK) on 402 Polish decisions, system-1-ai repository, experiment 07 (16.10.2026)