Verdict in one line: both frameworks installed in seconds and talked to a free local model with no API key. The 3B model on CPU then got the actual business tasks wrong. That says more about the model than about the frameworks, so we have not scored either tool. See “Why no score” below.
Setup (what you’d need to repeat this)
| Machine | Linux cloud box, 8 vCPU, about 3 GB free RAM, no GPU |
| Model runtime | Ollama, installed with the official script (curl -fsSL https://ollama.com/install.sh | sh); it needed zstd first |
| Model | Qwen2.5-3B-Instruct, Q4_K_M GGUF (about 2 GB) downloaded from Hugging Face, loaded with ollama create. (ollama pull was blocked by our box’s network policy, not by Ollama.) A 1.5B model was also tried for CrewAI. |
| API keys | None. Cost of the test runs: £0 (electricity/compute aside) |
| Python | 3.12 in a uv virtualenv per tool |
| Versions | crewai 1.15.28, browser-use 0.13.11 (from pip, 14 Oct 2026) |
Installs: uv pip install crewai and uv pip install browser-use each finished in about 20 seconds from a cold cache (2–3 s on re-run, log). Browser Use used the existing Google Chrome on the box via executable_path, so we skipped the Playwright download.
Test 1: CrewAI, UK credit-control task
Task (UK SMB): given 5 unpaid invoices (CSV) and today’s date (14 Oct 2026), work out 20% VAT and gross totals, list the overdue invoices, total the overdue gross, and draft a polite chaser to the customer with the oldest overdue invoice. One agent, one task (script).
Correct answer (computed in Python): INV-101, INV-102 and INV-105 are overdue, total gross overdue £3,150.60, and the oldest is INV-105 (Tyne Accountants).
| Check | Qwen2.5 3B (40.6 s) | Qwen2.5 1.5B (17.9 s) |
|---|---|---|
| VAT and gross for all 5 | ✅ all correct | ❌ 2 of 5 wrong (e.g. INV-103 gross £2,600, should be £2,760) |
| Overdue list | ❌ marked all 5 as overdue | ❌ marked all 5 as overdue |
| Total gross overdue | ❌ £4,386.60 (should be £3,150.60) | ❌ no total given |
| Chaser to oldest overdue | ⚠️ right customer (INV-105) but quoted net £975.50 not gross £1,170.60, and US date format | ❌ wrong customer (INV-101) |
What CrewAI did well: the Ollama route works with one line (LLM(model="ollama/...", base_url=...)), no API key, and the Agent/Task/Crew structure is quick to read. What went wrong: date comparison and summing, which is exactly where small models are weak. For money tasks we’d give the agent a calculator/code tool, or use a stronger model, rather than trust free-text arithmetic.
Test 2: Browser Use, read a GOV.UK page
Task (UK SMB): open https://www.gov.uk/vat-rates and report the standard and reduced VAT rates exactly as stated, then finish. Headless Chrome, max_steps=8, vision off (script).
Result: did not complete. 10 history entries over 256 s; final_result() returned None.
- Step 1 loaded the right page, and from then on the agent’s own “memory” said it had found 20% and 5%.
- It never called
done. Instead it looped: clicking the GOV.UK cookie banner buttons, re-navigating to the same URL, and trying to click a “VAT Rates” link that doesn’t exist. - Later steps failed with
23 validation errors for ActionModelUnion. The 3B model’s JSON didn’t match Browser Use’s action schema.
Full log: Browser Use run.
What Browser Use did well: it was easy to install, driving the existing Chrome worked first time, and the logs are very readable (each step shows Eval / Memory / Next goal), so it’s obvious why it failed. What went wrong: structured tool-calling needs a model that reliably emits valid JSON, and a 3B CPU model doesn’t.
Why no score
Our methodology scores the tool on task success, autonomy, setup, value, safety and integrations. In these runs, task success was decided by the model we could afford to run (a 3B model on CPU). Both projects document hosted frontier models as their normal route. Scoring CrewAI or Browser Use on these runs would mostly be scoring Qwen2.5 3B, so we would be publishing a number we don’t believe. Both tools stay Listed with no score.
Update (14 Oct 2026): we re-ran both tasks unchanged on a hosted Grok model and scored them: CrewAI and Browser Use with a hosted Grok model.
Takeaways for UK small businesses
- “Free and local” is real for setup and privacy. Nothing left the box except the page fetch. It isn’t enough for accurate money or browsing tasks on a laptop-class CPU.
- Whatever framework you use, never let an agent’s free-text arithmetic reach a customer. Check totals in code and keep a human approval before any chaser is sent.