A few days back, you might remember that we ran a different head-to-head experiment: we built the same integration on Boomi and Celigo from a terminal, through each vendor’s own headless tooling for coding agents. That effort was about comparing the headless building experiences outside of both platforms.
This one is more relevant to users who prefer to use the UI instead: when you ask the AI inside the product to do a job, does the job actually get done? Or do you end up manually working through what the AI created, which means more work and more frustration?
As you know, building the integration is only part of the job; ongoing management is equally important and sometimes even more cumbersome. So we picked one small integration and followed it across the whole lifecycle to understand the full breadth of both platforms’ in-app AI capabilities: building, testing, updating, troubleshooting, and even adding an agentic step. We prompted the in-app AI experiences the way a typical business user would.
The jobs
The integration we asked each AI to build moves shopping carts from a public test API to a webhook endpoint. It’s a stand-in for one of the most common integration patterns: pull records from one system, filter and reshape them, and send them to another on a schedule.
The five jobs:
| Job | What we asked | How we checked |
|---|---|---|
| 1 · Build | An hourly integration that reads all carts (the source pages its results), keeps carts over 30,000, maps four fields, and posts each one to the endpoint. | 41 records should arrive. |
| 2 · Test | Run it once and show us what was sent. Also: how would we test a NetSuite flow without credentials? | 41 records at the endpoint, payloads visible. |
| 3 · Update | Change the filter to carts with at least three distinct products, and add a field. | 22 records. |
| 4 · Troubleshoot | The integration is failing with a 404. Find the root cause and fix it. (We had changed the path to /cartz.) |
Root cause named, fix applied, re-run reads 30 carts. |
| 5 · Agentic flow | Read 10 carts, have an AI agent label each cart’s main product group and summarize it, then post. | 10 records with correct labels. |
Each job had a time limit (12 minutes for the build and the agentic flow, 8 for the rest) and a reply limit, which we set before the test, so neither side could grind to an answer. Each AI had two chances at each job, and we report the better attempt. The same coding agent (Claude) drove both browsers, sent each prompt word for word, answered questions only from a fixed answer sheet, and recorded what happened. We checked every result against the records at the endpoint.
One rule matters for reading the results. A person was allowed to do platform housekeeping on both sides: enable or deploy a flow, approve a change, click continue, reload a page, or cancel a stuck run. A person was not allowed to build or configure any part of the integration itself. Where the AI stopped and asked us to build something, we stopped too, on both platforms.
Before we share the results, it’s important to note the differences between the in-app AI experiences in Celigo and Boomi.
On Boomi, the AI interface isn’t one thing: there’s a chat assistant labeled as “Ask Boomi AI”, the Orchestrate design canvas and Resolve Agent, each a separate experience built for a different part of the job. We used whichever experience fit each job best and credited Boomi with the best result any of them reached. In practice the chat handled almost every job; on one attempt, the agentic job started in the chat and continued in Orchestrate. We also ran Boomi GPT and Resolve Agent on their own; neither did better on these jobs.
Celigo Ora, by contrast, has full contextual awareness of every flow, connection, API and dependency across the whole account before it executes on any prompt. Its interface travels with you across the whole platform: every job started from “Ask Ora” in the integrator.io header, and wherever we were in the platform, it was the same natural language chat interface.
What got finished
Correct records at the endpoint
- 73 / 73 Celigo Ora
- 22 / 73 Boomi
We counted a record as correct only if it arrived at the endpoint we asked for, with the AI having done the build or change itself, and we took each side’s better attempt at each job. The 73 counts the records expected from the build (41), update (22) and agentic (10) jobs.
All of Boomi’s 22 come from the update. Neither of its builds got records to the endpoint we asked for: one posted 7 records to a different address, and the other couldn’t run (see Build). Its agentic flow never ran.
One caveat on Ora’s side: the agentic posts arrived only after we cleared a queue (see Agentic flow).
| Job | Ora | Boomi |
|---|---|---|
| Build | 41 of 41, when run in the test job | 0 of 41: one build posted to another address, the other couldn’t run |
| Test | 41 of 41, payloads shown | 0 of 41 at our endpoint; counts only, no payloads |
| Update | 22 of 22 | 22 of 22 |
| Troubleshoot | Fixed and re-run; all 30 carts read | Fixed, deployed and re-run; 30 of 30 |
| Agentic flow | 10 of 10, after we cleared a queue | 0 of 10 |
Records at the endpoint we asked for, better attempt per job
Build
Ora built the integration in 2.7 minutes, and when we ran it in the next job, 41 of 41 records arrived.
Neither of Boomi’s builds delivered. The first took 8.1 minutes, stopped once at its per-turn working limit and needed a “Continue”. Its own plan said it would fetch every cart in one call, but it reused an existing operation without that setting, so it read only the first of seven pages and posted to a different address than the one we gave. The second was done in about 2.5 minutes with no stops, read every cart in one call and used the right address. But where its plan said it would use the account’s existing connections, it created two new ones on a connector type that doesn’t support these calls, so that build couldn’t run as delivered. Neither build set the hourly schedule.
Test
Ora asked to enable the flow first (it won’t run a disabled one, by design) and enabled it after our approval. Then it ran the flow: 41 of 41 arrived, and it showed us the captured JSON bodies.
Boomi’s chat deployed and ran its own build from the conversation, which was among the most impressive things it did. It reported counts (7 orders, 7 posts, 0 errors) but no payloads, described the run as posting each matching order to the webhook, and didn’t notice it had read one page of seven or posted to another address. 0 of 41 reached the endpoint we’d asked for. On the NetSuite question, both gave sensible answers.
Update
Ora got this one wrong first. On its first attempt it flagged its own draft as not ready (the filter still let two-product carts through and the new field was a placeholder), a second pass failed validation, and nothing was applied. On its second attempt it again said at first that the field wasn’t in the source data, then corrected itself after one reply and applied the filter and the field after approval. When we ran the integration ourselves, 22 of 22 records arrived.
Boomi’s AI also needed both attempts. On its first, it applied the field correctly, but its filter failed platform validation twice, and it stopped and asked us to add that step by hand. We’d rather an AI say “I can’t safely guess the exact XML syntax Boomi’s platform expects” than guess, and that’s what it said. On its second, the filter step passed validation. It added the filter and the field, deployed the change and ran it, in under six minutes with one approval and three replies. 22 of 22 records were right, with none extra.
Troubleshoot
Ora took 7.7 minutes in total, about two of which were a hung approval button that needed a page reload. It had its fix staged after about three minutes and 20 seconds, then applied it and re-ran, reading all 30 carts.
Boomi’s AI read the run logs and had a fix ready in under a minute and a half, against more than three minutes for Ora, and named the root cause in under a minute. After one approval it saved the fix, then deployed it and re-ran the integration itself: 30 of 30 arrived within about three and a half minutes of our prompt.
Agentic flow
Ora read the carts, built the agent step, labelled 10 of 10 correctly in 5.6 minutes, after three approvals. The ten posts arrived once we cancelled an earlier stuck run that was holding up the connection’s queue.
Boomi’s AI built a real Agentstudio agent from one prompt on both attempts, which is worth saying twice, but nothing ran either time. On one attempt it moved the job from the chat to Orchestrate and ran out its 12 minutes with the posting process unbuilt and the agent not deployed. On the other it stayed in the chat, planned the whole flow, said up front which step would need us, and in about three minutes built the agent, the payload format and the posting step. Then it stopped at the step it had flagged. Calling an agent from an integration needs an agent operation, which, in its words, “is created only in the Boomi UI”, and our rules don’t let the tester build anything by hand. 0 of 10.
Where Boomi’s AI was better
We want to be fair about this, so first, where Boomi’s AI was ahead.
- It asked for fewer approvals: four, to Ora’s nine. Ora stops and checks more often by design; some people will prefer that, some won’t.
- It showed us a plan before building anything. Ora builds and validates a staged draft first, then asks you to approve it.
- The home-page chat deployed and ran integrations from the conversation, including its update and its fix, and read the results back. Boomi’s Orchestrate documentation says “Orchestrate cannot test or deploy assets”; the chat we used went further than that, and it was in Boomi’s favour.
- It got to a ready fix faster than Ora, and named the root cause in under a minute.
- It stopped honestly when it couldn’t be sure, instead of guessing.
- It built a real Agentstudio agent from a single prompt (ready, but not deployed).
Where Ora fell short
And here is our side of the ledger.
- Answers took two to five minutes each. Boomi’s ranged from under a minute to eight minutes.
- It asked for more approvals, up to three approvals and three replies on one job.
- A preview refresh saved a mock output onto an export.
- One more difference is a matter of taste: Ora builds a draft and then asks for approval, while Boomi shows a plan before it builds anything.
What we think it means
Boomi is a capable established platform. It has added capabilities over the years, some built and some bought, and its AI has arrived as a set of assistants, each attached to a different part of the platform. In the agentic era we think that shape has its limits, whether you are in the UI or working with the platform headlessly.
When we put Boomi’s in-app AI through five real jobs it did real, concrete things that would be valuable to a business user. It deployed an integration from chat, diagnosed a failure quickly, built an agent from a prompt, and knew when to stop. With two chances, it also carried two jobs all the way through: it changed the filter and fixed the broken path, deployed and ran both, and the right records arrived.
The jobs that start something new went less well. Neither of its builds delivered, for a different reason each time, and its agentic flow never ran. On those jobs, work was still left for a person after both attempts: getting the build to run, setting the schedule and, for the agentic flow, deploying the agent and creating its operation in Boomi’s UI.
Here’s our read on why. Boomi’s AI shows up as separate experiences, each good at one piece of the integration puzzle, rather than one with context across your whole account, and that makes it harder for any one of them to carry a job end to end. Even when the chat kept the agentic job to itself, the agent it built in Agentstudio couldn’t be called from an integration until a person connected the two in Boomi’s UI. Again, that’s just our read and not something a single test can definitively prove.
Celigo did things differently. We chose not to grow through acquisition, and instead built one platform from the start. Every core capability (building, testing, changing, fixing) works against the same platform APIs, so the experience is unified, whether you’re in the UI or working agentically. And Ora was built on top of that same shared foundation.
That’s why Ora isn’t a layer stitched across separate tools. It’s one assistant working across the whole unified platform, so it found the existing connections and traced the failing flow’s dependencies in the troubleshooting job. When we asked it to build, test, change or fix something, it did the whole job and showed us what it did. It needed approvals along the way, and its update needed a second attempt, but in the end it carried the work through to records at the endpoint.
Given two chances, Boomi’s AI finished the update and the fix on its own.
On the build and the agentic flow, the last mile still fell to a person.
Try it yourself
You can repeat this experiment. Pick a small integration and ask the AI inside your current iPaaS platform to build it from scratch, run it, change it, break it and fix it, and add an AI step. Count how many times it hands something back to you that is unfinished. Give each job a second chance, as we did, because results can change between attempts. If your account has several integrations, also ask each assistant what else would break if you changed a shared connection or field, which our five jobs did not test. Then try the same test in Ora.