14 min read

We gave five integration jobs to two in-product AIs. Here’s how far each one got.

Published Nov 18, 2026
We asked Celigo Ora and Boomi’s in-product AI to build and manage an integration in the browser, the way a business user would, and gave each two chances at every job. Here’s what each one finished and what it handed back.
Youssef Zouhairi

Solutions Consultant

Youssef Zouhairi
ora vs boomi

A few days back, you might remember that we ran a different head-to-head experiment: we built the same integration on Boomi and Celigo from a terminal, through each vendor’s own headless tooling for coding agents. That effort was about comparing the headless building experiences outside of both platforms.

This one is more relevant to users who prefer to use the UI instead: when you ask the AI inside the product to do a job, does the job actually get done? Or do you end up manually working through what the AI created, which means more work and more frustration?

As you know, building the integration is only part of the job; ongoing management is equally important and sometimes even more cumbersome. So we picked one small integration and followed it across the whole lifecycle to understand the full breadth of both platforms’ in-app AI capabilities: building, testing, updating, troubleshooting, and even adding an agentic step. We prompted the in-app AI experiences the way a typical business user would.

The jobs

The integration we asked each AI to build moves shopping carts from a public test API to a webhook endpoint. It’s a stand-in for one of the most common integration patterns: pull records from one system, filter and reshape them, and send them to another on a schedule.

The five jobs:

Job What we asked How we checked
1 · Build An hourly integration that reads all carts (the source pages its results), keeps carts over 30,000, maps four fields, and posts each one to the endpoint. 41 records should arrive.
2 · Test Run it once and show us what was sent. Also: how would we test a NetSuite flow without credentials? 41 records at the endpoint, payloads visible.
3 · Update Change the filter to carts with at least three distinct products, and add a field. 22 records.
4 · Troubleshoot The integration is failing with a 404. Find the root cause and fix it. (We had changed the path to /cartz.) Root cause named, fix applied, re-run reads 30 carts.
5 · Agentic flow Read 10 carts, have an AI agent label each cart’s main product group and summarize it, then post. 10 records with correct labels.

Each job had a time limit (12 minutes for the build and the agentic flow, 8 for the rest) and a reply limit, which we set before the test, so neither side could grind to an answer. Each AI had two chances at each job, and we report the better attempt. The same coding agent (Claude) drove both browsers, sent each prompt word for word, answered questions only from a fixed answer sheet, and recorded what happened. We checked every result against the records at the endpoint.

One rule matters for reading the results. A person was allowed to do platform housekeeping on both sides: enable or deploy a flow, approve a change, click continue, reload a page, or cancel a stuck run. A person was not allowed to build or configure any part of the integration itself. Where the AI stopped and asked us to build something, we stopped too, on both platforms.

Before we share the results, it’s important to note the differences between the in-app AI experiences in Celigo and Boomi.

On Boomi, the AI interface isn’t one thing: there’s a chat assistant labeled as “Ask Boomi AI”, the Orchestrate design canvas and Resolve Agent, each a separate experience built for a different part of the job. We used whichever experience fit each job best and credited Boomi with the best result any of them reached. In practice the chat handled almost every job; on one attempt, the agentic job started in the chat and continued in Orchestrate. We also ran Boomi GPT and Resolve Agent on their own; neither did better on these jobs.

Celigo Ora, by contrast, has full contextual awareness of every flow, connection, API and dependency across the whole account before it executes on any prompt. Its interface travels with you across the whole platform: every job started from “Ask Ora” in the integrator.io header, and wherever we were in the platform, it was the same natural language chat interface.

What got finished

Correct records at the endpoint

  • 73 / 73 Celigo Ora
  • 22 / 73 Boomi

We counted a record as correct only if it arrived at the endpoint we asked for, with the AI having done the build or change itself, and we took each side’s better attempt at each job. The 73 counts the records expected from the build (41), update (22) and agentic (10) jobs.

All of Boomi’s 22 come from the update. Neither of its builds got records to the endpoint we asked for: one posted 7 records to a different address, and the other couldn’t run (see Build). Its agentic flow never ran.

One caveat on Ora’s side: the agentic posts arrived only after we cleared a queue (see Agentic flow).

Job Ora Boomi
Build 41 of 41, when run in the test job 0 of 41: one build posted to another address, the other couldn’t run
Test 41 of 41, payloads shown 0 of 41 at our endpoint; counts only, no payloads
Update 22 of 22 22 of 22
Troubleshoot Fixed and re-run; all 30 carts read Fixed, deployed and re-run; 30 of 30
Agentic flow 10 of 10, after we cleared a queue 0 of 10

Records at the endpoint we asked for, better attempt per job

Build

Ora built the integration in 2.7 minutes, and when we ran it in the next job, 41 of 41 records arrived.

Neither of Boomi’s builds delivered. The first took 8.1 minutes, stopped once at its per-turn working limit and needed a “Continue”. Its own plan said it would fetch every cart in one call, but it reused an existing operation without that setting, so it read only the first of seven pages and posted to a different address than the one we gave. The second was done in about 2.5 minutes with no stops, read every cart in one call and used the right address. But where its plan said it would use the account’s existing connections, it created two new ones on a connector type that doesn’t support these calls, so that build couldn’t run as delivered. Neither build set the hourly schedule.

Test

Ora asked to enable the flow first (it won’t run a disabled one, by design) and enabled it after our approval. Then it ran the flow: 41 of 41 arrived, and it showed us the captured JSON bodies.

Boomi’s chat deployed and ran its own build from the conversation, which was among the most impressive things it did. It reported counts (7 orders, 7 posts, 0 errors) but no payloads, described the run as posting each matching order to the webhook, and didn’t notice it had read one page of seven or posted to another address. 0 of 41 reached the endpoint we’d asked for. On the NetSuite question, both gave sensible answers.

Update

Ora got this one wrong first. On its first attempt it flagged its own draft as not ready (the filter still let two-product carts through and the new field was a placeholder), a second pass failed validation, and nothing was applied. On its second attempt it again said at first that the field wasn’t in the source data, then corrected itself after one reply and applied the filter and the field after approval. When we ran the integration ourselves, 22 of 22 records arrived.

Boomi’s AI also needed both attempts. On its first, it applied the field correctly, but its filter failed platform validation twice, and it stopped and asked us to add that step by hand. We’d rather an AI say “I can’t safely guess the exact XML syntax Boomi’s platform expects” than guess, and that’s what it said. On its second, the filter step passed validation. It added the filter and the field, deployed the change and ran it, in under six minutes with one approval and three replies. 22 of 22 records were right, with none extra.

Troubleshoot

Ora took 7.7 minutes in total, about two of which were a hung approval button that needed a page reload. It had its fix staged after about three minutes and 20 seconds, then applied it and re-ran, reading all 30 carts.

Boomi’s AI read the run logs and had a fix ready in under a minute and a half, against more than three minutes for Ora, and named the root cause in under a minute. After one approval it saved the fix, then deployed it and re-ran the integration itself: 30 of 30 arrived within about three and a half minutes of our prompt.

Agentic flow

Ora read the carts, built the agent step, labelled 10 of 10 correctly in 5.6 minutes, after three approvals. The ten posts arrived once we cancelled an earlier stuck run that was holding up the connection’s queue.

Boomi’s AI built a real Agentstudio agent from one prompt on both attempts, which is worth saying twice, but nothing ran either time. On one attempt it moved the job from the chat to Orchestrate and ran out its 12 minutes with the posting process unbuilt and the agent not deployed. On the other it stayed in the chat, planned the whole flow, said up front which step would need us, and in about three minutes built the agent, the payload format and the posting step. Then it stopped at the step it had flagged. Calling an agent from an integration needs an agent operation, which, in its words, “is created only in the Boomi UI”, and our rules don’t let the tester build anything by hand. 0 of 10.

Where Boomi’s AI was better

We want to be fair about this, so first, where Boomi’s AI was ahead.

  • It asked for fewer approvals: four, to Ora’s nine. Ora stops and checks more often by design; some people will prefer that, some won’t.
  • It showed us a plan before building anything. Ora builds and validates a staged draft first, then asks you to approve it.
  • The home-page chat deployed and ran integrations from the conversation, including its update and its fix, and read the results back. Boomi’s Orchestrate documentation says “Orchestrate cannot test or deploy assets”; the chat we used went further than that, and it was in Boomi’s favour.
  • It got to a ready fix faster than Ora, and named the root cause in under a minute.
  • It stopped honestly when it couldn’t be sure, instead of guessing.
  • It built a real Agentstudio agent from a single prompt (ready, but not deployed).

Where Ora fell short

And here is our side of the ledger.

  • Answers took two to five minutes each. Boomi’s ranged from under a minute to eight minutes.
  • It asked for more approvals, up to three approvals and three replies on one job.
  • A preview refresh saved a mock output onto an export.
  • One more difference is a matter of taste: Ora builds a draft and then asks for approval, while Boomi shows a plan before it builds anything.

What we think it means

Boomi is a capable established platform. It has added capabilities over the years, some built and some bought, and its AI has arrived as a set of assistants, each attached to a different part of the platform. In the agentic era we think that shape has its limits, whether you are in the UI or working with the platform headlessly.

When we put Boomi’s in-app AI through five real jobs it did real, concrete things that would be valuable to a business user. It deployed an integration from chat, diagnosed a failure quickly, built an agent from a prompt, and knew when to stop. With two chances, it also carried two jobs all the way through: it changed the filter and fixed the broken path, deployed and ran both, and the right records arrived.

The jobs that start something new went less well. Neither of its builds delivered, for a different reason each time, and its agentic flow never ran. On those jobs, work was still left for a person after both attempts: getting the build to run, setting the schedule and, for the agentic flow, deploying the agent and creating its operation in Boomi’s UI.

Here’s our read on why. Boomi’s AI shows up as separate experiences, each good at one piece of the integration puzzle, rather than one with context across your whole account, and that makes it harder for any one of them to carry a job end to end. Even when the chat kept the agentic job to itself, the agent it built in Agentstudio couldn’t be called from an integration until a person connected the two in Boomi’s UI. Again, that’s just our read and not something a single test can definitively prove.

Celigo did things differently. We chose not to grow through acquisition, and instead built one platform from the start. Every core capability (building, testing, changing, fixing) works against the same platform APIs, so the experience is unified, whether you’re in the UI or working agentically. And Ora was built on top of that same shared foundation.

That’s why Ora isn’t a layer stitched across separate tools. It’s one assistant working across the whole unified platform, so it found the existing connections and traced the failing flow’s dependencies in the troubleshooting job. When we asked it to build, test, change or fix something, it did the whole job and showed us what it did. It needed approvals along the way, and its update needed a second attempt, but in the end it carried the work through to records at the endpoint.

Given two chances, Boomi’s AI finished the update and the fix on its own.

On the build and the agentic flow, the last mile still fell to a person.

Try it yourself

You can repeat this experiment. Pick a small integration and ask the AI inside your current iPaaS platform to build it from scratch, run it, change it, break it and fix it, and add an AI step. Count how many times it hands something back to you that is unfinished. Give each job a second chance, as we did, because results can change between attempts. If your account has several integrations, also ask each assistant what else would break if you changed a shared connection or field, which our five jobs did not test. Then try the same test in Ora.

One assistant that finishes the job.

Ora builds, changes, troubleshoots and runs integrations in plain language, from any page.

Start a free trial

How we ran it. This is a small recorded test by a Celigo engineer, not a benchmark, and results should be read as indicative. Each AI had two chances at each job, and we report the better attempt. The same coding agent (Claude Code, running Claude Opus 5.5) drove both platforms’ browser interfaces and graded the written results, and a Celigo tester signed in to both platforms and recorded the test sessions; the endpoint counts are objective, the written judgments were not independently verified. The time and reply limits for each job were set by us before the test. A person was allowed to do platform housekeeping on both sides (enable or deploy, approve, click Continue, reload, cancel a stuck run) and was not allowed to build or configure any part of an integration on either. Celigo: Ora, on a free-trial account with 20 AI-agent credits. Boomi: a free trial account where we used the “Ask Boomi AI” chat and the Orchestrate canvas as each job called for, also ran Boomi GPT and Resolve Agent on their own, and credited Boomi with the best result any tool reached. Source: DummyJSON, a public test API; sink: webhook.site, a public webhook endpoint. Limits per job: 12 minutes and 5 replies for the build and the agentic flow, 8 minutes and 4 replies for the rest. The two platforms were tested at different times of day. The Boomi agent we inspected was set to Claude Sonnet 4.5. Claude also drove and graded both sides, and we make no claim about model quality. Timings and counts describe these runs and will vary with configuration and operator behaviour. Redacted excerpts of the sessions are available on request under NDA. Boomi’s product documentation (Orchestrate FAQ) was checked and is quoted in short excerpts for comparison.

Boomi, Agentstudio, Boomi GPT and Resolve Agent are trademarks of Boomi, LP. Claude is a trademark of Anthropic. NetSuite is a trademark of Oracle.