Every competitive-intelligence vendor now says 'AI'. Some of them mean an autonomous system that decides what to look at, checks it and routes the result. Most of them mean a language model that summarises whatever a crawler found. The difference matters because the two fail differently, and because buyers are starting to notice. This is a field guide to what agents actually do well, where they still need a person, and how to tell the two products apart in a demo.
A crawler plus a summary is not an agent
Reviewers already draw the line. A study of G2 reviews flags shallow automated-intelligence depth as a complaint for Kompyte at nine percent (Flares), and one review of Crayon describes its AI as layered onto a legacy monitoring engine (Copy.ai). For AlphaSense, around forty percent of reviewers in a G2 summary raise the quality of search and agent answers. The pattern is consistent: when 'AI' is a feature added to an alert feed, it inherits the feed's noise and adds a confident paragraph on top.
Six jobs agents do well, around the clock
Watching sources on a schedule without getting bored. De-duplicating the same press release across forty syndication sites. Extracting entities: which competitor, which product, which price, which region. Classifying the type of change: pricing, positioning, hiring, regulation, review sentiment. Drafting a first-pass, one-line summary with the source attached. Keeping the archive searchable so that 'what did they do last March' takes ten seconds. These are the hours an analyst used to spend before doing any thinking, and an agent does them at three in the morning.
Three jobs where judgment still needs a human
Deciding materiality: whether this change alters what your company should do, which depends on strategy the agent does not hold. Verifying under pressure: confirming a claim against a primary source when the source is ambiguous or the event is unfolding. Writing the recommendation: the paragraph a leadership team will act on, which has to weigh risk, timing and politics. An agent can draft all three; a person has to own them. In our experience the draft is right often enough to save hours and wrong often enough that sending it unreviewed would cost a client.
Hallucination and false-positive risk
The specific danger in market intelligence is a fluent, plausible, wrong summary. A model asked to summarise a competitor's page will happily invent a price if the page was ambiguous. The controls are boring and effective: a primary-source link on every item, a system rule that no specific number appears without a citation, uncertainty marked as an estimate or a range, and a human review gate on anything that leaves the system. If a vendor cannot show you the source link on an alert in the demo, the alert is a guess with good grammar.
How to evaluate 'AI-powered' claims in a demo
Ask to see a false positive the system caught itself and why it suppressed it. Ask to click through from an alert to the exact source and timestamp. Ask what happens when a source goes down or an API is rate-limited, and how you would know. Ask which decisions the system makes on its own and which it routes to a person. Ask for the names of the people who review outputs before clients see them. A vendor with real agents answers these in specifics. A vendor with a summariser changes the subject to the model.
The agent-plus-strategist loop
The architecture that works in production is a loop. Agents collect on their own cadence, each specialised in one signal type. A shared knowledge base holds what is known, with sources. A strategist reads what the agents flagged, decides what is material, and writes the brief and the document changes. The agents read the strategist's decisions back, so the next cycle is better filtered. Robit runs six agents this way with a human strategist as the decision layer; the client receives the strategist's output and can see the agents' feed in the portal if they want the evidence.
A buyer's checklist
Source link on every alert. Numbers only with citations. Explicit list of signal types monitored. Named human reviewer. Written deliverable with a cadence. Clear statement of what the system does not cover. A demo of a suppressed false positive. If a product meets all seven, the word 'AI' on its website is earned. If it meets three, you are buying a feed with a paragraph generator.
