AI Watch methodology

Vigilo watches AI.

AI Watch is a public, dated, verifiable record of two things: where AI is heading, and what deployed AI actually does. This page explains what goes into the record, how each number is counted, and how you can check that nothing was changed after the fact. It is written to be read by an owner, a parent, and a regulator alike. This file is the source for the page at vigilolabs.ai/watch/methodology.

Taxonomy version v3. Last revised October 14, 2026.

Three ways to read it

The same record is shown at three depths. The data never changes; only how much of it you see.

1. The glance. One number from 1 to 5, the Vigilo Vector, with one sentence under it.
2. The picture. Ten tiles, one for each signal, each with an arrow and the newest headline.
3. The record. Every entry, every count, the hash that covers them, and this page.

Above all three sits one count, the Rogue AI tracker (see "Rogue AI activity").

The glance is only worth trusting because the record is underneath it.

The Vigilo Index

The Vigilo Index is one number for how fast the record is moving, compared with its own usual pace. 100 means the counters are moving at the pace they kept over the last 90 days. 150 means half again as fast. 50 means half as fast.

What it runs on. Only the counters built from the record's own entries: the ones that count approved, dated entries. Counters built on figures from outside sources, estimates and official counts alike (weekly ChatGPT users, the bot share of web traffic, fraud complaints, the children's survey, models trained at GPT-4 scale), stay on the wall as context, labelled as they are, and are not part of the index. The index moves only when the record does.

How it is calculated. For every counter built from entries:

• its velocity is how much it moved in the last 7 days, per day;
• its baseline is how much it moved in the last 90 days, per day;
• its ratio is the velocity over the baseline.

The index is the weighted average of the ratios, times 100, rounded to a whole number. Harm and Power counters weigh 2. Every other counter weighs 1.

Two guards. A counter that counts entries uses add-one smoothing: (entries in the last 7 days + 1) divided by (entries the 90-day pace would give in 7 days + 1). Without it, a counter that sees one item a quarter would jump the index on its own the day that item arrives. And no counter's ratio may exceed 5.

What is left out. A counter whose entries cannot be counted yet, because none has been classified to it, is left out of the index, not counted as zero.

The change and the arrow. The 24-hour and 7-day changes compare today's index with the index worked out the same way for 24 hours and 7 days ago, from the record as it stood then. The arrow compares today with 7 days ago.

The index is worked out from the same entries as the counters. Given the snapshot and the moment, anyone can recompute it.

Vigilo Vector

The Vigilo Vector is a whole number from 1 to 5.

The rule. Count the approved entries that are marked confirmed, that carry signal 3 (AI acting without being asked), 4 (what is hidden from view), 5 (how capable it is getting), 8 (AI loose in the world), or 10 (AI used as a weapon), and that were published in the 14 days before the snapshot. Then:

That is the whole rule. It is computed by software from the record, it is stored with every snapshot, and the snapshot's hash covers it. It will not be changed without a new taxonomy version and a new edition of this page.

Signal 9 (what is shipping) is not counted, and neither are signals 1, 2, 6, and 7.

Changed October 14, 2026 (taxonomy v3). Signal 10, AI used as a weapon, now counts toward the Vector. Before v3 the rule counted signals 3, 4, 5, and 8. The bands did not change. Every snapshot carries the taxonomy version it was taken under, and snapshots taken before v3 keep the Vector they published.

The direction. Beside the Vector is an arrow: rising, steady, or falling. It compares the same count for the 14 days before the snapshot with the count for the 14 days before those. More is rising, fewer is falling, equal is steady. It is published with every snapshot as the stat vector-direction, and it is shown as an arrow, never as a second number.

What the number is and is not. The number carries no opinion. It is a count of confirmed events of four kinds, put into bands. It is not a forecast, not a measure of danger, and not comparable to any other organization's scale. A busy fortnight of reporting raises it; a quiet one lowers it.

The sentence. The one sentence under the Vector is the only opinion in the glance. Software drafts it from the fortnight's approved entries. It is published as drafted when it is one sentence of under 30 words, in the active voice and the present tense, and every number in it is the count; otherwise the previous sentence stays and a person at Vigilo looks at the draft. If the count moves to a new Vector before a sentence has been approved for it, the page keeps showing the previous Vector and its sentence until one is. The count itself is always published as it is, in stats.json.

The arrows on the tiles. Each tile compares the number of approved entries for its signal in the last 14 days with the 14 days before: more is rising, fewer is falling, the same is steady. Every approved entry counts here, whatever its confidence.

Embed the Vector

Anyone may show the current Vector on their own site.

• The badge: https://vigilolabs.ai/watch/vector.svg, an image of the eye mark with its dot in the Vector colour, the number, the direction arrow, and the label. Paste this to show it linked to the record:

<a href="https://vigilolabs.ai/watch"><img src="https://vigilolabs.ai/watch/vector.svg" alt="Vigilo Vector" width="240" height="72"></a>

• The data: https://vigilolabs.ai/api/watch/vector.json gives the number, its label and colour, the direction, the sentence, the hash of the snapshot it comes from, and when that snapshot was taken.

Both are drawn from the newest snapshot and may be up to an hour behind it. The rule that produces the number is published above and never changes without a new taxonomy version.

The earlier name. Until October 11, 2026 the Vector was published as the "Watch Level", and the stat was named watch-level. Snapshots taken before the rename keep that name, because nothing published is edited; snapshots taken after it publish the stat as vector. stats.json gives it under vector in both cases, and until November 10, 2026 also under watch-level with the same value, marked deprecated.

What is in the record

The feed. Individual entries: a headline, a short summary written by Vigilo, a link to the source, a date, and labels. Entries come from public sources. Vigilo does not republish a source's article or its data; it links to it.

The statistics. Counts computed from public datasets, with the dataset named and its license stated beside the number.

Nothing appears on the public page unless it passed the published approval rule (see "The approval rule") or a person at Vigilo approved it. Software gathers and sorts candidates; the rule decides what is approved, and every decision is logged with each check it made.

The record as data. The whole record can be read as JSON, in three forms. Each carries the snapshot hash, a months list of every month that holds an entry with its count, and a next link.

• https://vigilolabs.ai/api/watch/feed.json gives what this page opens on: the last 30 days, or more when those days hold fewer than 10 entries. next is the newest month not included.
• https://vigilolabs.ai/api/watch/feed.json?month=2026-09 gives one whole month. next is the month before it that holds entries, and is empty at the start of the record.
• https://vigilolabs.ai/api/watch/feed.json?all=1 gives every entry in one file.

Following next from the first form reaches every entry. A month can repeat entries already seen in the first form, so match on id. Nothing is left out of the record by any limit.

The three source classes

Every entry and every number carries one of these labels.

Measures that are not about AI behavior

Some numbers in the record are there to give the others scale, not to describe what AI did. The share of US businesses that report using AI (US Census Bureau) is one; the number of frontier models published each month (Epoch AI) is another. These carry no signal and no category. Each is marked as a denominator or as context, and is never added to a count of incidents. No rate that divides incidents by such a measure is published yet.

The two labels on every entry

An entry may carry a signal, a category, or both. An entry with neither is not published.

Signal: where AI is heading. Ten signals: (1) lab research automation, (2) researcher exits, (3) lab control failures, (4) transparency going dark, (5) capability thresholds, (6) regulatory and legal motion, (7) money signals, (8) deployed-agent incidents, (9) model and agent releases, (10) weaponized AI and malicious use.

Signal 9, model and agent releases ("What is shipping"). The record of what the labs ship, with a line drawn between a release and a press release. It counts: a new model or model family from a frontier lab, or a major version of one; a new agent product or an agent capability that acts on the user's behalf, such as a browser agent, a computer-use agent, an autonomous coding agent, or an agent with its own accounts or email; a capability a lab itself calls new, such as a modality, a context window step, a tool-use ability, or an autonomy level; and a deprecation or withdrawal of a frontier model. It does not count: customer stories, case studies, and cost-savings posts; pricing changes, usage tips, guides, tutorials, and best practices; partnerships and distribution deals with no new capability; engineering posts about infrastructure; essays, opinion, and speeches; or hardware announcements from chip vendors, unless a frontier lab ships a model on them. Signal 9 does not feed the Vigilo Vector.

Signal 10, weaponized AI and malicious use ("AI used as a weapon"). Harm a person or group chose to do with AI. It counts: AI-assisted cyberattacks, intrusions, or malware seen in the wild; fraud, extortion, or scam campaigns run with cloned voices, deepfake video, or generated messages, where people were targeted; influence or disinformation operations using AI-generated content, attributed by a platform, a government, or a named researcher; attempts to use AI for weapons uplift; and abusive or non-consensual imagery generated with AI, as reported by the police, a court, a regulator, or a platform. It does not count an AI system acting on its own (that is signal 3 or 8), red-team results or jailbreak research with no misuse in the wild, security products and vendor surveys, laws or warnings with no case behind them (signal 6), or opinion. The line against signal 8 is who acted: a person wielding AI is signal 10, and is never rogue; only an AI system acting on its own can be. Entries recorded before October 14, 2026 keep the signal they were given.

Category: what a deployed AI system did. Ten categories: data loss, data exfiltration, unauthorized action, injection, impersonation, misinformation, physical safety, bias and discrimination, availability, and other.

Full definitions, and the table that maps each source's own categories onto Vigilo's, are in the taxonomy.

What this means for you

Every entry carries a short line, one to three sentences, that says what it means for a person at home, at work, and in government. It is written for a reader who is new to AI. Software drafts it, the approval rule checks it, and an entry cannot be approved without it. A person can correct it after the fact. The "Plain" switch on the feed shows these lines in place of the fuller summaries.

Confidence

Every feed entry carries one confidence label.

Software proposes a label. The person approving the entry sets it.

How an entry reaches the feed

1. Gather. Once a day, software reads each source through an interface the source offers for that purpose: an API, a published data file, or an RSS feed. Vigilo does not scrape pages, and does not use a source whose terms forbid it. Sources with no such interface are read by a person.
2. No repeats. A link that is already in the record is dropped. A headline that is more than 80 percent similar to another from the previous 14 days is set aside and marked as a possible duplicate for a person to decide.
3. Sort. A small AI model proposes a signal, a category, and a confidence label, with a one-line reason, and says whether the item looks like rogue AI activity (see "Rogue AI activity"). An item it thinks is rogue is put to a larger model a second time before the proposal is recorded. Software then checks that the link resolves and that the page it points to matches the headline. The model's proposal is a suggestion only. An item that is not about AI at all is set aside at this step with the reason recorded; it is kept, not deleted, and a person can put it back.
4. Approve. Once a day, after sorting, the published approval rule reads each candidate: it approves it, rejects it, or holds it for a person (see "The approval rule"). A person at Vigilo decides what the rule holds, and can correct what it approved. Rejected candidates are kept, not deleted.
5. Publish. Once a day a snapshot is taken (see "How to verify the record"). Approved entries become public with the next snapshot.

Vigilo's own commentary does not name a vendor, lab, or product unless the source names it. Statistics are by category, signal, sector, and time, never by company.

A second automated pass. Some candidates come out of the first sorting step with no label at all. A second automated pass then reads the source page and drafts, for every item it reads, the closest-fitting signal, a category where the item is an incident, the headline, the summary, the confidence, and the "what this means for you" line. When the model doubts that an item belongs in the record, it says so in one sentence for the reviewer and drafts the item all the same. It never publishes and it never discards: the model proposes, and the approval rule or a person decides. Before October 14, 2026 every published entry was approved by a person, who could change any word of the draft or reject it. The proportion of drafted entries approved without edits is published every month as pass2-unedited-share, described below. Some sources refuse automated readers. When a page will not load, the pass makes one more request and, for the Federal Register, reads the document through the agency's own public API. If the page still cannot be read, the pass drafts from the feed's own headline and summary and nothing else, and the draft is marked unverified, because no source was read. The approval rule never approves an unverified draft; a person decides.

The approval rule

Entries are approved by a published rule and logged with every check. A person reviews what the rule holds and can correct what it approved. Corrections are public.

Until October 14, 2026, a person at Vigilo approved every entry. From that date a rule does, because what makes an entry trustworthy is a named source, a link that was checked, a fixed set of labels, and a record of why it was approved, and a rule can enforce all four and leave its reasons behind. Each entry says which approved it: "Approved by rule" or "Approved by person". Entries approved before October 14, 2026 were all approved by a person.

Once a day the rule reads each candidate the sorting steps have finished with. It approves a candidate only when all ten of these hold, and when the second pass raised no doubt that it belongs:

1. The candidate is waiting for a decision: not a possible duplicate, not already decided.
2. Software opened the source link and the page's title matches the headline.
3. The source qualifies for the confidence given: "confirmed" needs a source on the published list that can confirm (an incident database, a lab's own system card or disclosure, a court, a regulator, a company breach notice, a peer-reviewed report); "single source" needs any named publisher.
4. It has a signal, a category, or both.
5. If it is marked as rogue AI activity, its signal is 3, 4, or 8.
6. The headline is 20 to 140 characters, not in capitals, and ends in a letter, a number, a period, or a question mark.
7. The summary is one to three sentences and 40 to 400 characters.
8. The "what this means for you" line is one to three sentences, 20 to 320 characters, with no link and no company the headline or summary does not name.
9. It was published in the last 45 days.
10. Its headline is no more than 80 percent similar to any entry approved in the last 14 days.

It rejects a candidate whose source link is gone (there is none, or the page answers that it no longer exists), and one whose headline is more than 95 percent similar to an entry already approved: a repost. It holds everything else for a person: a possible duplicate, a source that cannot confirm, a draft that fails a check, anything the second pass doubted. A held candidate nobody decides on within 14 days is rejected as stale. Every decision, approved, rejected, or held, is logged with the result of all ten checks.

The rule is version rule-v1. It does not change without a new version and a notice on this page, the same discipline as the Vector.

Rogue AI activity

One number at the top of the page: how many times the record shows a deployed AI system going rogue. It is defined narrowly on purpose.

What counts. An entry is rogue AI activity when a deployed AI system did at least one of the following, and a named source documents it.

An entry can carry more than one type. A rogue entry always sits on one of three signals: what AI does when it is tested, what the labs are not saying, or AI loose in the world.

What does not count. Research about what a model could do in a lab, policy and regulation, funding, hiring, product launches, and opinion. A person using an AI tool to impersonate someone, such as a deepfake or voice-clone scam, does not count either: the person did it, not the system. Those stay in the record under their signals and are never in this number.

The number. The count of published entries that carry a rogue type and are marked confirmed. For a rogue entry, confirmed means a qualifying source documents it: the AI Incident Database, the lab's own system card or disclosure, a court filing, a regulator's action, a company's breach notice, or a peer-reviewed post-incident report. A news report on its own is single_source. Single-source entries are counted beside the number and never in it.

• The 7-day and 30-day figures are the confirmed rogue entries whose source is dated in the 7 or 30 days before the snapshot. An entry whose source gave no date is dated by when it entered the record.
• The twelve-month strip shows the twelve complete calendar months before the snapshot's month, by type. An entry with two types appears under both, so the types can add up to more than the month's entries.
• Where it comes from. From the record itself. The type is stored on the entry as a tag (rogue:R1 to rogue:R4), and tags are part of what the snapshot hash covers. The number is published with every snapshot as the stat rogue, in stats.json, and can be recomputed by anyone from feed.json.
• Who decides. Software proposes the type. The approval rule or a person approves the entry, and a person can correct the type afterwards.
• Earlier incidents. Incidents from the AI Incident Database in the twelve months before the tracker opened were loaded for review in one batch: those whose category is one of the R4 categories, proposed as confirmed when the database lists two or more independent reports and single_source otherwise. They were approved by a person like every other entry.
• Known gaps. The count is of what was documented and then approved, not of everything that happened. The AI Incident Database classifies incidents about three months after they are reported, so recent months are undercounted at first.
• Source class. DERIVED.

pass2-unedited-share

How often a person took the second pass's draft exactly as it stood.

• What is counted. For each calendar month, by the date an entry was approved: the entries approved; of those, the entries the second automated pass drafted and a person approved (an entry the approval rule approved is counted as approved and nothing more: the rule takes a draft as it stands by definition); of those, the entries approved with the drafted headline, summary, signal, category, confidence, and "what this means for you" line all unchanged. The share is the last number over the one before it. When no entry was drafted in a month the share is empty, not zero.
• Where it comes from. From the record itself. When an entry is approved, the record notes on the entry whether the second pass drafted it and whether the approver changed anything. That note is part of what the snapshot hash covers, so the count can be recomputed by anyone from the published record.
• What it does not say. It does not say the unedited entries are right, only that a person read them and changed nothing. It says nothing about the model's doubts, which are notes to the reviewer and are not published.
• Source class. DERIVED.

incidents-by-category-12m

AI incidents per month by category, trailing 12 months. DERIVED. Licensed CC BY-SA 4.0.

Source. The AI Incident Database (incidentdatabase.ai), a curated, human-edited record of AI incidents linked to news reports. Vigilo reads its weekly public snapshot. It is the only source for this statistic in v1. Data from the AI Incident Database is licensed CC BY-SA 4.0, and so are these counts.

What is counted. One for each incident in the database, in the month of the incident's date. The chart covers the twelve complete calendar months before the month of the snapshot; the current, incomplete month is left out.

How an incident gets its category. The AI Incident Database classifies incidents using the MIT AI Risk Repository's taxonomy. Vigilo maps each MIT subdomain to one Vigilo category. The full table is in the taxonomy. The MIT subdomain is kept with every count, so the mapping can be checked or redone.

Counted once. The record is append-only, so one incident can appear in it more than once: first without a category, and again when the database classifies it, or when the database corrects its date. The most recent version of each incident is the one that counts.

Known gaps. Read these before using the numbers.

• Recent months are incomplete in two ways. The database classifies incidents about three months after recording them, so recent incidents are shown as "unclassified" until then. Incidents are also often added to the database weeks or months after they happen. Both mean the most recent months will grow.
• "Other" is large. One MIT subdomain, "lack of capability or robustness," is broad and has no single home among Vigilo's categories. It is counted under "other" in taxonomy v1.
• These are reported incidents. The database is built from news reports. An incident nobody reported is not in it. Quiet failures, such as a system that silently deleted records at one business, are undercounted, probably severely.
• This is not a rate. A rising count can mean more incidents, more AI in use, or more reporting. v1 does not divide by a measure of deployment.
• One source. A second incident source (the OECD AI Incidents Monitor) will be added when Vigilo has sanctioned access. The two will be matched on the identifiers the OECD publishes for that purpose.

The digest and the monthly brief

The digest is an email sent daily or weekly to people who asked for it. It carries the Vigilo Vector and its sentence, then the entries that became public since that reader's last digest, each with its "what this means for you" line. It is sent only when there is something new. Vigilo stores the reader's email address and nothing else, and every digest has an unsubscribe link.

The monthly brief says what the record showed in the month before. Software drafts it from the snapshot and a person at Vigilo edits and publishes it. One rule is enforced by the software, not by care: every figure in a brief must be a figure that appears in the snapshot it was drawn from. A brief containing any other number cannot be published. A published brief is not edited afterwards.

Each brief comes with a one-page summary: the Vector and its sentence, the ten tiles, the month's three leading entries, and the hash of the snapshot it was drawn from. The three entries are chosen by a fixed rule: confirmed entries first, then those on the four Vector signals, then the newest.

How to verify the record

Every snapshot commits to the exact content of everything it covers and to the snapshot before it, using SHA-256. The snapshot's hash is printed on the public page next to what it covers. Anyone holding a copy of the record can recompute every hash without trusting Vigilo. The exact method is in the hash chain specification, section 8.

If any past entry or number were altered, every later hash would stop matching. The daily snapshot job runs this check over the whole record before it publishes anything, and publishes nothing if the check fails.

Corrections

Nothing in the record is edited or deleted. When something published turns out to be wrong, a correction is added to the corrections log on the public page, saying what was wrong and which snapshot carries the corrected version. The original stays in the record.

Changing the taxonomy

Changing a category, a signal, or how a source's categories map to Vigilo's requires a new taxonomy version and a new edition of this page. Entries and counts recorded under an earlier version keep that version.

• v1, October 10, 2026: ten categories, eight signals.
• v2, October 11, 2026: adds signal 9, model and agent releases. Nothing else changed.
• v3, October 14, 2026: adds signal 10, weaponized AI and malicious use, and counts it in the Vigilo Vector. No other definition or mapping changed.

Sources

The full register of sources, with each one's license and attribution, is in AI_WATCH_SOURCES.md.

The counters

Every counter on the Watch, with what it counts, the period it covers, how often it moves, and where its figures come from. A counter marked reported stands on reported incidents or official figures; one marked est. is an estimate. A counter with nothing to stand on yet shows no number, never a guess.

Dates. An item counts only with a dateline from its source. Undated items, items dated in the future, and unverified items are held out. Anything older than 48 hours is context, not a signal: it still counts toward its period, but is not shown as new.

Live estimates. Three counters move every second: AI agents running, daily chatbot users, and the bot share of traffic. Each is a straight line between the two most recent dated figures from a named source, shown one interval behind real time and marked “as of” that moment, so its digits move without ever going past a published figure. When the line reaches the newest figure, the number holds until the next one.

Figures in use

Every dated figure an estimate stands on, with the words that give it on the publisher’s own page. A figure is added only after it is read there.

Defined, not shown yet

These sum dollar or people amounts stated in entries. The record does not store amounts yet, so they are kept off the Watch: Announced compute spend, AI lobbying dollars, Jobs displaced, Jobs created, Dollars invested.

Taxonomy, version v3

Signals

Categories

Source register

Every source Vigilo has examined, including those it does not use. "Republished" means Vigilo may publish numbers taken from the source's data; a feed entry is a headline and a link, and may come from any source.