Comment-to-DM benchmark methodology

Three numbers describe how well a comment-to-DM campaign works: match rate, delivery rate, and time to first DM. All three are defined below, along with the campaign to build, the comments to post, and the file the results are published in. No data has been collected, so no result appears on this page — the method is published first, on purpose.

Status: methodology published; data collection not started. Every measured field in this document is empty until the first run completes, at which point the results and the raw dataset are published here.

What the three metrics mean

Each metric is a count divided by a count, taken over one run against one connected page, and each has a denominator that excludes trials the platform never delivered to AFP at all.

Metric	Definition	Unit
Trials started	Comments posted during the run that produced an execution of the test campaign	count
Match rate	Trials whose execution left the condition node through Match, divided by trials started	percent
Delivery rate	Trials whose execution recorded a sent DM, divided by trials whose execution reached the send DM node	percent
Time to first DM	Seconds from the observer posting the comment to the DM appearing in the recipient’s conversation	seconds, reported as median and 90th percentile

Match rate measures the condition node, not the trigger. A trigger node holds no keyword configuration, so every comment on the post in scope starts an execution and the condition is the only place text is compared. Keyword matching rules defines the comparison each match type performs.

Delivery rate measures what AFP recorded, not what the recipient saw. A send DM node that raises an error records that error and the run stops advancing there, so the failure is visible; a message accepted by Facebook or Instagram and then filtered afterwards counts as delivered under this definition.

Time to first DM is measured on the observer’s clock at both ends — when the comment is submitted, and when the DM appears in the recipient’s conversation. Timestamps recorded inside AFP come from the platform and serve as a cross-check only.

Where each number is taken

How the test campaign is built

One campaign, four nodes, no waits. Anything else in the graph adds a variable that has nothing to do with the three metrics.

Create a campaign bound to the page under test and open it on the canvas.
Add a comment trigger. Set its post scope to a specific post and select a post created for the run, so ordinary traffic on other posts cannot enter the sample. Leave unique per user off — with it on, a repeat commenter is short-circuited and the trial never starts.
Add a condition node with the Keyword list match type, pointing at a keyword list created for the run. Leave reverse matching off, leave the image, GIF, and link filters off, and leave wait for reply off so the run resolves in one pass.
Add a send DM node with a fixed text message, connected to the condition’s Match output. Connect nothing to No Match, so a non-matching run ends at the condition and is unambiguous in the history.
Save the campaign, then activate it. Activation validates the graph and is a separate action from saving.
Deactivate every other campaign on that page. Each active campaign whose trigger matches starts its own execution, and two campaigns answering the same comment make delivery rate meaningless.

How the run is executed

Environment first: one Facebook page and one Instagram business account, both connected and subscribed, under an account whose subscription is active. A comment arriving while the subscription is inactive is recorded as a subscription error, and a page whose stored access has stopped working records a re-login error. Both count as environment failures and are excluded from every rate.

Fix the keyword list to a single keyword, and fix the DM text. Record both verbatim in the run notes.
Assign one distinct commenter account per trial. One person gets one active run of one campaign at a time, so a second comment from the same account while the first run is still active is skipped and would silently shrink the sample.
Comment from accounts other than the page itself. Comments the page posts are ignored, which rules out driving the run from the page account.
For each trial, post the comment text for its bucket and record the submission time on the observer’s clock.
Watch the recipient’s conversation and record the arrival time of the DM on the same clock. Record a non-arrival after a fixed cut-off, chosen before the run starts and stated with the results.
Read the campaign’s execution history and record, per trial, whether the run left the condition through Match and where it stopped. The history returns the 20 most recent runs and does not refresh on its own, so keep each batch under that and read it before starting the next.
Filter the activity feed to the page and the run’s time window, then export it. The export applies the feed’s filters and downloads as JSON, and it is the audit trail behind the trial-by-trial table.
Repeat on the second platform without changing the campaign shape, the keyword, or the DM text.

Comment buckets

Every trial belongs to one bucket, and the bucket fixes both the text posted and the branch it is expected to take.

Bucket	What is posted	Expected branch
Exact keyword	The keyword alone	Match
Keyword in a sentence	A sentence containing the keyword as a substring	Match
Keyword in upper case	The keyword typed in capital letters	Match
Near miss	A word that shares a prefix with the keyword without containing it	No Match
Unrelated text	A comment on the post that contains no keyword	No Match
Media only	A sticker or image with no text	No Match

A media-only comment reaches the condition as a placeholder text rather than as an empty string, which is why it belongs in the sample: it is the one bucket where what the condition compares is not what the commenter typed.

Results

No results. Data collection has not started, and nothing on this page has been measured.

The definitions, the campaign shape, the buckets, and the cut-off are fixed in public before any number exists, so the eventual figures can be checked against a protocol nobody could have tuned after seeing the data.

What the raw dataset will contain

One JSON file per run, published on this page alongside the summary table, with one object per trial. The activity export taken in step 7 is published with it unchanged, so every trial can be traced back to the row AFP recorded.

Field	What it holds
`trial_id`	Sequence number within the run
`platform`	`facebook` or `instagram`
`bucket`	One of the six bucket names above
`comment_text`	The exact text posted
`comment_posted_at`	Observer clock, when the comment was submitted
`execution_started`	Whether an execution was created for this comment
`branch_taken`	`match`, `no_match`, or `none`
`dm_recorded`	Whether the execution recorded a sent DM
`dm_observed_at`	Observer clock, when the DM appeared in the conversation
`time_to_first_dm_seconds`	Difference between the two observer timestamps
`outcome`	`delivered`, `no_match`, `no_execution`, `environment_failure`, or `timeout`
`notes`	Anything unusual about the trial, in plain text

Measured fields are null until the run fills them in. One trial from the published skeleton looks like this:

[
  {
    "trial_id": 1,
    "platform": "facebook",
    "bucket": "exact_keyword",
    "comment_text": "price",
    "comment_posted_at": null,
    "execution_started": null,
    "branch_taken": null,
    "dm_recorded": null,
    "dm_observed_at": null,
    "time_to_first_dm_seconds": null,
    "outcome": null,
    "notes": ""
  }
]

Limitations

Delivery is not AFP’s alone. AFP hands the message to Facebook or Instagram; what happens after that is outside anything measured here, and time to first DM includes the platform’s own delivery time.
One account, one audience, one window. A run covers one Facebook page and one Instagram business account at one time of day. It says nothing about pages with different traffic, different follower counts, or a different region.
Observer clocks are manual. Both ends of the timing measurement are recorded by a person watching two surfaces, so the resolution is coarse and human reaction time is inside the number.
The sample is synthetic. Comments written to fill buckets are not comments real followers write, and the near-miss bucket in particular is chosen to sit close to the matching boundary.
Case handling is unconfirmed. The upper-case bucket is measured, but the expected branch for it is unverified until engineering confirms how the comparison treats letter case.
No competitor is measured. These metrics describe AFP only. The comparison pages state plainly that no competitor performance number has been measured.
Rates are recorded, not guaranteed. Nothing here is a service level. A published figure describes one run on one day.

Research — the standard every study on this site is held to, and what is published so far.
Route comments to a DM — how to build the campaign this method measures.
Test and debug a campaign — reading the execution history the run depends on.
Execution states — what a stopped, waiting, or failed run means when you read it back.
Moderate comments — the activity feed and its export.