APIs, Analytics and AI-Surface Measurement

Overview

This is the most technical course in the curriculum, and its purpose is entirely non-technical: to make the decision rules from Course 0 executable. A kill criterion you cannot evaluate is a wish. A concentration limit you cannot measure is a slogan.

The design constraint throughout is ownership. Measurement that lives only inside a third-party platform is measurement you can lose, and the whole premise of v2 is that intermediaries change their behaviour without asking you. So the data layer is portable by construction, the storage is yours, and the export is a first-class feature rather than an afterthought.

9.1 What is a data layer you actually own?

What is it?

A small, explicit schema of events and metrics that you define, store somewhere you control, and can export in full — independent of any analytics vendor's naming, retention or continued existence.

Why it matters

Vendors rename metrics, change retention windows, sunset products and reprocess history. If your twelve-month trend lives only in a platform's interface, then the platform owns the evidence for your own kill criteria. Defining your own schema costs a day and makes every later migration a data-transfer problem rather than a loss.

How to do it

1. List the decisions you need to make first — prune, kill, invest, diversify — then define only the events those decisions require. Instrument backwards from decisions, never forwards from curiosity.
2. Name events in your own stable vocabulary and write them down. tool_completed means the same thing in month one and month twenty-four.
3. Record a minimal, non-identifying payload per event: what happened, on which asset, at what time, and the coarse source category. Nothing more.
4. Store a durable copy you control — this site uses IndexedDB in the visitor's own browser for tool state, and a monthly manual roll-up for portfolio metrics.
5. Make export the primary interface. If you cannot produce a JSON or CSV of everything in one action, you do not own it.
6. Write a data contract: every field, its meaning, its unit, its source, and who updates it. Undocumented metrics silently change meaning.

Event schema & data contract
  EVENTS (name, when it fires, payload, why it exists → which decision)
    asset_view          asset id, coarse source, date        → pruning gate
    tool_started        tool id, date                        → tool value
    tool_completed      tool id, date                        → durable conversion
    output_kept         tool id, method (copy/export/link)   → 8.2 keepable output
    subscribe           source asset, date                   → owned audience
    commercial_click    asset id, partner, date              → concentration limits
  CONTRACT: field | meaning | unit | source system | owner | update cadence
  EXPORT: single action → full JSON/CSV. Verified working on ___ (date)

Tools needed

Your component library, js/db.js (IndexedDB) or equivalent, and a spreadsheet for monthly roll-ups.

Lab 9.1

Write the event schema for one site, starting from the four decisions you must make at the Month 3 gate. Implement the events, then perform a full export and re-import to prove the round trip works. If the re-import fails, you do not have a data layer.

Artifact

Event schema & data contract, plus a verified export→import round-trip with the date

Common mistakes

• Instrumenting everything measurable, producing a mass of events that inform no decision and cost real maintenance.
• Adopting a vendor's metric names, so a vendor change breaks your history.
• Never testing the import path, and discovering at migration time that the export is unusable.

Pro astuces

• Write the decision beside every event in the schema. Any event without a decision gets deleted — that rule alone keeps the layer small enough to maintain.
• A monthly manual roll-up of six numbers is more durable than an elaborate automated pipeline you abandon in month five.

9.2 How do you do analytics privacy-first?

What is it?

Collecting the minimum data needed for your decisions, with a lawful basis, honest disclosure, and no tracking that you would be embarrassed to describe on your own privacy page.

Why it matters

Three reasons, in order. It is a legal obligation whose specifics depend on your jurisdiction and audience. It is a trust asset — "we do not track you across the web" is a genuine differentiator in every one of the Phase 1 niches. And it is cheaper: less data means less liability, less storage and fewer consent obstacles in front of your content.

How to do it

1. Start from the six metrics in 9.3 and collect nothing that does not feed them.
2. Prefer aggregate, cookieless measurement where it answers the question. Most portfolio decisions need trends, not individuals.
3. Where consent is required, ask honestly and make refusal genuinely equivalent. Do not degrade the page to punish a refusal.
4. Keep tool data local to the visitor's device wherever the tool allows it — as this site does — and say so on the page.
5. Document retention: what is kept, for how long, and how it is deleted. Then actually delete it.
6. Write the privacy page from what the code really does, not from a template. Then re-audit it whenever you add a script.

Tools needed

A privacy-respecting analytics option, your consent mechanism, your privacy page, and a script inventory.

Lab 9.2

Inventory every third-party script on one site and, for each, write what data it can access and which of your six metrics it serves. Remove every script that fails that test. Then rewrite the privacy page to match what remains.

Artifact

Analytics & consent record
  Script / vendor | Data it can access | Metric it serves | Lawful basis | Consent required? | Kept? Y/N
  Scripts removed: ___
  Retention: what | how long | deletion method | last deletion run
  Privacy page rewritten from actual behaviour on ___ (date)

Common mistakes

• A template privacy policy describing data practices the site does not have — which is itself a misrepresentation.
• Consent banners that make refusal harder than acceptance.
• Keeping an analytics vendor "in case we need it later", accepting the liability for no decision benefit.

Pro astuces

• Deleting scripts usually improves Core Web Vitals and reduces consent friction at the same time. It is the rare change with no trade-off.
• "Your inputs never leave your browser" is a strong, checkable claim in finance and security niches. If it is true, put it next to the tool.

9.3 Which six numbers actually decide things?

What is it?

The minimum metric set that drives every decision rule in this curriculum: branded share, owned-audience engaged size, revenue per visit, contribution margin, concentration ratios, and cost-per-asset maintenance load.

Why it matters

Dashboards fail by containing too much. Six numbers, each attached to a specific rule and a specific threshold, will actually be looked at monthly — and each one already has a rule waiting for it from an earlier course. Anything that is interesting but attached to no rule is a distraction.

How to do it

1. Baseline all six with a date. An undated metric cannot support a trend claim.
2. Attach each to its existing rule and threshold, so the number triggers an action rather than a discussion.
3. Measure monthly on the same day, using the same definition, and never silently change a definition.
4. Track direction over three months. Single-month moves are usually noise, as Course 6.4 established.
5. Review all six together at each pruning gate. They interact: rising revenue per visit with falling branded share is a very different situation from both rising.
6. If a metric has not informed a decision in six months, delete it and free the collection cost.

Tools needed

Revenue & RPM calculator, unit economics model, your analytics and your email platform.

Lab 9.3

Baseline all six metrics for every site in the portfolio, on one day, with definitions written down. Then state which rule each number currently triggers, if any. Any metric triggering nothing is a candidate for deletion.

Artifact

Decision metric baseline
  Metric | Definition (verbatim) | Value | Date | Rule it serves | Threshold | Status
  Metrics collected but serving no rule: ___ (delete these)

Common mistakes

• Reporting raw list size instead of engaged subscribers, which flatters the owned-audience number most.
• Changing a definition mid-year, destroying the comparability the baseline existed for.
• Watching total revenue instead of contribution margin, so an unprofitable site looks healthy.

Pro astuces

• Put the threshold next to the value on the dashboard. A number without its threshold invites negotiation with yourself.
• Six is a ceiling, not a target. If you can drive decisions with four, use four.

9.4 How do you instrument AI surfaces properly?

What is it?

Turning the manual Course 6.4 routine into a repeatable, logged pipeline: the frozen prompt set, a fixed schedule, structured scoring, recorded raw responses, and the noise floor carried alongside every reading.

Why it matters

Manual measurement decays. By month four the run is late, the scoring drifts, and the series becomes unusable exactly when you need it for a pruning decision. Instrumenting it — even crudely — is what makes the AI-surface number trustworthy enough to spend money on.

How to do it

1. Store the frozen prompt set as data, not as a document, with a version identifier. If it ever changes, the version changes and the series restarts.
2. Log every run completely: prompt id, engine, date, locale, account state, the full raw response, and the 0–3 score.
3. Keep the raw responses. Scores are interpretations; responses are evidence, and you will want to re-score them later.
4. Re-measure the noise floor quarterly by repeating a subset within one session. It changes as engines change.
5. Report only three-month directional movement, with the noise floor printed next to it.
6. Respect the terms of every service you query, and never present the results as findings about the engines. This measures your own visibility.

Tools needed

AI-surface tracking sheet, official APIs where available, and a recurring monthly calendar entry.

Lab 9.4

Convert your Course 6.4 prompt set into versioned structured data, run one full logged cycle recording raw responses, and re-measure the noise floor. Then write the one-paragraph interpretation rule you will apply to future readings — before you have a reading you want to over-interpret.

Artifact

Surface measurement pipeline
  Prompt set version | Frozen on | Prompt count
  Run log: prompt id | engine | date | locale | account state | raw response stored? | score 0-3
  Noise floor: measured ___ on ___ | re-measure due ___
  Interpretation rule: report movement only when 3-month change exceeds noise floor
  Terms-of-service check: service | terms reviewed on | automation permitted? Y/N

Common mistakes

• Storing scores but discarding raw responses, making re-scoring and auditing impossible.
• Automating first and reading the terms of service afterwards.
• Reporting month-to-month movement smaller than the measured noise floor.

Pro astuces

• Write the interpretation rule before the data exists. It is the only reliable defence against reading a story into noise.
• Stored raw responses are themselves original data — they can legitimately support the Course 10 data campaign, provided you describe the method and limits honestly.

9.5 Should you publish your data as an API?

What is it?

Exposing your original dataset through a documented, versioned, rate-limited endpoint with clear licensing and attribution terms — turning a static dataset into infrastructure other people build on.

Why it matters

An API is the strongest attribution mechanism available to a small publisher: every consumer has a contractual reason to name you, and integrations create switching costs no algorithm change can remove. It also directly serves Course 10, because a usable endpoint is a far better story than a PDF report.

How to do it

1. Only publish data you genuinely own and collected. Re-publishing someone else's data through an API is a licensing problem, not a growth tactic.
2. Start with one static, versioned JSON file on a CDN. It is an API, it costs nothing, and it cannot fall over.
3. Version from day one — /v1/ — and never change a field's meaning within a version.
4. Document the schema, the update cadence, the collection method and the limitations in the same place as the endpoint.
5. State the licence and the attribution requirement explicitly, and keep both machine-readable.
6. Publish a changelog and a deprecation policy. Consumers only build on data they can trust not to vanish silently.

API spec & terms
  Endpoint         /api/v1/<dataset>.json      (static file on CDN is fine)
  Schema           field | type | unit | nullable | meaning
  Cadence          updated ___; next update ___
  Method & limits  how collected, sample, what it cannot show
  Licence          ___ ; attribution required: "Source: <brand>, <url>"
  Versioning       /v1/ frozen; breaking changes → /v2/
  Deprecation      minimum ___ days notice, published in changelog
  Ownership        data collected by us? Y/N  (if N, do not publish)

Tools needed

Your Cloudflare static hosting from Course 4, your dataset, and legal advice on licensing.

Lab 9.5

Publish one dataset as a versioned static JSON endpoint with a documentation page covering schema, method, limitations, licence and attribution. Then consume your own endpoint from a separate page to prove it works as infrastructure rather than as a download.

Artifact

API spec & terms, the live versioned endpoint, and a working consumer of your own API

Common mistakes

• Publishing data you licensed from someone else, or scraped, without the right to redistribute it.
• No versioning, so the first schema change breaks every consumer and destroys the trust that created the value.
• Documenting the schema but not the method, which makes the data unciteable for exactly the audience you wanted.

Pro astuces

• A static versioned JSON file on a CDN handles enormous traffic for effectively nothing and never has an outage you caused.
• Requiring attribution in the licence converts every integration into a durable, contractual citation — the Course 6 objective achieved by other means.

9.6 What belongs on the one-page dashboard?

What is it?

A single page showing the six metrics per site, each with its threshold and status, the concentration ratios, the maintenance load against capacity, and the next pruning gate date. Nothing else.

Why it matters

The purpose of the dashboard is to make the decision rules unavoidable. A dashboard with forty charts gets skimmed; one page where three cards are amber gets acted on. This is the operating instrument for Course 12's pruning gates, so it has to be readable in two minutes on a Monday morning.

How to do it

1. One row per site, one column per metric, threshold beside every value. No metric appears without its threshold.
2. Show status, not just value: within limits, warning, or breach. Colour is a shortcut, not the information.
3. Put the four concentration ratios on the same page. They are portfolio-level and invisible from any single site's view.
4. Show maintenance load against capacity, because that is the constraint that silently caps everything else.
5. Print the next pruning gate date and what happens at it, so the gate cannot be quietly skipped.
6. Update on a fixed monthly date and archive each snapshot. The archive is your trend, and it is also the evidence a buyer or partner will ask for.

One-page dashboard layout
  PER SITE (one row each)
    Branded share ___ (≥ target?) | Engaged audience ___ | Revenue/visit ___
    Contribution margin ___ (kill threshold ___) | Maintenance hrs ___ / capacity ___
  PORTFOLIO
    Affiliate ___% (≤30) | Largest site ___% (≤50) | Top traffic source ___% (≤60) | Top sponsor ___% (≤25)
  GATES
    Next pruning gate: ___ (Month 3 / 6 / 9) | Actions triggered if breached: ___
  Snapshot archived: ___ (monthly, same date)

Tools needed

Your metric baseline, the pruning scorecard, the risk register and a monthly calendar entry.

Lab 9.6

Build the dashboard for the whole portfolio on one page. Then hand it to someone who does not work on the project and ask them to tell you what needs attention. If they cannot answer in two minutes, simplify it.

Artifact

One-page dashboard, first snapshot archived, plus the two-minute comprehension test result

Common mistakes

• Building a beautiful dashboard nobody opens, because it shows values without thresholds and therefore demands interpretation.
• Omitting the concentration ratios, which are the only portfolio-level early warning you have.
• Failing to archive snapshots, so a year later there is no trend to look at.

Pro astuces

• The two-minute outsider test is the real specification. If a stranger cannot identify the problem, you will not either when you are busy.
• Archived monthly snapshots are the artifact that makes a portfolio saleable. Buyers pay for evidence, not for assertions.

Scenario assessment: the analytics migration that erases the evidence

Your analytics vendor announces that the current product is being sunset in 90 days, historical data before this year will not be migrated, and two metric definitions are changing. Your Month 6 pruning gate is in five weeks, and the decision to kill or keep the SMB Finance site depends on a branded-share trend that exists only inside that vendor. Meanwhile a partner offers a "free" enhanced analytics suite in exchange for embedding their script on all three sites and sharing visitor-level data. The AI-surface run has not been done for two months because it was manual, and the operator wants to automate it this week by scripting queries against three assistants.

Final project: a measurement stack you own

Build the instrumentation that makes every decision rule in this curriculum executable — and prove you can take it with you.

Deliverable
1. Event schema & data contract, every event mapped to the decision it serves
2. A verified export → import round trip, dated
3. Script inventory with every non-essential third-party script removed, and a privacy page rewritten from actual behaviour
4. All six decision metrics baselined per site, with definitions, dates and thresholds
5. AI-surface pipeline: versioned frozen prompt set, full run log with raw responses, current noise floor, terms-of-service check
6. One dataset published as a versioned API with schema, method, limitations, licence and attribution — plus a working consumer of it
7. One-page dashboard covering all sites, the four concentration ratios, maintenance load vs capacity, and the next gate date; first snapshot archived
8. The two-minute outsider comprehension test, passed

Pass standard: a stranger can read your dashboard in two minutes and correctly name what needs attention, and you can reproduce every number on it from data you own.

Course 9 checklist

• Every event mapped to a specific decision it serves
• Event names written in your own stable vocabulary, documented
• Data contract records field, meaning, unit, source, owner, cadence
• Full export → import round trip tested and dated
• Third-party script inventory complete; non-essential scripts removed
• Lawful basis recorded per data collection, with advice taken
• Refusing consent does not degrade the page
• Tool data kept local to the device where possible, and stated on the page
• Retention documented, and deletion actually run
• Privacy page rewritten from actual code behaviour
• All six decision metrics baselined with dates and written definitions
• Engaged subscribers used, not raw list size
• Contribution margin used, not gross revenue
• Prompt set stored as versioned data and frozen
• Raw AI-surface responses stored, not just scores
• Noise floor re-measured, and interpretation rule written in advance
• Terms of service checked before any automated querying
• API versioned from day one; schema, method and limitations documented
• Licence and attribution requirement stated; redistribution rights confirmed
• Dashboard shows a threshold beside every value
• Four concentration ratios present on the dashboard
• Monthly snapshot archived; two-minute outsider test passed

AI-agent prompt for Course 9

Act as a measurement architect for an independent publisher running three monetised sites. The operating principle is that measurement must be OWNED: portable, exportable, and independent of any single vendor.

I will paste: my current analytics setup, my third-party scripts, my decision rules and thresholds, my datasets, and my AI-surface results if any.

Do this:
1. Work BACKWARDS from decisions. List the decisions I must make (prune, kill, invest, diversify), then define the minimum event set those decisions require. Delete any event that serves no decision and say so explicitly.
2. Write the event schema and a data contract: field, meaning, unit, source, owner, update cadence. Use my own stable vocabulary, never a vendor's metric names.
3. Audit every third-party script: what data it can access, which of my six decision metrics it serves, and whether it should be removed. Flag any script whose data collection contradicts a privacy claim I have published.
4. Baseline the six decision metrics: branded share of arrivals, engaged owned audience (active in 30 days, NOT list size), revenue per visit, contribution margin (revenue minus fully loaded cost), the four concentration ratios, and maintenance load vs capacity. Attach each to the rule and threshold it serves.
5. Design the AI-surface pipeline: versioned frozen prompt set, run log storing RAW responses as well as scores, noise-floor procedure, and an interpretation rule written BEFORE data exists. Tell me to check each service's terms of service before automating, and do not help me build automation that breaches them.
6. Specify one dataset as a versioned public API: endpoint, schema, cadence, method, limitations, licence, attribution requirement, deprecation policy. First confirm I have the right to redistribute the data; if I do not, refuse.
7. Lay out a ONE-PAGE dashboard: a threshold beside every value, the four concentration ratios, maintenance load vs capacity, and the next pruning gate date. Nothing else.

Rules:
- Never propose collecting data that serves no listed decision.
- Never propose visitor-level data sharing with a partner in exchange for features.
- Treat privacy, consent, retention and cross-border transfer as legal questions: state that requirements vary by jurisdiction and that you are not giving legal advice.
- Do not invent benchmarks for any metric. If I have not measured it, mark it UNMEASURED and give me the measurement procedure.
- Never report movement smaller than the measured noise floor; if I have no noise floor, say the reading cannot be interpreted yet.
- Every metric definition must be written verbatim so it cannot drift.
Output as tables, with removals, refusals and UNMEASURED items called out in plain text above the tables.

Primary sources

• Search Console documentation — what impressions, clicks and position actually mean before you build trends on them.
• Web Vitals — field versus lab measurement, which matters when you compare your own numbers month to month.
• FTC endorsement, influencer and review guidance — relevant when you publish data or metrics that support a commercial claim.
• IndexedDB API (MDN) — the local-first storage used by every tool on this site.

Vendor metric definitions, retention windows and terms of service change without notice. Every statement here is a dated claim with a 90-day expiry — log it and re-verify.