The dataset is rebuilt weekly from live job boards. When a number moves because the pipeline was wrong rather than because the market moved, it belongs here.
Every revision that changed a published number. Newest first. Dates are the day the corrected data went live. Current build: 266 companies, 18,758 de-duplicated postings, 8,049 with a published salary, median $200,000.
content
Company pages answered 'what does X pay' — nothing answered 'is remote cheaper'
What was wrong
Not a break — a hole. All 165 crawlable pages answered one shape of question (what does this company pay) which means the site only ever earned the company-name queries. The comparative questions people actually type into a search box had no page to land on, and no internal links pointed anywhere except back to the company index.
What changed
Built a four-page deep-dive cluster under /topics/ from the same CSV, each page printing its own sample size: remote pay vs office pay (n=2,254 vs 5,795, medians $190,781 / $204,000, with the per-family split showing security +9.8% and sales +9.4% but data/ML -17.6%); role families ranked (data/ML $237,500 down to support/ops $162,000, spread $75,500); title words vs level ('senior' $185,791 vs unlabelled $193,500 — the inversion is a labelling problem, not a data bug); location bands (SF $252,500; multi-metro postings $345,000 vs $195,000 single-location). Sitemap 165 → 170 URLs, hub linked from the homepage and the dataset page, llms.txt given the same entry points.
measure
before
after
crawlable urls
165
170
deep-dive pages
0
4
pages answering a comparative query
0
5
infrastructure
The homepage kept advertising yesterday's data while every other page was current
What was wrong
The line "last measured …" (and the pricing note under it) on /index.html was written by hand. Every refresh rebuilt data/, /freshness.html, /salary-data.html, /llms.txt and the sitemap, and all of them moved to the new date — but the first page a visitor reads stayed two days behind. The one page whose whole job is to say how fresh this data is was the only page that could go stale, and a reader comparing the homepage against the dataset could only conclude the numbers were abandoned.
What changed
tools/stamp_homepage.py reads generated_at from site/data/freshness.json and rewrites exactly those two phrases; it is idempotent and refuses to touch the dated entries on /changes.html, where an older date is the point. Wired into scripts/refresh_dataset.ps1 as step 9b/13, before the deploy step, so the two can no longer drift apart.
measure
before
after
days the homepage lagged the dataset
2
0
data
Weekly snapshot: 8,049 postings with a stated pay range, 42.9% of everything scanned
What was wrong
Nothing broke this week — this is the scheduled recompute. Recording it anyway, because a dashboard that only ever logs failures teaches readers that silence means success, and there is no way to tell a quiet week from a stalled pipeline.
What changed
19,039 rows scraped across 259 company boards, 18,758 after de-duplication on company + title + location, 8,049 of them stating an annual range (42.9%). Median $200,000; p25 $150,000, p75 $253,000, p90 $323,000. 152 company pages rebuilt, 154 /pay/ URLs in the sitemap, 165 URLs submitted through IndexNow (HTTP 200). The open-data mirror got release snapshot-2026-10-26 with the same CSV.
measure
before
after
postings with a salary
8,004
8,049
median salary
$200,000
$200,000
classification
Role families: a single-tier classifier was sorting 21.5% of rows into "other"
What was wrong
The role-family bucket came from one rule list that tested `engineer` first. Sales Engineer, Solutions Engineer and Support Engineer were therefore counted as engineering, and the bare word `data` pulled data-center technicians into data/ML.
What changed
Four tiers: disambiguation, then specific titles, then generic words, then broad engineering words. 21 unit tests cover the traps (Technical Account Manager must not become sales; Senior Accountant must not become sales).
measure
before
after
engineering share
39.7%
27.4%
other share
21.5%
8.5%
remote vs on-site gap, engineering
-11.5%
-6.2%
statistics
Per-company publication rate divided two different things
What was wrong
The rate was computed as paid rows in the CSV divided by open postings on the board. Those are different populations, and the CSV is sampled, so one company reported a 105.9% publication rate.
What changed
Numerator and denominator now come from the same de-duplicated population. Companies with fewer than 25 open postings are left out of the table instead of producing a meaningless percentage.
measure
before
after
companies in the table
n/a
146
companies with 40+ postings publishing no salary
n/a
11
data
Currency conversion used a hardcoded table of ten currencies
What was wrong
Anything outside USD/EUR/GBP/CNY and six more came back as "no salary". Filtering by salary then deleted those postings, so the filter reported "this job has no pay" when the truth was "we could not convert it".
What changed
Live rates for 166 currencies from open.er-api.com, cached 24 hours, with the snapshot timestamp stored in every row and in the run summary. Unconvertible currencies stay null instead of being guessed.
measure
before
after
postings with a salary
7,984
8,004
rows hidden by the salary filter
~1,170
0
dates
Greenhouse publish date was actually the last-edit date
What was wrong
The pipeline read updated_at because created_at does not exist in Greenhouse responses. A 2024 evergreen role edited for a typo looked like it was posted today, which also broke the "only new jobs" mode.
What changed
Read first_published, fall back to created_at, then updated_at. Documented per board on the freshness page, because each board exposes a different date field.
measure
before
after
postings older than 180 days, Greenhouse
0.0%
14.5%
median posting age, all boards
22 days
51 days
dedupe
De-duplication ignored location and deleted 17% of real postings
What was wrong
Rows were merged on company + title. The same title in Dublin, Singapore and Chicago is three different jobs, so about 17% of the dataset disappeared. A normal de-duplication rate on a multi-board scrape is 1-3%.
What changed
The key is company + normalized title + normalized location. Hyphens are normalized too, otherwise US-Chicago and us chicago stay apart.
measure
before
after
share removed by de-duplication
17.4%
1.4%
postings with a salary
6,833
7,984
median salary
$204,750
$200,000
infrastructure
Site moved off a host that disallowed crawling
What was wrong
The static host served a platform-injected robots.txt with Disallow: / on the bare path while our own file appeared under any query string. Every URL answered 200, so nothing looked wrong, but crawlers that respect robots.txt never entered.
What changed
Moved to GitHub Pages under a path that serves our own robots.txt, and added an automated crawlability check that compares the bare path against a cache-busted one and fails if a sitemap entry points at another host.
content
Four salary deep dives published under /topics/
What was wrong
The site only answered 'what does company X pay'. Questions like 'does remote pay less' or 'how much does seniority matter' had no page, so every visit was a dead end for a reader who did not already have a company in mind.
What changed
Built /topics/ with an index and four analysis pages computed from the same published ranges (remote vs office, role family, job-title seniority, location bands). Each page states its own sample size and links back to the underlying data and to the other dives.
measure
before
after
seo
Topic pages wired into the site graph and submitted
What was wrong
New pages with no inbound links are invisible: crawlers only followed company pages, and the sitemap did not list /topics/ yet.
What changed
Linked the dives from the data page, the changes page and llms.txt, added them to the sitemap, and pushed the URLs through IndexNow.
measure
before
after
distribution
Two dev.to write-ups syndicate the deep dives
What was wrong
Every argument the data supports was trapped on our own domain: a reader who never lands on the site never sees the analysis, and the deep dives had no external discussion pointing back at them.
What changed
Rewrote two findings as long-form posts for an engineering audience on dev.to (canonical points home), then linked those posts back from the dive pages so the two sides reference each other instead of one-way.