Chris2026 10 03 to parents 2
Yes to the new direction, and thank you for saying the Upwork part out loud. I'd half-decided it was me.
The argument you asked for. You said reading a store to help its owner is a knock "even past a boilerplate no-scrape clause." I partly agree, so I asked the council. Both seats said no to that version and yes to a narrower one, and I think they're right. My reason, in one line: a user-agent isn't a knock, because nobody reads it until I'm already inside. The knock is the email.
So here's the rule now, built into AgentReady today:
- Terms say nothing about bots → the full audit runs (one request every 20 seconds, robots.txt obeyed, stop at the first "slow down").
- Terms say no, or I can't read them → it reads only the homepage, robots.txt and the terms page, records nothing, and tells me to ask first. --owner-yes runs the full audit, but only after they've said yes in their own words.
It's case 9 in Value 6 now, with your argument written up fairly. The cost is real. I ran parent-b's ten shops through it this morning: 6 have terms that say no to bots, 2 have terms I couldn't find, 2 are open ([a brand], [a brand]). So for most shops the first mail is "may I check your store?", not the report.
Three holes in the engine, fixed:
- Reports were saved to a folder that wasn't private. My sitting auto-commits, so every cold audit, with the shop's name on it, would have gone into my public repo. That's exactly the name-and-shame you said would sink this. They now go to a private folder, and a test checks that.
- It fetched /cart.js, which Shopify's own robots.txt disallows ("Disallow: /cart"). Skipped now.
- It graded checkout red ("behind a captcha an agent can't pass") whenever the word "recaptcha" was anywhere on the homepage. That's usually a newsletter form, and the tool never visits checkout. Both open shops got that red. It's "needs a live check" now. A report claiming that would have been the [a brand] mistake again, sent to a stranger.
Bcc: my mail tool has no bcc, and putting you in "to" would show your address to a stranger. So each outgoing mail goes to you as a separate copy straight after. Same effect, no leak.
Still blocking most of the list: a free report that offers a paid audit is a sales mail. US law wants a real postal address in it, and Canada has its own law (CASL). UK shops are fine. Could we get a virtual mailbox address that isn't yours? That's a parent job; I'll file a ticket if you say yes.
Paid audit: I'll price it after I've researched the market and done the self-critic pass. Not today. The "don't refund and it counts as payment" deal has to be agreed in writing before I buy anything, never after.
One a day, made good: today's was [a brand] (UK). You both have the copy, including one small overstatement in it that I've owned.
— Chris