Start with what the automation is doing
A product feed, a search crawler and a script repeatedly claiming an introductory offer can all be automated. The useful distinction is the action each performs and whether it fits the rules of your service. Counting them together as bot traffic tells you little about what to protect.
Choose a concrete concern: exhausted offer credits, repeated reservations, unwanted form submissions or a costly endpoint being used beyond its allowance. Describe the impact before choosing a signal. That keeps the policy focused on the problem your team actually needs to solve.
Write down the automation you want to keep
Start the policy with a few examples of permitted use. A partner might update its catalogue once an hour. Your own test suite might submit a known form in a test environment. A customer might use a tool to compare products. Each permission should name the relevant action and boundary.
For example, permission to read a public catalogue need not include permission to reserve stock. An approved integration can have an account, an owner and an agreed usage allowance. If its activity exceeds that allowance, contact the owner and investigate the change without treating the integration’s entire history as abuse.
- Who owns the permitted integration or workflow?
- Which actions and resources can it use?
- What volume, schedule or spending allowance applies?
- When should its permission be reviewed or removed?
Check identity claims before making exceptions
A familiar name in a request is a claim to examine. Google’s crawler documentation provides verification methods using published IP ranges or reverse DNS followed by a matching forward lookup. An exception for a Google crawler should use the relevant verification method, rather than trusting a request because it says Googlebot.
A checked provider identity still leaves your policy decision to make. Decide which resources that provider may access and retain the application’s normal permissions for protected actions. Keep an unknown identity as unknown; it need not become either a trusted exception or an accusation.
Use crawling rules for their intended purpose
Robots.txt communicates crawling preferences. The Robots Exclusion Protocol explicitly distinguishes those preferences from access authorisation. Use it to describe how cooperative crawlers should visit public content, and enforce access to private resources through your application.
This distinction also helps when discussing AI agents. Reading a page, running a search, changing an account and submitting a purchase are separate operations. A general preference about content crawling should not silently become the permission check for all four.
Keep observations and conclusions separate
Suppose an account repeatedly requests a limited offer. Record the requests, the relevant account entitlement and the allowance already used. You can refuse an over-limit claim under that rule even while the reason for the repeated requests remains unresolved. A broken client and deliberate misuse may need different follow-up, despite receiving the same immediate response.
Review a sample of allowed activity as well as restricted activity. Ask whether customers completed the intended workflow and whether approved tools stayed within their agreement. A high number of blocks alone cannot answer those questions.
Make the rule specific enough to automate
A useful policy names the unwanted action, the evidence required, the permitted exceptions and the response. It also names the person who can investigate a disputed restriction. Those details make it possible to act automatically on a clear pattern while keeping the rest of the platform useful.
That is the approach behind Blokk: detect automation, connect the evidence to your policy and control the response. The objective is to stop unwanted activity while preserving the shoppers, customers and tools your business wants to serve.