#Letting AI agents run a large website
Context
As a side project, I have been running a specialized website for the past few years. It now serves more than 100K pages, and has reached significant traffic. The promise of this website is to provide structured and up-to-date data on a specific topic. In order to do this, the content is updated daily, sometimes multiple times in the same day when the news justifies it.
This website serves two purposes for me: have fun by going deep on a topic of interest, and experiment with state-of-the-art AI techniques on a project where I'm the only decision maker.
Today, my daily work on this website takes about 5 minutes: I review and merge a GitHub pull request with my morning coffee. This PR is quite substantial: it usually contains a couple of news articles with their illustrations, several updates to the fleet inventories, based on a broad web search of the previous day's news from reputable sources, and new entries in a tracker of geopolitical events.
Reaching this level of automation wasn't a single-step process. It's the result of three years of progressive use of AI capabilities. Today, the optimisation goal I have given to my agents for the morning PR is simple: "I should almost never have to refuse a PR that you propose".
2023 to 2026: from manual with AI help, to agent-run
When I launched the website in August 2023, I chose Django, a Python framework that I had never used before. GPT-4 was extremely helpful to get me started, and over a weekend I had a first version up and running.
In 2023 and 2024, coding and writing the content were still very much manual. AI was helping write the code within an IDE, and each file was reviewed and tested. For the content, web search and page fetching were not reliable, and many URLs were hallucinated, so I still had to code scripts to get the raw content first, then process it with LLM assistance.
In early 2025, coding agents were able to complete more complex code modifications, but I was still working inside VS Code. For the content, LLMs were now able to generate competent descriptions, with much less fluff or made-up content, if provided with a reliable source such as Wikipedia or a news article. You still had to point to the relevant source URLs.
Things started to accelerate in January 2026, when I adopted Skills and Claude Code. I used Skills to write down, for each type of content update, the full procedure the agent had to follow. For example, to add a new event, the Skill described how to search the web for the latest events, read reliable news sources, check the database for events already present, generate the description and the relevant fields, ask for my approval, then insert the new events. This allowed me to create new content much faster, with very good accuracy.
On the technical side, I walked away from VS Code literally overnight and started working directly in Claude Code. I no longer looked much at the code, but rather asked for increasingly complex tasks: structural database changes, full-fledged new features, large design changes. My technical productivity exploded.
In the first half of 2026, I created many Skills, to cover all categories of new content, but also to update and review the quality of existing content at scale. Still, I reviewed batch edits quite thoroughly before any update to the database, and I was the one who caught errors. I was not confident enough to let agents run autonomously.
Acceleration in the last six months
Until June, I still ran one Skill per task, and launched them manually. They were now reliable and efficient, running the full process end to end from the web search to the database insertion after approval.
Since the reliability had improved across all Skills (news searching, content writing, content quality review), I no longer needed to check each Skill independently. So I bundled them all under a meta-Skill that I ran every day. The orchestrating agent handed me its result as one consolidated approval request.
After running these bundled automated tasks manually for a few weeks, I realized that I had become a bottleneck rather than a useful guardrail.
I had already removed the Django admin, which I had not used for months. The last step was to schedule this routine in the cloud, running at night, and to present me with a GitHub PR that I review in the morning.
Since mid-September, I have only refused 1 PR out of 45.
The detailed setup
Everything runs as Claude Code routines on claude.ai, so my laptop can stay closed. There are two main routines:
- The large daily update starts at 3 a.m. UTC.
- A second routine runs at 2 p.m. UTC and watches for traffic spikes: when a news story sends readers to an unusual, low-traffic page, it updates that page's content while the story lasts.
Each routine has a short prompt. It tells the agent to read the daily-update skill and follow it, with a few hard rules: work only on a claude/daily-<date> branch, never push to master, never merge, never send an email. The real procedure lives in the skill, inside the repo, so I change it with a normal commit.
The run starts from a fresh clone. The database is SQLite, versioned in git as a text dump. The agent rebuilds the database from the dump, works on it, then dumps it again. The agent makes one commit per topic and pushes after each one, so if the run crashes in the middle, the work already done is kept.
Then the orchestrator (Opus 5.5) works on three kinds of content:
- Events: it searches the news of the last few days, compares it with what is already in the database, and inserts only new and sourced items. Locations come from a geocoding API, never from the model.
- Official data: daily official publications, fetched and parsed by deterministic scripts, without an LLM. One of these official sites refuses connections from the cloud sandbox, so a GitHub Action fetches it at 2:30 a.m. and pushes it to a branch, where the agent reads it.
- News: it scans topics, ranks them by freshness, search demand and whether the site's data adds something useful, and keeps 1 to 3 of the candidates.
The news articles are not written by the Opus orchestrator. It writes a brief for a sub-agent running Fable 5.1, which I prefer for its prose. The brief contains all the figures the article may use, already verified, in a data block, with one rule: no number may come from anywhere else. It also gives the list of allowed links and the caveats that must stay in the text. Then the orchestrator checks each figure against the data block. Illustrations are generated with Gemini.
At the end, the agent opens the pull request through the GitHub API and sends a notification to my phone. The PR description is a summary I can read in one minute: what landed, what needs a decision, what went wrong, and the list of commits. No "tests passed", no justification of the process. I gradually removed the useless verbose parts that LLMs tend to write, as if they want to prove that they work hard.
Before I read it, the PR goes through CI. The tests, and a sourcing test over all articles, block the deploy if they fail. In parallel, another LLM reviews the diff for privacy and security risks. It is Sonnet 5.5 called with claude -p, with no tools and a fixed JSON output, because the diff contains text from the web and must be treated as untrusted. A simple pattern scan also looks for things like a personal email or a home directory path.
When I merge the PR, the merge runs the tests, the deploy and the CDN purge.
Effect of "I should never have to refuse"
When you tell an agent to be useful, it tries to do as much as possible. When you tell it that a refused PR is a failure, it does less, but it does it more correctly. This change of goal led to a few concrete rules in the skills.
When in doubt, leave it out. The routine only executes what an interactive session would have marked as "recommended". Borderline items get one line each in the PR description, so I can still ask for them. I prefer false negatives to false positives: a missing item costs me one line to read, while a wrong insert costs a revert.
Nothing should be invented. Coordinates come from the geocoding API or stay empty. Vague sources like "analysts say" are banned: either the agent names the source, or it doesn't include the claim.
Official documents before press articles. Each news article cites the primary document (typically a statement from an official website) before the press coverage, and should never have more press links than first-hand documents.
None of this advice is new, but what changed is how I judge the agent: by how often I have to say no.
What I still do
My role today is close to that of an editorial director. I decide what the site covers, define the editorial line, and set the standard for quality. Every morning, I read one PR and merge it. Sometimes I leave a comment, so that the agent adjusts its proposition with my guidance.
When an agent makes a mistake, I don't just fix the entry in the database. I update a rule, add a test or a check, so that the same mistake does not come back. However, there's a risk with this: if every problem becomes a rule, the skills could eventually overfit to past errors, and in the long run the agents would lose some of their openness to new cases. So every few months, I review the rules, and I drop the ones that only say how to do the work: frontier models now understand the goal very well, and often find the right way to reach it without step-by-step instructions.
The real work is not in the prompts that write the articles. It is in the guidelines I added one by one, until I almost never had to refuse a PR. Most of them describe what a good result looks like, and what I will refuse. I could not have written them on day one: they came from a lot of hands-on experiments.
But in the end, I have achieved alone what would have needed a small team a few years ago: a full-stack developer, a data engineer, a couple of analysts, a writer, an illustrator, and possibly part time devops and SEO expert.