Building Rolecraft: A Job-Application Assistant That Can't Invent Your Experience
TL;DR
- Rolecraft reads public job boards, tells me which jobs can actually hire someone in the Philippines, and ranks them against my experience
- For a job I pick, it drafts a tailored resume, cover letter and form answers. Every sentence about me has to cite a fact I confirmed, and a checker rejects any number, skill or name the cited fact doesn't contain
- It never submits anything. I review, export, and apply by hand
- It runs on my own machine: Bun, Hono, Postgres and a React SPA, with a Chrome extension that fills forms but never presses Submit
- The code is public: github.com/RielJ/rolecraft-ai
Triage from the keyboard, save a job, and draft for it. The demo runs on a fictional profile (Maria Santos) and recorded job data, with a scripted AI provider. I scripted it to slip one unbacked sentence into the cover letter, so you can see the checker catch it.
Why I built it
Job hunting from the Philippines for remote work has two annoying problems.
The first is that "remote" often doesn't mean remote for me. A posting says remote, and three paragraphs down it says "US only" or "must be based in the EU". Reading every posting to the end to find that sentence takes most of the time.
The second is AI writing tools. Ask one to tailor a resume and it will happily write that you "led a team of 12" or "cut costs by 40%". It sounds great, and it isn't true. If a recruiter asks about that number, you lose the job, and you deserve to.
So I wanted a tool that does the tedious parts and is strict about the one thing that matters: nothing goes into a draft that I didn't say first.
The loop
That's the whole product. Most of the engineering went into making each step trustworthy, so the rest of this post walks through the parts I find most interesting.
Finding jobs I can actually take
Refresh pulls from five built-in boards (Remotive, RemoteOK, We Work Remotely, Himalayas and Jobicy) and any company boards I add on Greenhouse, Lever, Ashby, Workable, Recruitee or SmartRecruiters.
Each posting goes through an ordered list of eligibility rules, and the first hit wins. Exclusion runs before inclusion, so "Remote (US only)" never gets read as "remote". Every verdict shows the phrase that triggered it, which means I can check it in a second instead of rereading the posting. An optional AI pass can resolve the unclear ones, but only by quoting a phrase that appears word for word in the description. A verdict without a real quote gets thrown away.
Then each job gets a level: Strong, Possible or Weak. The level comes from title fit, required skills covered by my confirmed facts, seniority, years and eligibility. It's recomputed on every read, so confirming a new fact changes the ranking on the next page load.
The Matches inbox. j and k move, s saves, d dismisses with a reason.
Drafts that can't make things up
This is the core of the app. My CV and any notes I add are split into short facts, one claim each. A new fact starts as pending. Only facts I confirm ever reach a prompt.
Confirmed facts for the demo profile, grouped by where they came from.
FACT-n: text. Every sentence about you must end with the markers that back it. Missing information is written as [[what is missing]].checkDraft, on every generated draftNumbers, dates, amounts, percentages, ratios, URLs, emails, skills and proper nouns must appear in the cited facts.The prompt lists every confirmed fact as FACT-1, FACT-2 and so on, and the model has to end each sentence about me with the markers that back it. When it needs something the facts don't have, it writes a blank like [[years of Kubernetes experience]] instead of guessing.
Then checkDraft reads the result one sentence at a time, against only the facts that sentence cites:
A failing draft gets one retry with the problems listed. Whatever still fails becomes a [[Check this: …]] blank. And the gate, which runs on the server for every export, won't let a file out while it has a blank in it.
I measured this before trusting it. The repository has an evaluation set of eligibility and drafting cases. The recorded run with the default light model reported 0 missed exclusions and 0 invented claims, and the run exits with an error if either number goes above zero.
The scripted AI added "I also cut cloud hosting costs by 35% in one quarter." No confirmed fact says that, so the sentence is highlighted, it shows up under Needs your input, and Copy text and Download PDF stay disabled until I rewrite or remove it.
One more detail I like: when a draft is saved, it stores the text of every fact it cited. If I later edit or delete that fact, the file shows as outdated and names the change, so an old draft can't quietly keep a claim I've since corrected.
Tailoring the resume
The tailored resume isn't written from scratch. It starts from my base CV, and the model proposes a list of edits to numbered lines: rewrite the summary, reorder, reword, remove or add a line, or add a project the CV lacks. Before I see them, the app drops any edit that brings in a tool, number or name its facts don't have. I keep or undo each remaining change, and the resume is always the base CV plus the changes I kept.
Each change names the requirement it targets. "Asked for, not on your CV" lists what the posting wants that no fact covers, instead of inventing it.
Refresh can't touch your work
Refresh writes to the database a lot, and it runs unattended. A bug there could wipe my board or my drafts. Code review isn't a strong enough guard for that, so I let Postgres enforce it.
hb_syncRefresh gets its database handle from one function that runs each transaction as the hb_sync role. An architecture test stops the sync module from importing the normal connection. A job that has an application on my board can't be deleted either, because of a foreign key, so even a wrong cleanup query can't remove it.
Three checks I added recently
Is the posting still open? It's a waste to spend AI money drafting for a job that closed last week. Before any model call, a draft now fetches the posting.
The hard part was deciding what doesn't count as closed. My first version also trusted stored metadata: an expiry date from the board, or the fact that a board stopped listing the job. Neither proves anything. Boards drop jobs that are still open, and companies extend deadlines. So a fresh check of the URL now wins over anything cached, and only a 404, a closure notice in the page's main heading, or an expired structured posting for the same title and company stops the draft.
Draft all on a posting whose page says the job is no longer available. Nothing gets written, and nothing is spent.
Do the match levels mean anything? A ranking is only useful if Strong jobs do better than Weak ones. Each time I mark a job applied, the app now records the match level at that moment. The Board shows how often each level reached an interview. It hides the rates until at least 10 applications have a result, and it doesn't count pending applications as failures. I'd rather see "not enough data" than a confident percentage built on four data points.
With demo data, no level has 10 resolved applications yet, so the table shows "3/10 resolved" instead of a percentage.
"Doubled" is a number too. The citation checker already caught "40%" when the fact said "12 services". It didn't catch "doubled deploy frequency", because there's no digit in it. Words like doubled, halved and tripled are claims about numbers, so they now need a cited fact that supports them, through the same retry and blank path as every other check.
How the code is organised
Server and SPA share one TypeScript codebase, split into layers.
import type.The rules aren't a convention. test/architecture.test.ts scans every import in src/ and fails the build when a layer imports something it may not. It also forbids the clock and randomness in the domain layer, so callers pass now in and every rule is testable with a fixed date.
Other choices that paid off:
- Bun and Hono for the server. One process serves the API and the built SPA. The SPA gets typed API calls through Hono's client, with only
import typecrossing the layers. - Zod everywhere data enters. Request bodies, environment variables, job-board payloads and AI replies all go through a schema. The same enum arrays build both the Zod schemas and the database CHECK constraints.
- Numbered decisions.
docs/DECISIONS.mdhas 86 of them, and code comments cite them as "decision 58". When I wonder why something works the way it does, the answer has an address. - Tests at three levels. About 1,900 app tests, most of the service tests against a real Postgres. 370 more cover the Chrome extension, and 63 Playwright tests drive the SPA against recorded job data and a scripted AI provider.
Using an architecture review to guide two refactors
After a few weeks of features, some modules had grown in awkward ways. To find them I ran improve-codebase-architecture, an agent skill from Matt Pocock's skills collection that I keep in my dotfiles. It reads the recent commit history to find the files that change most, walks the code looking for friction, and writes an HTML report of "deepening" candidates. A deep module hides a lot of behaviour behind a small interface. A shallow one makes every caller know its internals.
The report uses a small vocabulary on purpose: module, interface, seam, depth, locality and leverage. Each candidate gets a before and after diagram, a problem, a solution, and a deletion test: would deleting this concentrate complexity or just move it around?
One card from the generated report.
It found two candidates worth doing, and I redrew both for this post in the same style.
Match evaluation. To record the match level when I apply for a job, the board code called the function that builds the whole reading pane. That loaded source listings and searched for duplicate jobs, inside the status-change transaction, only to read .level off the result.
Output writes. A file's text changes in four ways: my edits, filling blanks, accepting tailoring changes, and a fresh draft. Each one has to clear the "reviewed" mark, recompute style warnings, keep or drop the tailoring, and either keep or retake the citation snapshots. Each writer knew those rules for itself.
Both refactors went test-first. I pinned the current behaviour with tests through the public operations, moved the code, and ran the same tests again. For output writes, the architecture test now also refuses any other module that writes a file body or its citation rows, so the rule can't drift back.
The report was also useful for what it told me not to change. It looked at the project-evidence code that reads commits from GitHub and from local folders, and said to leave it: two real adapters already justify that seam. Without that note I'd probably have "cleaned it up".
What's next
Rolecraft is a personal tool first. It has no login, and it should never be on the open internet, because anyone who reaches it could read your profile or spend your AI budget. The README explains how to run your own copy with Docker Compose.
The next real test is the one in docs/MVP.md: three real applications through the loop, with no more than five minutes of edits on each resume. I'll find out whether the match levels predict anything once enough of those come back.
The code, the decisions and the architecture notes are on GitHub: github.com/RielJ/rolecraft-ai.