What we test for
Typical work
- Taking a feature that works in a notebook and making it survive real users
- An evaluation suite, so a prompt change can be shipped without crossing fingers
- Cutting the cost of a feature that works but is too expensive to leave on
- The retrieval layer: what goes in the context, in what order, and what gets left out
The coding test
Here is a prompt that works on five examples and fails on the sixth. Build the evaluation set first, then fix it, and show the change in the score rather than in a demo. Then we swap the model underneath without warning. Scored on whether the evaluation caught the regression.
Inside Prompt engineering
The ones people search for by name get a page of their own, with their own coding test and interview questions.
Five interview questions you can reuse
We use questions like these in our soft-skills and technical rounds. Steal them for your own interviews.
A prompt works in testing and fails for users. What is usually different?
A strong answer covers: The inputs. Test cases are tidy, short and in one language; real ones arrive with pasted formatting, contradictions, missing fields and someone being rude to the model. The second difference is position: what sat at the top of a short context is buried in the middle of a long one. Collect real failures and make them the evaluation set, because a prompt tuned against imagined input is tuned against the wrong thing.
How do you know a prompt change is an improvement?
A strong answer covers: Only against a scored set, run before and after, with the failures that prompted the change in it. Anything else is a demo. The set does not need to be large to be useful, but it needs the hard cases and it needs a scorer you trust; where a human has to judge, say so and sample rather than pretend a number exists.
Where does retrieval usually go wrong?
A strong answer covers: Chunking that cuts a table or a clause in half, embeddings asked to carry a keyword they were never going to carry, and top-k fixed at a number that is too many for a simple question and too few for a real one. Hybrid search fixes more than a better embedding model does. And the honest failure mode is retrieving nothing relevant and answering anyway, which is why the model needs a way to say it does not have it.
How do you stop a model inventing things in a product?
A strong answer covers: Give it the material and constrain the answer to it, make the structure machine-checkable so a malformed answer is caught rather than displayed, and verify anything a user will act on — a number, a date, a citation — against the source. Then design for the case where it still happens: show the source next to the claim so the reader can check in one glance.
A feature works but costs too much per call. What do you look at?
A strong answer covers: The context first, because it is usually mostly waste: history nobody needs, instructions repeated every turn, documents retrieved and never cited. Then caching the stable prefix, then routing the easy majority to a smaller model and keeping the large one for what actually needs it. Measure on the evaluation set, or the saving turns into a quality drop nobody logged.
One invoice a month.
The shortlist is free. Pick the contract that suits you: monthly, six months or twelve.
What every seat includes
- Contracts, payroll and compliance handled
- Vetting: coding test, soft skills, English and a level check
- Free replacements: unlimited on 12+ months, 2 on 6 months, 1 on monthly
- A direct line to our CEO
Terms in plain words
- Monthly, six months or twelve, your choice
- Your rate is locked for the whole contract on 6 and 12+ months
- Monthly ends with 30 days’ notice
- Want to hire the developer full-time? The buyout is set in the contract from day one
Questions tech leads ask us
How senior are your prompt engineers?
6+ years of professional software engineering, with recent production work on LLM features. We count engineering years on purpose: the field is younger than the seniority we ask for.
Can I see the coding test results?
Yes. Every profile card includes the score and the reviewer's notes.
Skip the sales rep. Talk to the CEO.
Tell us what you're building, your stack and when you need people. Three vetted profiles per role within 48 hours of the call, and a straight answer on whether we're the right fit.