For AI and software consultancies that present options to clients
Reference comparisons of AI configurations, each with its date, its sources and its limits
Theatre is a project to publish dated reference evidence about AI systems, for consultants who present options to clients. You pick a topic and get one comparison of AI configurations, with its date, its method, its sources and a list of what it does not show.
Try it with sample dataTell us which comparison you would use in a proposal
The samples are invented. Their figures are placeholders, not measurements.
Leave out client names. Mail to this address goes to the project inbox and is read there. A reply may be written with the help of software.
Where it stands
- The request form and the sample page work. The samples in them are invented to test the software and must not be used in client work.
- No measured sample has been published. A real one will need a named editor to approve it before it is shown.
- Free public sources already publish dated results with their method, and a consultancy can run its own comparison. Whether a standard sample is worth more than either is not known.
- No consultancy has been asked. Nothing is for sale. No customers.
Checked
What a sample contains
- A results table
- Each configuration against each measure. A figure that was not measured is shown as not measured.
- The method
- The task set, the settings and the scoring, so a reader can see what was held constant.
- Two dates
- The date the sample describes, and the date it is due for review.
- Sources and limits
- Each figure points to its source. The limitations and sources stay with the table when you download it.
How to read a reference result
A reference result describes one task set on one date. It shows how these configurations compared on that task set on that date.
It is not a forecast for a client's own work, and a proposal that quotes it should say so. If your own evaluation would change the task set, the settings or the scoring, the sample tells you what to hold constant, not what you will find.
What Theatre does not do
It runs no custom evaluations and it is not a leaderboard of every model. Where a topic has no approved sample, the page says so and shows nothing in its place.
It asks for no email address and sends nothing. A sample opens on a private page.