R, Julia, Python and notebooks
Analysis code has a particular failure mode: it runs once, produces a number, and the number goes into a figure. Nothing ever tells you the number was wrong.
Why this is the hard case, not the easy one
A web application has a test suite because an outage is visible. A pipeline that normalises an expression matrix often does not, because its output is a plot that looks plausible either way. When an AI agent edits that pipeline and reports success, the report means only that the agent encountered nothing it recognised as failure — and in a repository with four tests and no continuous integration, there is very little for it to recognise.
This is exactly the situation MAESTRO is built to be honest about. A
change to analysis code that could not be verified comes back marked
completed_unverified, with the reason, rather than green.
The amber badge is not a degraded success: it is the system declining to
tell you something it does not know.
What the sandbox can run
The sandbox image carries more than a Python interpreter, because research repositories are not a Python monoculture.
-
R — the interpreter with
testthat,renvanddevtoolsinstalled from the distribution's own packages. A suite is run through testthat and its TAP output is parsed, so the pass and fail counts in the record are the ones the runner printed. - Julia — 1.10.4, with the project's own environment activated and the package manager's test entry point used, rather than a guess at which files are tests.
-
Python — including conda environments. An
environment.ymltakes precedence over a plain requirements file when both are present, because a project that ships one usually cannot be built from the other. -
Jupyter notebooks —
.ipynbfiles are detected as code rather than skipped as data, executed, and reported per cell, so a failure names the cell it came from.
Dependency installation works inside the sandbox through an allowlisting proxy. It reaches PyPI, CRAN, the Julia registry, the conda channels and npm, and — because Julia's registry resolves packages to their source repositories by design — GitHub. Nothing outside that list resolves at all. The sandbox isolation page describes that boundary in detail.
What the repository index does and does not cover
Before it plans a change, MAESTRO parses the repository into a structural graph of functions, classes, methods and import relationships, and queries it for callers and impact chains. That graph currently parses Python, TypeScript, JavaScript, Java and Go. R and Julia are searched lexically — full-text search across every symbol — and run in the sandbox, but they are not graphed.
That is a real difference and worth knowing before you choose a project to start with: on a Python codebase the planning phase can answer "what else calls this" structurally, and on an R codebase it answers the same question from text. Both are better than nothing and neither is the same as the other.
When the runner is missing, it says so
A missing interpreter in the sandbox is our gap, not your repository's. When a mission dispatches a suite and the sandbox answers "command not found", the mission reports that the runner was absent — not that the repository has no tests. The distinction matters because the second sentence is an accusation about your code, and in that situation it would be false.
The same applies to the cases a sandbox honestly cannot reach. A suite that needs a headless browser to drive a charting library is outside what a research-code image provides; a suite that was already broken on the sandbox's interpreter version was broken before MAESTRO arrived. Both come back unverified with the reason, and neither is reported as a pass.
Across an internal benchmark of ten open-source repositories, the honest ceiling was not ten. Two were structurally unwinnable — one whose suite could not collect on a modern interpreter, one that needed a browser — and reporting those as unverified with the reason is the correct answer rather than a defect to engineer away.
Where it fits in a research group
The work this is good at is the work nobody wants: adding tests to a script that has none, porting an analysis to a maintained library, tracking down why a pipeline's output moved between two versions, making a notebook reproducible outside the laptop it was written on. Each of those is a well-specified change to code whose correctness nobody can eyeball, which is where a written specification you approve before any file changes earns its keep.
Every mission leaves a verification record — the specification, the diff, the test output and whether it was verified — which is also the thing to attach when a reviewer or a funder asks how a piece of analysis code was changed and checked.
The thirteen phases covers the mission pipeline, and pricing covers what a mission costs.