A Nextflow plugin · prototype
nf-labrobotics puts robotic wet-lab steps in Nextflow pipelines, with no change to the language. A wet step never runs twice, an aborted run stops a robot safely, and every plate's provenance lands in Nextflow lineage. Press Next, or click any station, to send plate Q01 down the line.
examples/qpcr/plate_map.csv), its idempotency key (SHA-256, as labctl builds it) and each well's Ct (as the simulated qPCR instrument gives it).Everything above runs on your machine against simulated robots. You need Java 17, uv and Nextflow 26.10.0 or later.
git clone https://github.com/vagkaratzas/nf-labrobotics && cd nf-labrobotics
make install # builds the plugin into ~/.nextflow/plugins
cd dashboard && uv run uvicorn nf_lab_dashboard.app:main --factory --port 8080Open http://127.0.0.1:8080 and pick a demo: qpcr runs this page's pipeline, 1000 samples over ten simulated robots, while a narrator plays the person at the bench. plate_transfer → E-stop shows the abort, the safe stop and a -resume that runs only the stopped plate again.
Wet steps are ordinary processes. Take a wet module, give it a plate record, send process_wet to the labrobotics executor.
Start the device adapter, list your robots in pools, set your limits for protocol parameters, keep the deck page open at the bench.
Device adapter →If PyLabRobot drives it, nothing to add. Otherwise write a binding, or your own adapter for openapi.json, and prove it with the conformance kit.
Before the run
From a Nextflow pipeline, through the lab, and back into lineage.
The left half of the line is Nextflow: your pipeline, a wet module, the labrobotics executor and labctl. The right half is the lab: the device adapter, its ledger and pools, the person at the bench, and the robots. Between them, one HTTP contract.
Plate Q01 waits at the start. Press Start, or Play tour to let it ride on its own.
Station 1 · Nextflow
Wet and dry steps in one run, with no change to the Nextflow language.
Second time round The same run again with -resume, after Q01 was done. Watch what doesn't happen.
This tour follows examples/qpcr, an RT-qPCR screen: 1000 samples on 11 plates, 92 samples and 4 controls each. The plate map is a CSV with one row per filled well. The workflow groups it into one typed record per plate and hands the plates to the RT_QPCR subworkflow: RNA extraction on a liquid handler, then qPCR.
plates = params.plate_map
.map { w -> tuple(w.plate, w) }
.groupBy()
.map { plate, wells -> record(id: plate, wells: wells.toSorted { a, b -> a.row <=> b.row ?: a.column <=> b.column }) }
wet = RT_QPCR(plates)Our plate is Q01, and the tour picks it up after its extraction, at the qPCR step. The plugin is one line in the config: plugins { id 'nf-labrobotics@0.1.0' }.
Read more: examples/qpcr/main.nf · the RT_QPCR subworkflow
Station 2 · Nextflow
One protocol, one ordinary process.
A wet step is a process like any other: the label process_wet, a capability hint, and a script that runs labctl. It asks for what it needs, a qpcr instrument, never a named robot.
process QPCR {
tag "${plate.id}"
label 'process_wet'
hints 'labrobotics/capability': 'qpcr'
input:
plate: Plate
output:
record(id: plate.id, result: file('result.json'),
provenance: file('provenance.json'), log: file('labctl.log'))
script:
"""
labctl run --wait --protocol qpcr --protocol-version 1.0 \\
--plate ${plate.id} --wells ${wells} --params - ...
"""
}nf-labrobotics ships 19 wet modules and 6 subworkflows, nf-core style, each with nf-tests: transfers, extractions, PCR and qPCR, reads, spins, seals, incubations. ext.params sets a protocol's parameters, e.g. withName: 'QPCR' { ext.params = [cycles: 45, channel: 'HEX'] }.
Read more: wet-modules/README.md
Station 3 · Nextflow
Wet tasks go to the lab, not to the cluster.
The plugin adds an executor named labrobotics. It runs each wet task where Nextflow runs, like the local executor: the task is labctl waiting on the lab. It also hands the task its routing: the labrobotics/capability hint, the queue directive and the run's session become NF_LAB_CAPABILITY, NF_LAB_QUEUE and NF_LAB_SESSION. The adapter address, poll interval and grace period come from the labrobotics {} scope.
plugins { id 'nf-labrobotics@0.1.0' }
labrobotics {
adapter = 'http://127.0.0.1:8000' // unset: $NF_LAB_ADAPTER
poll = 2.s
grace = 5.m
}
process {
errorStrategy = 'finish'
withLabel: 'process_wet' {
executor = 'labrobotics'
errorStrategy = { task.exitStatus in [69, 75] ? 'retry' : 'finish' }
}
}finish: robots already busy finish their plates, and nothing new starts.Dry tasks keep their own executor, HPC or cloud, in the same run.
Read more: In a pipeline
Station 4 · Nextflow
The client inside the task.
The wet task's script. labctl run --wait builds the job's idempotency key, submits the job, polls its status and, when the job ends, writes provenance.json, result.json and labctl.log into the task's work dir.
The key is a SHA-256 of the protocol, its version, the plate, its wells and the params. Here is Q01's, computed in your browser the way labctl computes it:
On -resume Nextflow finds RT_QPCR:QPCR (Q01) in its cache and doesn't run the task at all. From a fresh launch directory, with no cache, labctl would submit the same key, and the ledger would hand back the done job and its outputs.
| Exit | Means | Then |
|---|---|---|
0 | Done | Outputs in the work dir |
75 | Failed before any liquid moved | Retried |
69 | Adapter unreachable past the grace period: outcome unknown | Retried; the ledger answers with the same job |
1 | Failed in protocol, stopped, voided by a person, or refused | The run finishes; a person decides |
143 | The run was aborted (TERM) | safe_stop sent; the ledger keeps the outcome |
Read more: From a pipeline · labctl --help
Station 5 · The contract
Six calls between Nextflow's side and the lab's.
labctl and the device adapter talk over HTTP, through six calls. adapter/openapi.json is the whole contract: another adapter, in front of SiLA 2 or a vendor's scheduler, can implement it, and the pipelines don't change.
POST /jobs submit: 201 a new job, 200 the key's existing job
GET /jobs/{job_id} status: state, phase reached, still safe to stop?
GET /lookup/{key} lookup: the job for a key
POST /jobs/{job_id}/safe_stop safe_stop: the run was aborted
POST /jobs/{job_id}/release release: the plate is off the deck
GET /jobs?session=... session_jobs: for the lab reportThe conformance kit checks any adapter against the file: one key is one job, one job at a time per device, a safe stop never repeats material work, release only after the job finishes.
NF_LAB_TOKEN set, every call needs it as a bearer token. A pipeline keeps it as a Nextflow secret, so it never lands in a work dir or a log.Read more: The contract · The token
Station 6 · The lab
A wet step never runs twice.
The device adapter keeps every job in a SQLite ledger, under its key. A key has at most one live job: a retry, a -resume or a run from a fresh launch directory asks with the same key and gets the same job back.
Before queuing, the adapter resolves the job's program: the binding's defaults, then the lab's limits file, then the params asked for. An unknown param, a wrong type or a value out of range is refused (422) and nothing is queued, so a typo never runs at the default.
The ledger also records which Nextflow task collected each job, by task hash. That hash is also the task's lineage ID.
Read more: Protocol parameters · A plate's life
Station 7 · The lab
Ask for a capability, get whichever robot is free.
The job names qpcr, not a device. It waits in one queue shared by every qPCR instrument, and the free one whose deck holds the plate takes it, one job at a time. The lab, not the pipeline, says what it has; this demo runs NF_LAB_DEVICES=liquid-handling:4,qpcr:6. A queue narrows a pool further: liquid-handling/opentrons:2 is a sub-pool that Nextflow's queue directive can ask for.
Above the belt: the qPCR pool as the deck page shows it. The other plates of the run keep five robots busy; Q01 waits for a person to load it on qpcr-1.
Read more: Free and ready
Station 8 · The lab
The person at the bench.
Robots don't load their own plates. The adapter serves a deck page for people: one row per robot, and a queue that says, in words, what each waiting job needs.
| Status | What to do |
|---|---|
| empty | Load a plate |
| ready | Nothing: its queued jobs start here |
| running | Wait, or press E-stop |
| unload | Take the plate off, press Unload |
| needs_person | Check the deck, press Succeeded or Failed, then Unload |
Read more: A plate's life · When a job fails
Station 9 · The lab
Where nf-labrobotics stops and robot control begins.
nf-labrobotics ships no device drivers. The adapter drives each robot through a binding, a small class over the lab's own control stack: PyLabRobot, SiLA 2 or a vendor API. Today's bindings run PyLabRobot 0.2.2's simulated machines: liquid handlers, plate readers, thermocyclers, centrifuges, incubators and more.
class Backend(Protocol):
PROTOCOLS: set[str] # the protocols this binding runs
viewer: str | None # URL of a live view of the deck
async def setup(self) -> None: ... # once, when the adapter starts
async def run(self, job: Job, progress: Progress) -> dict: ...
async def load(self, plate: str | None) -> None: ...
async def read_result(self, job: Job) -> dict: ...run reports through progress the job's state and whether it's still safe to stop. The binding never deals with retries, -resume or two pipelines wanting one robot: the adapter keeps all of that.
uv run pytest conformance --adapter http://127.0.0.1:8000 --capability <yours> checks an adapter that uses your binding. Use a simulator, never a robot a lab is using: the kit submits real jobs.Read more: I make robots: what do I add?
Station 10 · The lab
One-step RT-qPCR, simulated.
On -resume Nothing happens here. Nextflow took QPCR (Q01) from its cache, and the ledger would have returned the done job anyway: qpcr-1 never sees Q01 again.
A real-time PCR instrument: PyLabRobot's simulated thermocycler runs the program, and the binding simulates a fluorescence read every cycle (PyLabRobot simulates no qPCR instrument). The program is the protocol's defaults: reverse transcription at 50 °C for 600 s, then 40 cycles on FAM. A well's Ct is the cycle where its curve crosses the threshold.
Read more: When a job fails · the QPCR module
Station 11 · Back in Nextflow
The plate stays on the deck; its data travels.
Not done This run's job didn't finish, so labctl exited without a result and the run ended. Go back to station 10 to deal with it.
labctl sees done. From the ledger it writes result.json, each well's Ct (null when undetermined), and provenance.json: the device, the job, its attempt and the program as it ran. It exits 0, and Nextflow's task is done.
The physical plate stays on qpcr-1, holding it, until a person presses Unload. labctl never releases a robot.
In the workflow, the plugin's labPlate function turns the two files into plain data:
include { labPlate } from 'plugin/nf-labrobotics'
def qpcr = labPlate(q.provenance, q.result) // device_id, job_id, attempt, ... plus plate and wells, each with its ctRead more: From a pipeline
Station 12 · Back in Nextflow
The calls are compute, not chemistry.
A plate counts only if its controls pass: no Ct in the no-template controls or the extraction blank, a Ct in the positive control. Then a sample with a Ct below params.max_ct, 35, is detected.
def ok = qpcr.wells.every { w ->
w.type == 'sample' || (w.type == 'positive_control' ? w.ct != null : w.ct == null)
}
def call = !ok ? 'invalid_plate' : (w.ct != null && w.ct < maxCt ? 'detected' : 'not_detected')Here the calls are a function in the workflow. A heavier analysis is a process like any other, on your HPC or cloud executor, in the same run. Give dry processes errorStrategy = 'finish' too, so a software error never stops a robot mid-plate.
Read more: examples/qpcr/main.nf
Station 13 · Back in Nextflow
Every attempt, failures included.
The pipeline ends with LAB_REPORT, as an nf-core pipeline ends with MultiQC. It runs labctl report --session <id> and writes lab_report.json and lab_report.tsv: every job of the session, every attempt, each with the Nextflow tasks that collected it.
Lineage records only the attempt that worked; the report keeps the ones that didn't, with the reason. A run and its -resumes share a session, so the run that completes reports the attempts of the runs that failed before it.
Meanwhile, at the bench, someone takes Q01 off and presses Unload: qpcr-1 is free for the next plate.
Read more: The lab report
Station 14 · Back in Nextflow
Ask where a result came from.
The examples turn on Nextflow's lineage: every task, its inputs and outputs, each wet task's provenance.json and the lab report. The ledger records each job's collecting task by its hash, which is also its lineage ID, so you can walk from a robot to the results and back:
nextflow lineage list # one row per run; a -resume keeps the session
nextflow lineage diff $FIRST $RUN # what changed between two runs?
nextflow lineage check $RUN/samples.csv # did anyone change a result after the run?
# a device is suspect: which tasks collected its jobs?
sqlite3 ledger.sqlite "SELECT task_hash FROM collections JOIN jobs USING (job_id)
WHERE device_id = 'qpcr-1' AND state = 'done'"Next, the belt loops back to the start: the same run again with -resume. Watch what doesn't happen.
Read more: Questions lineage answers