Open source · 2022
Conduit
A single-binary streaming data engine. Drop-in for most Kafka Connect pipelines, runs anywhere, no JVM, and the dev loop is fast.
Why I built it
Kafka Connect is powerful and miserable. JVM tuning, opaque failures, painful local dev. Most teams aren't running Connect by choice — they're running it because the connector they need only exists there.
Process
- Set out to match Connect's connector ecosystem in Go without becoming Connect. Single binary, plugin-based, runs in a container or on a laptop.
- Built a generic source/destination interface and a connector SDK so writing a new connector feels like writing a small library, not a framework.
- Stream processors are real code, not a DSL. Boring on purpose.
- Open-sourced under Meroxa, now community-stewarded.
Screens
ArchitectureSources → processors → destinations.
CLISingle binary, single config.
Connector SDKWrite a connector in a couple hundred lines.
Lessons
- The 'no JVM' part sold the project in five seconds, every time.
- Plugin SDKs are a marketing surface, not an engineering one.
- Boring on the inside is a feature.