Precursor post:https://www.normansoven.com/post/a-pluggable-workflow-engine-is-something-always-been-promised-never-delivered
I DELIVERED BABY!
For the longest time, I've had this itch. You know the one – that recurring problem you see in project after project, the one you know there has to be a better way to solve. For me, that's been orchestrating those complex, multi-step business processes that are the backbone of almost any real-world application.
Think about it: user signup isn't just "create a database record." It's validate input, check for existing user, hash password, create user, send welcome email, maybe notify an internal team, perhaps even provision some initial resources. Or what about e-commerce checkout? That’s a beast: create order, validate cart, process payment (which itself might have multiple internal steps or even conditional routing to different gateways), update inventory, send confirmation, schedule shipping... the list goes on.
I spent a good couple of days just mulling this over, sketching out ideas on a whiteboard (okay, a digital one). How could we define these flows clearly? How could we manage the data that needs to pass between steps? How could we handle variations – like "if the user is a VIP, use payment gateway X, otherwise use Y"? And crucially, how could we do this in Rust, leveraging its power and safety, without getting bogged down by an overly prescriptive, heavy framework?
This kind of structured process management is a bit of a holy grail for many developers. We all want to establish robust, reusable patterns for common operations like authentication, payment processing, user onboarding, and complex data transformations. We want to build these patterns once, test them thoroughly, and then easily integrate them, adapt them, and compose them with various external services or internal modules.
And so, Orka was born.
What is Orka? (The Non-Framework Framework)
Orka isn't trying to be a massive, all-encompassing framework that dictates every aspect of your application. Instead, it's a lean, asynchronous, and type-safe workflow engine (or perhaps, an "orchestration kernel") for Rust. Its goal is to give you the tools to define and execute these multi-step processes in a way that's:
Clear: Break down complex operations into named, manageable steps.
Flexible: Easily adapt workflows with conditional logic and dynamic dispatch.
Type-Safe: Leverage Rust's compiler to catch errors early.
Asynchronous: Built for modern, I/O-bound applications using async/await.
Pluggable: Design steps as independent units that can interact with any service or library you choose.
A Glimpse of the Orka Way
Imagine you're defining a user onboarding flow. Instead of a long, tangled function, you might define an Orka "Pipeline":
// Conceptual Orka-like pseudocode/Rust sketch
// Your data that flows through the pipeline
struct OnboardingContext {
email: String,
user_id: Option<Uuid>,
plan_type: String,
welcome_email_sent: bool,
provisioning_status: String,
}
// Your application's error type
enum OnboardingError { /* ... */ }
// Define the pipeline and its steps
let mut onboarding_pipeline = Pipeline::<OnboardingContext, OnboardingError>::new(&[
("validate_input", /* ... */),
("create_user_account", /* ... */),
("determine_provisioning", /* ... */), // This step might have conditional logic
("send_welcome_email", /* ... */),
]);
// Attach logic (handlers) to each step
onboarding_pipeline.on_step("validate_input", |context| async {
// ... your validation logic ...
if context.email.is_invalid() {
return Err(OnboardingError::InvalidInput);
}
Ok(Continue)
});
onboarding_pipeline.on_step("create_user_account", |context| async {
// ... interact with your DB to create a user ...
// context.user_id = new_user.id;
Ok(Continue)
});
// Now, for "determine_provisioning", imagine conditional logic:
onboarding_pipeline.conditional_step("determine_provisioning")
.if_condition(|context| context.plan_type == "basic")
.run_sub_pipeline(basic_provisioning_pipeline_factory, /* extractor */)
.if_condition(|context| context.plan_type == "premium")
.run_sub_pipeline(premium_provisioning_pipeline_factory, /* extractor */)
.if_no_match(StopAndLogError) // Or some default
.finalize();
// ... and so on for other steps.
The beauty here is that basic_provisioning_pipeline and premium_provisioning_pipeline could be entirely separate, reusable Orka pipelines themselves, each with their own focused steps.
Key ideas you're working with in Orka:
Pipelines & Steps: The core structure.
ContextData: A shared, mutable state object that flows through the pipeline. Orka ensures you think about how you access it (read/write locks) especially around async operations.
Handlers: Your async functions that do the actual work for each step.
Conditional Scopes: The magic for if/else style branching or dynamic dispatch to different sub-workflows. You can provide "factories" to create these sub-workflows on the fly.
Error Handling: Your pipeline has its own error type, and Orka helps propagate these clearly.
Why This Matters
Building with Orka (or a similar principled approach) means:
Less Spaghetti Code: Complex logic becomes a series of well-defined stages.
Easier Reasoning: You can look at a pipeline definition and understand the high-level flow of an operation.
Better Testability: Individual steps or even entire sub-pipelines can be unit-tested more easily.
Reusable Patterns: That "premium_provisioning_pipeline"? You might reuse it elsewhere or adapt it for a different main workflow. Your authentication flow, once defined as an Orka pipeline, becomes a robust, pluggable component.
Orka is still young, and it's been a fascinating journey to bring this idea from a "what if" to working code. The goal was never to build another heavy framework, but to provide a lightweight yet powerful kernel for orchestrating the kinds of complex, asynchronous tasks that are so common today. It's about establishing those "holy grail" patterns for common backend operations in a way that feels native to Rust and gets out of your way once the flow is defined.
I'm excited to see how it evolves and what kinds of workflows people might build with it. If you've ever found yourself wrestling with tangled business logic, maybe Orka can offer a clearer path. Check out the repository and the examples – I'd love to hear your thoughts!