Igor Morozov

Choosing a Service Object Design in Ruby

2020-06-01T18:57:00+00:00

I’ve been programming for a long time and I’ve had countless arguments about different things. I’d like to list top four reasons I’ve had an argument online.

Style guide. Thankfully, the number of arguments reduces as I mature, but I’m still spending a lot of time on them. I’d rather have an extremely opinionated styleguide and just stop talking about it. Something like wemake-python-styleguide, but for Ruby.

Monads. I have to admit that this word is almost banned from my vocabulary because of how many arguments I’ve had about it. It’s getting better, but people still like to argue about them. I wrote an article recently about them in hope to show that there’s nothing special to argue about – monads are just abstractions that may or may not be helpful. It all depends on your problems and approaches.

How to design domain logic. It may be an extremely interesting and helpful discussion, or it may turn into a useless argument. When it goes bad, it’s usually because we’re trying to discuss insignificant details and lower-level things. Where do we put arguments? What about dependency injection? How do we use instance variables? Fat model? Service objects? Ughh!

Different interpretation of common terminology. What do we mean when we say “interactor”? What about “architecture”? Is it a state when we’re just passing values from function to function? Is duck typing really an absence of types? What is a type, anyway? What does it mean to write “object oriented” code? What about “functional” approach? Do we need immutability in OO design? Those topics lead to endless discussions with little output.

As much as I love learning about new things, those arguments are extremely energy-draining. They got me thinking: since we’re usually going over the same thing, why don’t we just dump the knowledge somewhere and refer to it instead of arguing? That’s what I’m going to do.

I’m starting a series of blog posts about different topics in Ruby world. My goal is to describe different approaches to the same problems and highlight pros and cons of each one. Perhaps, pick a favorite one and promote it.

Right now I want to focus on two larger topics:

Designing service objects
Handling errors in domain logic. Exceptions, values, result objects

This is a first post of the series, and it will cover the first topic: building helpful service objects.

We will go through the basics: what are we talking about when we say “service object”. We’ll look through different approaches and see which ones bring the most benefit and which ones should probably be put to rest. In the end, I’m going to suggest a working design and a couple of guidelines you can use to improve your service object game.

Our main challenge is domain logic

Service object is a common pattern in Ruby community, but you might also see something similar in other languages. Python’s stories were greatly influenced by dry-transactions and Trailblazer — some of the tools we could have used for service objects.

The sole purpose of a service object is to be a place for your domain logic. Remember the usual models vs controllers dilemma? Well, think no more. Complex domain logic goes to service objects.

Let’s step back a little and see why we need to care about the logic at all, and why we should care about its exact location.

Usually, we’re building applications that solve a set of problems for a specific business. Sometimes its logic is trivial, but usually it’s something a sophisticated system that backs a complex business. Sometimes the software is the product that we sell. Either way, the logic becomes complex quite easily.

When we’re modelling complex processes, we have to make a lot of decisions: what needs to be captured in the model; what are the names and processes; what are the boundaries; what’s the shape of our data; how to organize domain logic well. Service objects don’t answer all of our questions, but they nudge us.

Service objects nudge us to organize

When we were working with the common model-view-controller paradigm, we had to make trade-offs and try and design logic using what we’ve got: models and controllers. Concerns, if we’re advanced enough. We put domain logic in the models, application logic in the controllers, and we get our happily ever after.

What if the logic doesn’t fit the model, though? It happens when the relationship is not obvious, or when the process affects multiple models. Here’s a few examples where the solution is not as obvious:

Does an applicant get a job at a company, or does the company hire the applicant?
Does a buyer sell the goods, or does the customer buy the goods?
What if we’re trying to match people by their taste in music? How do we measure it? Is it jane#compare(john), or is it the other way around? Is it something else?
What if we’re firing someone? Does a manager fire the person? Does the person fire themself? What if HR initiated the process, not the manager? What if it’s a layoff?

Sometimes, the processes in our business are a bit more complex to be reasonably put into a specific model. There are usually different solutions:

Add multiple entry points like Group#add and User#add_to_group. Works best if you’ve found a way to avoid duplication.

Create a new model. It works best if the model matches the real-world domain. It’s reasonable to have a JobApplication which can be accepted or rejected.

Use different models in different contexts. It’s easier to make decisions when we contextualize things. This way, similar entities have different behavior, depending on the context.

Extract the logic into a function / procedure. It feels quite natural to have an option to call hire_candidate.

Note: I've been saying "model" quite a lot. It's not always about ActiveRecord::Model, though. It can be a plain Ruby object or Sequel::Model too.

Service objects nudge us to to the latter – extract the logic into a function or a procedure. Except, we’re using classes and objects instead of “real” functions. Hence the name “service objects”.

It’s not a silver bullet by any chance, but it’s a nice tool which you can combine with other approaches to build better software.

What they look like

Here’s the thing about service object: it’s not really a well-documented pattern. People try to figure out how to design them in a meaningful way, and they get different results. There’s a lot of ways to categorize the service objects, with different level of detail. I’m going to take a shot and categorize by the service object behavior.

The doer is an service object which has the -er suffix in the name. It has a name like OrderCreator, UserRenamer, PurchasePlacer or something similar. This object looks like it’s a person fulfilling their job, and usually doesn’t exist in anyone’s vocabulary, except for the developers. In some cases, it may clash with the terminology as OrderCreator may be both a person who placed the order and a service object. The doer usually has one public method named call, perform, or the one matching its purpose: create, update, assign, etc.

class OrderCreator
  def create(...) # or call / perform / etc
    ...
  end
end

The multitool is a service which fulfills many jobs at once. For instance, OrderManager may assign and remove couriers, update delivery dates and even cancel the order. Everything is centered around a specific entity.

class OrderManager
  def initialize(order)
    ...
  end

  def assign(...) 
    ...
  end
  
  def cancel(...) 
    ...
  end

  def reschedule(...) 
    ...
  end
end

The event is a service object which models a process which starts when a specific event occurs. It’s usually a complex multi-step process. The name usually captures the name of the event: ApplicationSubmitted, OrderShipped, UserBlocked, and so on.

class OrderPlaced
  def call(...) # or handle / perform / etc
    ...
  end
end

The command is a lot similar to the event, except it’s designed as an imperative action in your domain. The object has a name similar to SubmitApplication, SubmitOrder, BlockUser, etc. It may look like the doer except for one major difference: it has a proper naming. People also call it an “operation”, or even a “use case” or an “interactor”.

class SubmitOrder
  def call(...) # or handle / perform / etc
    ...
  end
end

Other service object implementations usually fit within one of those four groups. If I’ve missed out on something, please let me know at igor@morozov.is.

I’m going to be blunt and say that you should probably throw away the doer and the multitool and replace them with something else if you’re using them. Let’s see what their problem is:

The doer is like the command, except it has a poor naming. Instead of capturing something real like an action, it’s modelled as if it’s a full-grown entity in your business. In most cases your business does not have a SomethingCreator, but it has an process to CreateSomething. There’s a huge benefit in speaking a natural language instead of inventing your own, so my advise is turn all doers into commands.

The multitool is a nice attempt at a service object, but has a couple of fundamental flaws.

Similar to the doer, the name doesn’t capture domain pretty well. I’ve seen people choose names like OrderManager or OrderService. Neither of those really exists in the domain.
Is it really a “service”, or is it a model in disguise? You may achieve similar level of isolation by extracting the logic to a module / a concern and including it to your model.

While I can totally understand the desire to use this design because it extracts and isolates the logic and makes it feel like everything is better, I’d advise everyone to take a deeper look at their own paradigm and see if there are better tools to solve the same problems.

One of the question you have to ask ourselves: why did we have to stray away from the good old object-oriented model and common Rails ways? Perhaps, we’re better off using approaches described in Eventide’s useful object manifesto, Yegor Bugaenko’s Elegant Objects or Ivan Nemytchenko’s Rails Hurts → Painless Rails. I’m no expert in any of those things, so let’s speak about useful service objects instead.

Why some service objects are more useful than others

In his RubyRussia 2019 talk “The future of dependency management in Ruby” Anton Davydov mentioned the problems with service objects and showed the many ways to use them. When he mentioned the lack of standardization, I knew I wanted to write an overview and highlight the most useful ones, so let’s do it.

Oh the diversity of methods

As you can see, there are at least eight popular ways to name the service object’s primary method. It is not really a problem, as you just have to pick whatever works for you and stay consistent about it. I prefer #call, but you might want something else.

There’s a deeper problem: how do we actually use the objects? How do we build them? Where do we pass the parameters? What about configuration? What about dependencies? Oh my! Just take a look at the many ways to use service objects. The slide covers probably 99% of known service object usages, so kudos to Anton for putting together the list.

The problematic variety of ways

Let’s make sure we’re on the same page about terminology before going on.

When we’re talking about params, we’re usually talking about ordinary data that we pass to the object. It’s the same arguments we would normally pass to a method that does something.

Dependencies are a bit more tricky. Usually, our service objects can’t perform a task on their own. They need to know how to retrieve data from the database, how to send an e-mail, how to run some related logic. It’s impractical to implement all of this ourselves, so we delegate it to some external objects, services and modules. Those are the dependencies. They’re the owners of the knowledge.

Options are a bit like dependencies, but simpler. It’s a run-time configuration. Rule of thumb: if you’ve put some magic numbers, strings or other values in a constant, it’s likely one of those options.

Now that we’re clear about shared terminology, let’s speak about the list. I’ve rearranged it and split the items in three groups. The result is heavily opinionated, so I’ll explain it afterwards.

The most helpful service objects are the ones which give you the most power. They’re arguably the most pragmatic ones.

Service.new(options).call(params)
Service.new(dependencies).call(params), which is almost the same as the example above
Service.new.call(params), only if it’s a shorthand for the first two options with reasonable defaults
Service.call(params), when it’s an instance created via the first three options. i.e. Service = OtherService.new(...)

Moderately helpful won’t bring you as much benefit, but they’re still decent if you use them well

Service.call(params), when you just don’t need to instantiate anything. You won’t get the benefit of configuration, dependency injection or anything, but it’s still a decent piece of logic
Service.new.call(params). It’s not really helpful if you cant’t configure it at all, but oh well. A future-proof design may be helpful though.

Not really helpful are redundant or just poorly designed. You should probably reconsider when you meet one

Service.new(params).call
Service.call(params) is bad if it’s just a shorthand for most of the new.call variations

This classification is purely opinion-based, yet there’s reasoning behind all this. It’s mostly based on my own experience in software engineering, and a couple of other ideas. It mostly comes from the fact that I like my code to be deterministic and easily modifiable. I’m also a little product-oriented, so I fiddle around with different configuration quite often.

Each object must have a reasonable lifetime. Service objects are essentially complex functions and procedures, and their lifetime should probably be similar to one of any other function, module or class. Even if we’re into OOP, instantiating and object which can only be used once before being discarded seems to go against the general idea. Sure, there are cases where the function lifetime should be short, but those cases require extra thought.

Logic should be easily extendable. Especially if we’re building a start-up which is rapidly evolving. Want to pay your contractors a 10% bonus instead of a usual 5%? Just configure the service and use it. Handy for rapid and cheap experimenting. Want to refund a user even though we normally don’t? Just use the service with a different set of policies. Works best if I don’t have to write any code to customize it.

Code should expose bad design instead of promoting it. Writing an overly complex logic should be possible, yet the code must look and feel overly complex. This way, you’ll be able to improve your design before it becomes too time-consuming to maintain.

No mutable state is a common idea in functional programming and a not-that-common idea in the OOP world. It adds verbosity, but verbosity is not a problem. We’ll get better reusability, testability and composability if we follow the rule.

Services should be composable. It means we should be able to organize them in a nice pipeline to avoid clumsy interfaces. We can achieve it by returning composable values, like result objects, monads and stuff like this.

Our logic should be insighful. The code should help us figure out how the world works. We need to learn about our processes, their limitations and core participants. The complexity of the process, points of pressure, possible bugs and likely mismatch with the real-world domain. The code should help us gather the insights instead of obfsucating them.

That said, I’ve found that I get the most benefit when I’m using a constructor to configure the service object and provide dependencies, and pass the input parameters to the #call itself. It’s a bit more verbose because I have to explicitly declare all dependencies and I have to pass variables around. It brings a great benefit as I can feel that I have to refactor this place when it gets too complex. I also heavily use default dependencies, so I don’t have to be too explicit.

Whenever I feel like there’s no need to configure, or when the team has a different convention, I like to use class methods and avoid instances. This way, I’m still getting the benefits of a good lifetime and I get to expose the overly complex design. This works pretty well too.

Q: why use class methods when you can just use a module?

A: I like to think that modules are meant to be included into your classes, or serve as a namespace. There's nothing wrong with using module methods instead, though.

Other designs, especially the new(params).call have failed to meet my expectations. Its only benefit is that I can utilize instance variables to save myself a few taps. I don’t want to trade off all the benefits for that.

I’ll stick to the new(options/dependencies).call(params), as this is the most powerful way to use service objects. We’re going to dive deeper into the practice in the next part, so here’s the design that I’m promoting:

# A service object which pays a baker a bonus for an order
# It's a command, or an operation

class RewardBaker
  attr_reader :bonus_ratio

  def initialize(bonus_ratio: )
    @bonus_ratio = bonus_ratio
    # <= any other dependency / configuration goes here
  end

  def call(order)
    ... # <= logic goes here
  end
end

reward_baker = RewardBaker.new(bonus_ratio: 0.05) # a 5% bonus is nice
reward_baker.call(other_order)

What’s going to happen next

Service objects are a large enough topic, and I can’t cover them in one post. If I do, only a few people will have the time to read it – it’s going to be too overwhelming. So I’m going to release at least two more parts: “the practice” and “the next level”.

The practice will be a design exercise where we model a business process and illustrate decisions and trade-offs of service objects. It’s going to show how to use the event and the command and why those designs have a name which looks like they come from CQRS, Event-Sourcing or something similar.

In “The next level” I’ll talk about techniques which will help you get more from service objects: reduce boilerplate, organize a pipeline, and gather more insights. Afterwards I’ll address some of the flaws. I’ll finish it with a small guideline on designing service objects. I’ll also address some of my claims about the design decisions.

The rabbit hole

If you want to go down the rabbit hole and discover more yourself, feel free to dig through the resources I’ve mentioned below. Make sure to check the first two articles. They criticize service objects and provide nice alternatives — those may be extremely helpful for you too.

Avdi Grimm highlights the possible problems of service objects and provides alternatives. The “Domain-driven design” part is important, you should totally read it.

Jason Swett wrote a nice piece addressing service objects and their problem.

Toptal has a solid article on service objects which gives some advise on good designs. You may notice that I don’t agree with it, but it’s alright. You may like their reasoning more. Check it out.

Ivan Nemytchenko’s book on Painless Rails. If you like Rails, but not quite, this book might help you improve your game without service objects.

The future of dependency management in Ruby by Anton Davydov

Useful objects manifesto by eventide

Hanami guide has a decent article on the pattern and how to use it.

dry-transactions — a once-popular gem, which used to be deprecated, but right now it’s going throw a rework. It’s a DSL for domain logic, which essentially implements the “service object” and “railway oriented programming” ideas in a nice way

Trailblazer was aimed to simplify our lives and domain logic, and brought a lot of new ideas to Ruby world

How Python devs implemented the idea

Most strict linter for Python. I want something like this in Ruby. Hopefully, Standard will help with that.

Yegor Bugaenko’s blog contains a lot of controversial and thought-provoking content. One of the ideas is that immutable objects should be a default in the object-oriented paradigm.

Yegor Bugaenko wrote a book on his ideas. If you want a different look on OOP, it’s definitely going to help you.

I wrote Should I really use monads? to discuss a smaller topic which comes with service objects: monads. Together those two abstractions enable you to do railway oriented programming, which is nice.

Rob Race’s article on 3 tenets of service objects. It was an interesting read.

Scott Domes shows the ServiceObject.call(args) as a shorthand for new(args).perform in Service objects in Rails.

Should I _really_ use monads?

2020-04-01T20:19:00+00:00

A couple of weeks ago I witnessed a dialogue in a Ruby chat. I’m paraphrasing, but it went like this:

xxx: What is dry? I’ve seen this gem prefix and discussions, but never actually learned about it.
yyy: It’s a set of libraries to tackle some problems.
zzz: Yeah, and introduce new ones, such as “How do I explain to my colleague that they need monads”.

Let’s be honest. I felt so many emotions that I couldn’t think straight. I’ve been discussing this exact topic so many times that I’ve exhausted myself. There’s a lot of misconceptions, frustration and plain skepticism around monads, and it all leads to aggressive rejection by many.

Right now, I want to finish this topic once and for all. Not going to do it in this post, though. I’m writing a huge piece on error handling techniques in Ruby, which will cover strong and weak points of different techniques, including monads.

In this post, I will try to step back and speak about monads from a more practical and emotional perspective. I will briefly explain what a monad really is, why is it valuable, and some of the common issues with it. It’s about people and technology, so don’t expect to see any code.

What is a monad

When we’re speaking about monads in Ruby, we’re usually talking about dry-monads – a library that implements them. However, monad is a much broader concept that comes from category theory:

A monad is just a monoid in the category of endofunctors. What’s the problem?

The internet is full of jokes like this. It’s a comprehensive definition, but it’s so vague. To be fair, most definitions won’t give you anything concrete. Giving a definition to monads is a lot like trying to give a proper definition to a “musical instrument”. It’s going to be either vague and correct or understandable and incomplete; simply because each instrument, just like each monad, plays a different role. I’ll give you somewhat correct, but a little vague explanation.

A monad is just a ~~fancy~~ mathematical name for abstractions that behave in a specific way. In theory, it lets us chain them, compose them in different ways. In practice, we’d rarely notice the monadic nature of those abstractions. Let’s focus on individual monads and see what they bring to the table.

Fact: Ruby's built-in Array and String are monoids too. We never think about it — that's how we should treat monads too.

Maybe is an abstraction that allows us to express absence of data. In practice, it enables us to do nil-safe computations and never worry about getting undefined method for nil:NilClass. It acts like an extremely explicit alternative to nil. It may be good when you return it from a repository, but I don’t use it my models.

Try is a nice wrapper for exceptions. It’s helpful if you need to chain some actions, which may raise an exception. The standard exception-catching mechanism may break the flow and make you jump around the code to get the full picture. Try saves you from this.

Task is a wrapper around concurrent ruby’s Promise. I’ve seen folks use it as an easy way to do concurrent IO – download things, write to database, etc. Just take at look at Vasily Kolesnikov’s asynchronous file downloader. It receives a list of URLs and creates tasks to download them. Without many low-level asynchronous details.

Result is the most common monad in a Ruby world. We use it to express result of a function call. It’s most useful when the result can be a Success or a Failure, which works as natural as it sounds. You can just return Success(with_your_data) from your function and work with it. If something goes wrong, you would return Failure("and provide some details").

There are other monads, but I’ll omit them as those are not that common and won’t benefit to the story.

The beef with monads

So, what’s the problem with monads? They seem to be a pretty controversial topic, but why? They seem to be pretty good at doing their job. What’s wrong with that?

Everyone has their own problems with monads. Some folks get mildly annoyed by little things. Others are outraged that such an abomination exists in Ruby. Just see what folks have been telling me:

Monads are only useful in statically typed languages and only cause trouble in Ruby
Functional programming doesn’t look right in Ruby
Monads don’t feel like a right fit in Ruby
It’s just a syntactic sugar for if and else (speaking about Result)
We already have exceptions for that (speaking about Result)
It’s over-engineering
People who use monads look like a cult
Don’t try to bring Haskell into Ruby

It’s just a list of things off the top of my head. Sure, I’m paraphrasing, but the main idea looks like this.

Some people who share those things are nice, friendly and welcoming; while some are hostile and won’t accept any reason. I won’t try to generalize based on their background or any other factor. Instead, I will address some misconceptions and speak about emotions that drive them.

Is Ruby really the right place for those things?

This issue is probably the most popular one. There’s actually a lot of reasons for this skepticism, let’s think about them.

Monads are a concept from the category theory. Math. Their representation in software development is mostly Haskell, which uses them heavily, and it’s definitely not a mainstream language. It creates a subconscious association: haskell / monads = something difficult and clumsy.

They come from other languages. The Result type gets a lot of usage in Rust, Kotlin, Swift, F#, OCaml/ReasonML and Elm too. All of those languages are statically typed and compiled, which enables to do extra static analysis. For example, the compilers may check if you’ve handled all possible cases. Those checks are nearly impossible to implement in Ruby.

Ruby is an object-oriented language. Monads are a concept from functional programming, which may seem a bit odd and counter-intuitive.

You’ll have to learn a new interface. When you take a look at dry-monads, you’ll see unfamiliar methods like #bind, #fmap, #or_fmap. Those things require some additional learning and are not so trivial.

There’s something we never did before. The so-called do notation is a syntactic sugar which looks unfamiliar to Ruby developers.

Combining different kinds of monads may be troublesome. As an example, you may treat Result as a Maybe, which will definitely result in bugs.

It requires extra effort to learn and adapt. This is one of the most bitter points here. Having to learn may be annoying, and the whole process isn’t easy.

Those things boil down to four points:

Are they really idiomatic?
Are they useful?
What trouble will they cause? Will they create bigger problems?
Is it difficult to learn?

I’ll speak about the usefulness and problems in My own perspective, so let’s see if it’s idiomatic and/or difficult.

Let me show you a couple of examples that demonstrate many ways to use the library. Labels are expandable — click on them to see the code.

1. Basic usage. Trying to create a record and returning a value depending on the outcome.

# https://github.com/saintprug/rubytalks.org/blob/cb32cff14587e021e71f0e5547765e84cd014c0d/lib/domains/talks/operations/create.rb#L43-L51

def create_talk_speaker(talk_id, speaker_id)
  talk_speaker = talks_speakers_repo.create(talk_id: talk_id, speaker_id: speaker_id)

  if talk_speaker
    Success(talk_speaker)
  else
    Failure('could not create talk_speaker')
  end
end

2. Composing multiple operations. A function that fetches or creates a record. It shows chaining functions using #fmap — a method which works similar to Enumerable#map, but won't do anything if it's called on a Failure

# https://github.com/davydovanton/cookie_box/blob/c7e92db9b69b38eb85fb9d7ef1f81706ea4830e6/lib/repositories/libs/get_or_create_repo.rb#L13-L22

def call(repo_name)
  repo_name = truncate(repo_name)
  repo = repository.find_by_name(repo_name)

  if repo
    Success(repo)
  else
    info_getter.call(repo_name).fmap { |value| create_repository(value) }
  end
end

3. Composing multiple operations. If one of them fails, the method will return a Failure and rollback the transaction.

# https://github.com/saintprug/rubytalks.org/blob/cb32cff14587e021e71f0e5547765e84cd014c0d/lib/domains/talks/operations/create.rb#L17-L28

def call(talk_form) # rubocop:disable Metrics/AbcSize
  talk_form = talk_form.symbolize_keys
  oembed = yield generate_oembed(talk_form[:link])

  talk_repo.transaction do
    speakers = yield find_or_create_speakers(talk_form[:speakers])
    event = yield find_or_create_event(talk_form[:event])
    talk = yield event ? create_talk(talk_form, oembed, event.id) : create_talk(talk_form, oembed)
    yield create_talk_speakers(talk.id, speakers)
    Success(talk)
  end
end

4. Working with the computed result. Using if with predicates to handle different cases

# https://github.com/saintprug/rubytalks.org/blob/fe0a6f2c08f161e9bde9545227be6db5e1346539/lib/util/web/helpers/respond_with.rb#L9-L16

def respond_with(response, result, serializer, status: 200)
  if result.success?
    respond_with_success(response, result.value!, with: serializer, status: status)
  else
    status = fetch_error(result.failure)[:status]
    respond_with_failure(response, result.failure, status: status)
  end
end

5. Working with the computed result. Using case to handle all cases

# https://github.com/saintprug/retro-board/blob/b66a26a36cfc5ccfe8263fe0af31b3610ce2a896/apps/web/controllers/boards/show.rb#L10-L19

include Dry::Monads::Result::Mixin

def call(params)
  result = operation.call(params.to_h.slice(:id))

  case result
  when Success { |value| value.is_a?(Board) }
    @board = result.value!
  else
    halt 404, "These aren't the boards you're looking for"
  end
end

6. Working with the computed result. Using #fmap to access the wrapped data

# https://github.com/davydovanton/cookie_box/blob/c7e92db9b69b38eb85fb9d7ef1f81706ea4830e6/apps/web/controllers/decks/show.rb#L10-L17

def call(params)
  operation.call(params[:id]).fmap do |payload|
    @deck = payload[:deck]
    @issues = payload[:issues]
  end

  status 404, 'Not found' unless abilities['deck.read'].call(current_account, @deck)
end

7. Working with the computed result. Using pattern matching.

# a modification of example five, taken from saintprug/retro-board
# it shows a more "modern" syntax for include
# https://github.com/saintprug/retro-board/blob/b66a26a36cfc5ccfe8263fe0af31b3610ce2a896/apps/web/controllers/boards/show.rb#L10-L19

include Dry::Monads[:result]

def call(params)
  case operation.call(params.to_h.slice(:id))
  in Success(Board => board)
    @board = board
  else
    halt 404, "These aren't the boards you're looking for"
  end
end

Here’s a couple of things to notice about those code styles:

They use blocks more often. The interfaces use blocks or allow them. Longer chains become a norm, which goes against current Rubocop defaults.

There’s no need for unnecessary naming. Since there’s an option to easily chain transformations using #fmap and other methods, we can avoid coming up with names we don’t really need. I often feel frustrated about having to come up with names for intermediate data..

We don’t build objects using #new. Instead, we use constructors that look like Kernel#Array, Kernel#String and similar methods. It looks pretty much like the standard Ruby code.

We cherry-pick abstractions we need. This helps prevent clutter and communicate more clearly. It looks boilerplaty, so folks move the includes to base classes.

We can cherry-pick using a single include. It’s not a new pattern, but an uncommon one. Instead of using include multiple times to get each abstraction, we list whatever we need: [:result, :maybe, :try].

Conditional logic uses predicates. Nobody really reinvents the wheel, so if you need to add conditional logic — you’ve still got conditions and methods to check which value you’ve got.

There’s an extensive support for case and pattern matching. It enables us to avoid using many built-in interfaces altogether and write expressive and beautiful code. There aren’t too many examples, as the features are relatively new. If you’ve got something to share, please do!

The library introduces new semantics to yield. Conventionally, we use yield whenever we want to call a block. That’s exactly what’s going on here — we call a block. However, we bring the new semantics. Now, yield looks more like await in many languages, or similar to yield in Python or JavaScript.

The “functional” programming part is barely noticeable. None of those examples show us any mathematics-riddled word-buzzing functional programming some people expected to. Sure, it has some features: naming, using blocks, immutable expressions. That’s pretty common in Ruby.

In the end, is it idiomatic? I think so. It may be unfamiliar to many developers, but it’s idiomatic almost all the way through. Except for the yield expression, perhaps. After all, idiomatic just means following the conventions of the language.

What about exceptions?

We use Result to express the logic which may fail or not. Let’s imagine a simple use-case: user tries to log in and fills in their username/password combination. Here’s how we would handle it:

We make sure that user has provided username and password. If not, we fail
We make sure the username exists in our system. If not, we fail and suggest them to register
We make sure the password is correct.
- If not, we try to check if it is one of the user’s old passwords. If so, fail and tell them they’ve changed passwords a while ago.
- If it’s just an unknown password, fail and tell them the password is incorrect.
Make sure the user is not blocked in our system. Fail and warn them about it if they are
Create new session

This is a common, yet complex example. It consists of multiple steps and has six outcomes: five unsuccessful and one successful.

A lot of programmers would design this process using exceptions. They’d create five exception classes and just raise them on demand. The higher-level code would have to rescue them and handle as necessary.

Result is an alternative approach to this design. It suggests that you return a Success or a Failure with whatever data you need. The higher-level code would have to handle it using whatever technique they want – usually it’s a combination of Result’s built in methods and some if, case and case in calls.

Which way to choose is actually a matter of preference. How does your code make decisions? Do you need performance? Should we treat those errors like we treat out of memory and HTTP errors? Are errors data? What approach suits you better? We can’t answer all of those questions rationally. Trust me, I’ve tried.

Sure, there are differences and similarities of the two techniques that might help us make a more informed decision. However, they deserve their own article. I’m working on it, so I’m going to just cut it short:

Exceptions are not an adequate replacement for Result, nor is Result an adequate replacement for exceptions. At least in Ruby

Those things serve different purpose:

Result is a way to express your data
Exceptions are a control flow mechanism

You may replace one with another, but it might be better for them to coexist.

Remember I said about Rust, Kotlin, Swift, F#, OCaml/ReasonML, Haskell and Elm? Some of them have both Result and exceptions. I encourage you to take a look at how those two ideas coexist in those languages and learn from it. Perhaps, it could bring your Ruby code to the next level.

My own perspective

I’m a fan of simple, yet efficient designs. Life is harder than that, so I have to deal with complex designs most of the time.

When I’m dealing with something complex, I want to work with things that don’t confuse me. Yet, I’m ready to learn something new, especially if it’s something fundamental.

I’m a domain-driven design enthusiast. I’m always looking for better ways to communicate my intentions via code as clearly as possible. Monads help me with this – they make my intentions clear.

In What is a monad I’ve listed four different monads and their purpose. In my business logic, They help me answer the main question: “what’s going on here?”. I return Success(...) when everything is good, I Try to run some code which may fail with an error, I perform an asynchronous Task, and I will Maybe get a value when I query the database.

Sure, I can pretend I’m writing Haskell and write code that’s hard to read even for someone who’s experienced with monads. It’s always up to me.

I’ve been using monads since May/June 2016, and they’ve become as natural as if, Array and other built-in parts of the language. The learning curve was not that steep, but it took me over two years of mindless usage to somewhat understand them. I was using them blindly – just returning Result and Maybe from functions and using whatever methods they provided. I want to emphasize that you can get at least that long without needing to read about category theory.

If you’re still skeptical about “having to learn” it, stay with me. There’s a website called “F# for fun and profit”, which is a source of many delightful resources on functional programming. Even though the Result monad is a crucial part of many concepts described there, the word “monad” is actually banned from this site.

There’s a problem with a lot of monad tutorials, including mine — they focus on maths instead of the practical application. This make us sound and act like we’re enlightened, like we’ve just had a revelation. We’re trying to speak about it because now we get it. Regular folks don’t get it and think we’re some kind of a cult.

In reality, monads are just simple building blocks. They’re a nice addition to your toolset, but you don’t need to know that they are monads. It’s not important in the grand scheme of things — they only enable you to build greater things, like railway oriented programming.

Another example of a better naming for monads is Python’s returns, which was heavily inspired by Ruby’s dry-monads. They don’t use the name monad anywhere. It’s just a library of primitives to write business logic. Nothing about endofunctors and category theory – that’s what I love about them.

Monads sound scary and off-putting. They’re not that scary, and there’s nothing special about them. They just let us do greater things. Let’s move on and talk about that.

How do I persuade my team to use monads

I had to answer it for the first time when Janko approached me after EuRuKo 2018 to talk about it. I had no idea back then. It’s safe to say that I’ve learned something and can now speak about it.

When people speak about introducing monads, they usually speak about using Result to organize domain logic. I’m going to focus on that.

Here’s the thing:

Monads are not the goal. Think bigger

As you might have read in What is a monad, monads are just primitive tools that let you express something. Do you really want to get stuck on such details? I thought I did.

However, just using this piece of tech without seeing a bigger picture will likely be fruitless or even frustrating. Please, don’t follow this path unless you have the authority to just do it. You wouldn’t have needed this article in this case though, would you?

Let’s try and see the bigger picture here. What exactly are you trying to solve?

Are you stuck with plain old fat models / fat controllers?
Are you struggling with exceptions in domain logic?
Do you need better performance than exceptions give you?
Do you want to introduce railway-oriented programming?
Are you frustrated with your current interactor / use case / service object implementation?
Do you just want to bring something new to your project to make it more appealing?
Do you want your development team to get out of their comfort zone and persuade them to learn something new?

Once you’ve answered those questions honestly, we can move on to the next question.

What larger problems does it solve?

Please take your time to carefully assess the problems your project and business is facing. Talk to your peers, people from other departments and upper management. Talk about their problems, what they’d love to see and maybe you’ll find out that you’ve got the solution. Perhaps not a solution but a part of it.

Lots of things are easier when you've got allies

Once you’ve done those steps, you should have enough evidence that your project needs some change. It may seem like monads are the only solution. Trust me, they’re not.

What are the alternatives? How good would they fit your situation?

Perhaps, you’d be better off with things like Interactor, ActiveInteractor. They let you implement the same things you could with Result, but in a little different fashion.

If you're already using Interactor or ActiveInteractor, you might just patch the Context class to make it behave like a Result

If you’re thinking about something greater, you might consider other kinds of architecture or even something like Eventide. In this case, you’ll probably have to plan out the great refactoring, plan it out and try and push the idea. I’m in no position to give advice here, as it’s a pretty complex task and I can’t generalize it.

If you’re still looking for a way to organize domain logic, you’ll have to learn and teach. Best way to learn is to practise: build a couple of tiny playgrounds. Try using Result to organize the domain logic. Speak about it on dry-rb chat.

Once you’ve learned enough to hold a conversation, gradually introduce your team to the new approaches. It works well if you have a designated space for sharing knowledge – talks, articles, tutorials, etc. If you don’t, you might as well just create one. It’s a nice improvement to your engineering culture.

The answer

I’ve covered a lot of ideas in a way that looks like a rant, but haven’t gotten to the main point: “should I really use monads?”. Here’s the answer:

It depends.
Will they help you do greater things? Then the answer is yes.
Do they seem off-putting and less enjoyable than other things? The answer would be “no”.

It’s okay if you haven’t understood the new tool yet. It’s also okay if you didn’t like it.

Here’s the thing: the question is actually more of a stylistic preference than something substantial. Decision to use a monad or not is a lot like the decision to use if instead of case, or collect instead of reduce. Just like pattern matching lets you express yourself, each monad contributes to expressiveness to your code.

The bottom line: monads are alright. They’re just a low-level detail that helps you build something greater. Yet, there are other solutions. Focus on the greater things and use whatever suits you better. Meanwhile, be open-minded about the alternatives. They’re alright too.

Links and references

dry-rb 1.0: upgrading validations, types and schemas

2019-05-31T06:48:00+00:00

I’m enthusiastic about dry-rb gems. Actually, I’ve never worked on Ruby projects without a dry-rb gem. However, some people are sceptical, as a lot of core dry-rb gems are still in their 0.x phase, which leads to a lot of breaking changes and hours of refactoring.

I’m happy to see dry-rb mature: dry-monads entered 1.0 phase in Summer 2018, and now two more libraries hit v1.0 milestones: dry-types and dry-struct; and dry-validation is in its 1.0 RC phase.

I haven’t updated my dry-rb gems for a couple of months, so I’ve missed a lot of breaking changes. Finally, I decided to upgrade the gems and write about the process. I’ll take a swing at automating my upgrade process as much as I can.

Prerequisites

Here’s what my dry-rb gems look like:

$ bundle list | grep dry
  * dry-auto_inject (0.4.6)
  * dry-configurable (0.7.0)
  * dry-container (0.6.0)
  * dry-core (0.4.7)
  * dry-equalizer (0.2.1)
  * dry-events (0.1.0)
  * dry-inflector (0.1.2)
  * dry-initializer (2.5.0)
  * dry-logic (0.4.2)
  * dry-matcher (0.7.0)
  * dry-monads (1.2.0)
  * dry-struct (0.6.0)
  * dry-transaction (0.13.0)
  * dry-types (0.13.2)
  * dry-validation (0.12.1)

I’ve got 15 gems, but I only care about four of them: monads, types, struct and validation. Since monads are up-to-date, I’m only going to talk about types, struct and validation.

In this post, I’ll try to give a step-by-step guide that will simplify the upgrading process. It won’t give a 100% working solutions, but it will probably save you a couple of hours.

Note. I use macOS with GNU sed (gsed) instead of built-in sed command. So if you want to follow my instructions, install it via brew install gnu-sed. Since I’m using fish instead of bash / zsh, some commands might need slight modifications to work.

Note. I wrote this article while upgrading the dry-rb gems on my project. I decided to do it gradually — so you might encounter some redundant steps. If you do, please contact me via email and I’ll upgrade it.

dry-validation to dry-schema

The gem we knew as dry-validation has evolved from a complex schema validation & coercion into a high-level contract DSL with domain logic.

Meanwhile, it has become so complex they decided to break it down into two gems: dry-validation and dry-schema. The latter provides the old functionality of dry-validation — the schema validations, coercions, and they fixed all known issues. dry-validation adds domain rules and validations on top of that.

I don’t want to go around and update everything manually, so I’m going to replace dry-validation with dry-schema as much as I can, and manually refactor the rest.

Step 1. Upgrade dry-validation to 0.13. It’s the last version before the switch, so if your builds pass — you’re good to go. You’ll have to update dry-types to 0.14 too.

Step 2. Replace dry-validation with equivalent dry-schema version (0.1.0) and replace all Dry::Validation occurrences with Dry::Schema. Also replace all Dry::Validation.Schema with Dry::Validation.define.

$ bundle remove dry-validation && bundle add dry-schema --version 0.1.0`
$ grep -rl 'Dry::Validation' ./**/*.rb | xargs gsed -i 's/Dry::Validation/Dry::Schema/g'
$ grep -rl 'Dry::Schema.Schema' ./**/*.rb | xargs gsed -i 's/Dry::Schema.Schema/Dry::Schema.define/g'

If you’ve used struct extension, don’t forget to search for Dry::Schema.load_extensions and remove :struct from the list.

Step 3. Replace .each(&:type?) predicates with .each(:type?). The same goes for maybe, filled and value. You might get ArgumentError: no receiver given if you don’t.

$ grep -rl 'Schema' ./**/*.rb | xargs gsed -i 's/\.\(filled\|value\|each\|maybe\)(&/.\1(/g'

Step 4. Refactor schemas that use arrays as input.

The feature has been removed and it’s not coming back until dry-schema 1.0. Here’s an issue with the feature.

The refactoring will look like this:

# Before

ItemSchema = Dry::Schema.Params do
  each do
    schema do
      required(item_id).filled(:int?)
      required(option_ids).each(:int?)
    end
  end
end

ItemSchema.call(input)

# After

ItemSchema = Dry::Schema.Params do
  required(:input).each do
    schema do
      required(:item_id).filled(:int?)
      required(:option_ids).each(:int?)
    end
  end
end


ItemSchema.call(input: input)

Step 5. Check you’ve ever inherited from Dry::Validation schemas. If you did, do the following transformations:

Rename classes

Dry::Validation::Schema::Params → Dry::Schema::Params
Dry::Validation::Schema::JSON → Dry::Schema::JSON
Dry::Validation::Schema → Dry::Schema

Replace define! block with define
Move configure block under define

# Before:

class ApplicationSchema < Dry::Validation::Schema::Params
  configure do
    config.messages = :i18n
  end
end

# After:

class ApplicationSchema < Dry::Schema::Params
  define do
    config.messages = :i18n
  end
end

And update its subclasses:

# Before

class MySchema < ApplicationSchema
  configure do
    config.messages = :yaml
  end

  define! do
    ...
    # your params go here
  end
end

# After

class MySchema < ApplicationSchema
  define do
    config.messages = :yaml

    ...
    # your params go here
  end
end

Step 6. Update DSL inheritance.

Replace Dry::Schema.Params(BaseClass) with Dry::Schema.Params(parent: BaseClass).

$ grep -rl 'Schema' ./**/*.rb | xargs gsed -i 's/Dry::Schema\(\(::\)\|\.\)\(Params\|JSON\|Schema\)(\([[:alnum:]]*\))/Dry::Schema\1\3(parent: \4)/g'

Before you proceed Skip steps 7 and 8 if you’ve never used type specs API.

Step 7. Remove config.type_specs from your schemas

$ grep -rl 'config.type_specs' ./**/*.rb | xargs gsed -i '/config\.type_specs/d'

Step 8. Remove type spec usages from required and optional.

$ grep -rl 'Schema' ./**/*.rb | xargs gsed -i 's/\(required\|optional\)(\(:[[:alnum:]_]*\), [[:print:]]*)\(\.\|$\)/\1(\2)\3/g'

Updating dry-schema to 0.3

Step 9. Update your gemfile to specify gem 'dry-schema', '~> 0.3.0' and run bundle install

Step 10. If you’re using I18n, move errors under dry_schema namespace. This way,

en:
  errors:
    array?: must be an array

will turn into

en:
  dry_schema:
    errors:
      array?: must be an array

Step 11. Find any schema macro usages and replace them with hash, as schema no longer prepends value(:hash?) check.

# Before

required(:foo).schema do
end

# After

required(:foo).hash do
end

$ grep -rl 'Schema' ./**/*.rb | xargs gsed -i 's/schema \(do\|{\)/hash \1/g'

Step 12. Find any each macro usages and replace them with array to add type check. Since Ruby has a Enumerable#each function, we can’t automate it, but we can still find possible occurrences:

$ grep -rl 'Schema' ./**/*.rb | xargs grep -n 'each \(do\|{\)'

Feel free to skip if you feel like you don’t need type checks.

Step 13. Load hints extension if you use monads or .messages.

Dry::Schema.load_extensions(:hints)

The leap towards 1.0.0

Step 14. Update dry-struct, dry-types and dry-schema and run bundle install.

gem 'dry-schema', '~> 1.1.0'
gem 'dry-struct', '~> 1.0.0'
gem 'dry-types', '~> 1.0.0'

Step 15. Replace Dry::Types.module with Dry.Types(default: :nominal)

If you’ve never used nominal types (i.e. Types::Hash, Types::Integer), feel free to use Dry.Types instead.

$ gsed -i 's/Dry::Types\.module/Dry.Types(default: :nominal)/g' ./**/*.rb

Step 16. Replace legacy hash schemas with new ones. See https://dry-rb.org/gems/dry-types/0.15/hash-schemas/

Step 17. Update error message config

Replace config.messages with config.messages.backend
Replace config.messages_file = '/path/to/my/errors.yml' with config.messages.load_paths << '/path/to/my/errors.yml'
Replace config.namespace = :user with config.messages.namespace = :user

$ grep -rl 'Schema' ./**/*.rb | xargs gsed -i 's/config\.messages =/config.messages.backend =/g'
$ grep -rl 'Schema' ./**/*.rb | xargs gsed -i 's/config\.messages_file =/config.messages.load_paths <</g'
$ grep -rl 'Schema' ./**/*.rb | xargs gsed -i 's/config\.namespace =/config.messages.namespace =/g'

Step 18. Symbolize all string keys

$ grep -rl 'Schema' ./**/*.rb | xargs gsed -i 's/\(required\|optional\)(\(\'\|"\)\([[:alnum:]_]*\)\(\'\|"\)/\1(:\3/g'

Step 19. Replace Types.Definition with Types.Nominal

$ gsed -i 's/Types\.Definition/Types.Nominal/' ./**/*.rb

Step 20. If you rely on Types::Params and Types::JSON not to raise an exception on invalid input, decorate the definitions with .lax

$ gsed -i 's/Types::JSON::\([[:alnum:]]*\)/Types::JSON::\1.lax/g' ./**/*.rb
$ gsed -i 's/Types::Params::\([[:alnum:]]*\)/Types::Params::\1.lax/g' ./**/*.rb

Step 21. Replace :type? predicates with type checks wherever you need this

$ grep -rl 'Schema' ./**/*.rb | xargs gsed -i 's/\(filled\|maybe\|value\)(:str?/\1(:string/g'
$ grep -rl 'Schema' ./**/*.rb | xargs gsed -i 's/\(filled\|maybe\|value\)(:int?/\1(:integer/g'
$ grep -rl 'Schema' ./**/*.rb | xargs gsed -i 's/\(filled\|maybe\|value\)(:date?/\1(:date/g'

Step 22. Result#{messages, errors, hints} now return MessageSet, which can be converted to Hash. So we need to go and update the usages everywhere. Also Result#to_monad now wraps entire Result object, so we have to update our code.

# Before

render errors: Schema.call(params).errors
render errors: Schema.call(params).to_monad.failure

# After

render errors: Schema.call(params).errors.to_h
render errors: Schema.call(params).to_monad.failure.errors.to_h

I’ve used the scripts to help me look and trace those values:

$ grep -rl 'Schema' ./**/*.rb | xargs grep -n '\.messages'
$ grep -rl 'Schema' ./**/*.rb | xargs grep -n '\.errors'
$ grep -rl 'Schema' ./**/*.rb | xargs grep -n '\.to_monad'
$ grep -rl 'Schema' ./**/*.rb | xargs grep -n '\.failure'
$ grep -rl 'Schema' ./**/*.rb | xargs grep -n '\.value_or'
$ grep -rl 'Schema' ./**/*.rb | xargs grep -n '\.value!'

Refactoring to dry-validation

The steps above should be good enough to update most of the features, but if you ‘ve ever used high-level rules, validation blocks, you have two options: either remove those features from your schemas, or use dry-validation 1.0. I decided to refactor most of my schemas, that’s what came out of it.

There are things to keep in mind during the update:

dry-validation is a library to validate domain logic and rules. The core concept is a Contract.
All contracts must be instantiated — no more Schema.call. We need to use Contract.new.call now
The idiomatic way to define a contract is to use standard Ruby syntax: class Contract < Dry::Validation::Contract as opposed to dry-schema’s Dry::Schema.Params { }

Step 23. Update dependency injection. The new version uses dry-initializer under the hood, so it works like this:

use option for keyword arguments
use param for positional arguments

You’ll have to pass the arguments when you instantiate the method

# Before

Schema = Dry::Validation.Schema do
  configure do
    option :repo
  end

  ...
end

Schema.with(repo: my_repo).call(params)

# After

class Contract < Dry::Validation::Contract
  option :repo
end

contract = Contract.new(repo: repo)
contract.call(params)

I used the script to find the files I need to refactor:

$ grep -rl 'Schema' ./**/*.rb | xargs grep -n 'option :[[:alnum:]_]*$'

Step 24. Rewrite rules and validations. I can’t provide a comprehensive migration guide because I’ve just refactored everything and tried to make my specs pass without giving it much thought.

# Before


class CreditCardSchema < Dry::Validation::Schema::Params
  configure do
    config.type_specs = true
  end

  define! do
    required(:number, :string).filled(format?: /\A\d{13,19}\z/)
    required(:month, :string).filled(format?: /\A(0?[1-9]|1[012])\z/)

    validate(expired: %i[year month]) do |year, month|
      Date.new("20#{year}".to_i, month.to_i).end_of_month >= Date.current
    end
  end
end

# After
class CreditCardSchema < Dry::Validation::Contract
  params do
    required(:month).filled(:string, format?: /\A(0?[1-9]|1[012])\z/)
    required(:year).filled(:string, format?: /\A\d{2}\z/)
  end

  rule(:year, :month) do
    year = values[:year]
    month = values[:month]

    if Date.new("20#{year}".to_i, month.to_i).end_of_month < Date.now
      key(:expired).failure(:expired)
      # ^ a little duplication here to produce the expected error message
      # without refactoring anything else
    end
  end
end

I used this script to search for all schemas that need rewriting:

$ grep -rl 'Schema' ./**/*.rb | xargs grep -n 'rule\|validate('

Step 25 (optional). If you’re using Reform, you’re in for a disappointment, especially if you’ve been using its dry-validation DSL.

We have Reform 2.2.4 with ActiveModel validations, so we forked it and removed all the dry-validation stuff. Feel free to fork and use!

Step 26. Fix the rest of failing specs. All done!

Recap

The upgrade process took me about 3 work days of refactoring, and I was glad I learned basic sed to help me — it’s annoying to do so much manual work.

However, I think the improvements are worth it. The ones I like the most:

dry-types is stricter and less verbose now — if you’re not including nominal types, then Types::String is the same as Types::Strict::String
The known dry-validation bugs were fixed
Decreased complexity of schema validations
New library to design domain validations and contracts

I urge you to try the new dry-rb gems — and write about your experience. If you’ve upgraded your gems and wrote a post about your journey and update process — please send me an email and I’ll add a link to your page. And of course, it would be great to see new contributions to official docs.

References

Update (01.06.2019). flash-gordon pointed out that you don’t need to wrap config into the configure block. So I’ve replaced

define do
  configure do
    config.xxx = yyy
  end
end

with a less nested version:

define do
  config.xxx = yyy
end

Update (07.06.2019). solnic pointed out that I made a typo in Step 10: it used to say dry_struct instead of dry_schema. I’ve updated the step accordingly.

Partial application in Ruby

2019-01-11T21:00:15+00:00

Ruby is a multi-paradigm language with a strong bias towards object-oriented programming. You can argue that its design is influenced by Alan Kay and Smalltalk, as opposed to C++/Java-style object-oriented languages. Thankfully, this object-oriented design doesn’t mean we can’t use ideas from functional programming. There’s a small list of functional traits in Ruby:

Expression-oriented syntax
Geeky names for Enumerable methods: filter, map, reduce, flat_map
Idiomatic monads
Railway oriented programming
lambdas and procs
… I can go on and on

There’s also one specific empowering feature: built-in support for partial application. In this article, I want to talk about implementation and use-cases for partial application in Ruby.

Proxy methods

Let’s say we have a function of two arguments. As an example, we’ll take a function that accepts two strings: tag and text and formats them to look like [tag] text. The source code for this function would be fairly simple:

def tagged_string(tag, str)
  "[#{tag}] #{str}"
end

tagged_string('INFO', 'Hello, World!')
# => '[INFO] Hello, World!'

Let’s say we build a logger that only uses 3 tags: INFO, WARNING, and ERROR; and implements only one method, which logs to stdout.

class Log
  def write(tag, string)
    print(tagged_string(tag, string))
  end
end

So, to use it, we have to always call Log.new.write('INFO', 'log data'), which is not convenient. Besides, what happens if we add different log levels? What if we make a typo, or write DEbUG instead of DEBUG?

Let’s improve the code and write a couple of methods that will help us avoid any mistakes with the tags.

class Log
  def info(string)
    write('INFO', string)
  end

  def warning(string)
    write('WARNING', string)
  end

  def error(string)
    write('ERROR', string)
  end
end

Now, instead of a single method with two arguments, we have three methods that accept one argument. Those methods are just proxies to the original method as they don’t have any extra logic; they just fix the first argument of the original method. This technique is called partial application.

Partial application is the process of fixing a number of arguments to a function, producing another function of smaller arity

This Wikipedia definition explains what we’ve done here:

We have a function (write) that accepts two arguments — which means, its arity is 2
We defined functions error, warning, and info that accept only one argument, so their arity is 1
Those functions only pass their input to write write — we fixed the first argument and passed the rest

Whenever we make a function that only calls another one, but requires fewer arguments, we can talk about partial application. In practice, we use it to reduce boilerplate, encapsulate logic, and make our lives easier. I bet you’ve used it countless times already, but probably never considered that this “pattern“ might have a name.

Without methods

What if we don’t want to extend our class, but still want to use partial application? Our only solution is procs. Personally, I prefer to use lambdas for the task. Let’s see how they work.

We have a Log with a simple public interface: write(tag, string). I’m building a system which requires its own tag: SECURITY, but I will only need to use it in one class.

In this case, I would do something like this:

Instantiate the Log and save the object into a variable
Define a lambda that uses the object and calls #write on it
Use the lambda whenever I want to log something

This is how it looks in a class:

class SecurityService
  attr_reader :log, :logger_instance

  def initialize(logger_instance)
    @logger_instance = logger_instance
    @log = ->(message) { logger_instance.write('SECURITY', message) }
  end

  def call
    log.call('Hello, World!')
  end
end

SecurityService.new(Log.new).call
# => will print "[SECURITY] Hello, World!"

We managed not to define any extra methods, but we still had to manually create a lambda. What if we could avoid it? Then the code would be a little bit simpler:

@log = logger_instance.write('SECURITY')

log.call('Hello, World!')

Unfortunately, Ruby doesn’t work this way and we’ll just get an exception. However, it gives us a couple of tools to implement what we want.

Metaprogramming

Let’s use Ruby’s metaprogramming to write a helper will enable us to pass fewer arguments to our methods. Here’s how it would work:

You pass a function to the helper
Helper returns a modified function
If we call the modified function and provide all arguments, it works as usual
If we provide fewer arguments than required, we get a new function that requires the rest of the arguments

This is how it would look like:

enable_partial_application = ... # our helper

fun = -> (x, y) { x + y } # our function

new_fun = enable_partial_application.call(fun)

plus_two = new_fun.call(2) # => new function
plus_two.call(3) # => 5
plus_two.call(10) # => 12

new_fun.call(4, 3) # => 7

Sounds cool, right? Let’s see how we can implement this in Ruby. I’m going to pollute global namespace and define a method enable_partial_application that accepts a function and returns a wrapper function.

def enable_partial_application(fun)
  ->(*args) {
    fun.call(args)
  }
end

Let’s start adding logic piece-by-piece. Here’s first piece of logic: “If we provide enough arguments, we call the original function”. To do so, we need to know exactly how many arguments the function requires — so we use the built-in method #arity, which gives us the number.

def enable_partial_application(fun)
  arity = fun.arity

  ->(*passed_args) {
    # I use `<=` instead of `==` because I want Ruby to
    # handle cases when there are too many arguments.
    if arity <= passed_args.count
      fun.call(*passed_args)
    else
      # ???
    end
  }
end

Alright, let’s handle the case when there are fewer arguments than required. We’ll return a new function that remembers our previous input:

def enable_partial_application(fun)
  arity = fun.arity

  ->(*passed_args) {
    if arity <= passed_args.count
      fun.call(*passed_args)
    else
      ->(*args) {
        fun.call(*passed_args, *args)
      }
    end
  }
end

Okay, now we can test it:

fun = ->(x, y) { x + y }
new_fun = enable_partial_application(fun)

plus_two = new_fun.call(2)
plus_two.call(3) # => 5
plus_two.call(10) # => 12

new_fun.call(4, 3) # => 7

It works, alright. Let’s check out functions with more arguments:

fun = ->(x, y, z) { x + y + z }
new_fun = enable_partial_application(fun)

new_fun.call(2, 3).call(3) # => 8

plus_two = new_fun.call(2)
plus_two.call(3, 1) # => 6

plus_two.call(3).call(1) # => ArgumentError

# (wrong number of arguments (given 2, expected 3))

The last line fails because we need to make enable_partial_application work recursively. We can fix this by updating two lines:

def enable_partial_application(fun)
  arity = fun.arity

  apply = ->(*passed_args) { # <=
    if arity <= passed_args.count
      fun.call(*passed_args)
    else
      ->(*args) {
        apply.call(*passed_args, *args) # <=
      }
    end
  }
end

Now it works like a charm — the resulting function keeps calling itself until the user has provided enough arguments.

One last thing though. It doesn’t work with functions that take a dynamic number of arguments because fun.arity returns a negative value. It’s a weird [built-in behavior(https://ruby-doc.org/core-2.2.0/Proc.html#method-i-arity) of procs. There’s a simple fix:

arity = fun.arity

# replace with

arity = fun.arity.positive ? fun.arity : -fun.arity - 1

Finally, we’ve got a working helper that enables partial application for any function in Ruby. The final result:

def enable_partial_application(fun)
  arity = fun.arity.positive ? fun.arity : -fun.arity - 1

  apply = ->(*passed_args) { # <=
    if arity <= passed_args.count
      fun.call(*passed_args)
    else
      ->(*args) {
        apply.call(*passed_args, *args) # <=
      }
    end
  }
end

Built-in method

Thankfully, we don’t need to build those helpers. Ruby has a built-in method called #curry, which works on methods and procs, and does everything I’ve described above.

fun = ->(x, y, z) { x + y + z }
new_fun = fun.curry

new_fun.call(2, 3).call(3) # => 8

plus_two = new_fun.call(2)
plus_two.call(3, 1) # => 6

plus_two.call(3).call(1) # => 6

This method takes its name from currying, a process of transforming a single function of N arguments into N functions that only take a single argument. It’s a technique to assist partial application in statically typed functional languages like Haskell, OCaml, and F#. It’s a topic for a separate article so I won’t mention the details.

Recap

Partial application helps us fix values and pass fewer arguments
We use it quite often, even if we don’t do it explicitly
If a function has a variable number of arguments, its arity is negative
We need a recursive function to build our own partial application
Ruby comes with partial application out of the box: Proc#curry and Method#curry
Currying is less performant than plain methods/procs

Note: if you want to try currying, please keep in mind that it’s not a popular pattern in Ruby, so your colleagues might be skeptical about it. However, I urge you to try it out and compare with the conventional partial application.

References

Wikipedia article on partial application
Partial function application for humans by Andrew Berls
Hidden partial application in Ruby
Docs for Proc

Monad laws in Ruby

2018-09-08T10:00:00+00:00

I’ve been using monads in Ruby since May 2016, but I haven’t really understood the theoretical basis for them. I thought about learning Haskell, but I gave up pretty soon: I didn’t think I would benefit from it. Moreover, we started using ReasonML in Planado, which improved my functional programming skills to the point I didn’t really need a new functional language in my life. Why bother with learning Haskell when you know Ruby and Reason, right?

In early 2018, I became curious about theoretical aspects of functional programming, especially the monad laws. That’s when I realized that I really needed Haskell, mainly because everyone used it in their articles. It was extremely annoying because I couldn’t even read the code. How was I going to apply those things in Ruby if I can’t even understand what they’re saying? So I got a little help.

I grabbed my laptop and a friend who knows Haskell and figured out how to describe the three monad laws using Ruby’s dry-monads gem.

Monads

Monad is a concept from category theory. Some people describe it as a “monoid in the category of endofunctors”, some call it “computation context”, and some just call them “result objects”. I believe that each of those definitions is correct to some extent. However, neither of them explain the practical side of monads.

As of September 2018, dry-monads gem contains 5 monads:

Maybe — for nil-safe computations
Result – for expressing errors using types and result objects
Try – to describe computations which may result in an exception
List – for idiomatic typed lists
Task – for asynchronous operations

I guess that Result is the most popular monad in Ruby, especially since railway-oriented programming has become such a hot topic in Ruby. So I will use it to describe what’s going on.

Result

Result, also known as Either, is a monad helpful for building computations that might fail at some point. It is one of the most important parts of railway-oriented programming. Result has two constructors: Failure(a) and Success(b). Both of those constructors encapsulate a value of type a or b.

Result has a lot of useful methods, but there’s one that’s the most important: #bind – an essential part of monads. It lets us compose computations by applying a block to a value inside the Success.

require 'dry/monads/result'
extend Dry::Monads::Result::Mixin

def foo(x)
  Success(x).bind do |value|
    Success(value ** 2)
  end.bind do |value|
    if value > 50
      Failure(:number_too_large)
    else
      Success(value)
    end
  end
end

foo(5)
# => Success(25)

foo(10)
# => Failure(:number_too_large)

A couple of things to keep in mind when working with #bind:

Failure#bind doesn’t do anything – it’s a no-op. Use Failure#or as an alternative.
The block must return a Result. Technically, it can return any value – a number, a string, a Maybe monad – but your code will break if you fail to follow the rule.

Three axioms

Practically, a monad is a data type which obeys three axioms called ”monad laws”:

Left identity: return a >>= f ≡ f a
Right identity m >>= return ≡ m
Associativity: (m >>= f) >>= g ≡ m >>= ( \x -> f x >>= g)

Those things sound pretty basic when you know Haskell and category theory, but might get extremely complicated if you don’t.

The first problem I’ve had with those laws: I couldn’t even read them because I didn’t know haskell. Here’s a cheatsheet that helped me read and understand the formulae:

≡ means that expressions are the same
return is a default constructor. For Result, return is the #Success method
>>= is a bind operator. In Ruby, it’s a method #bind.
\x -> ... is an anonymous function. Read -> (x) { ... }
f is a function that accepts a value and returns Result
m is a value of type Result

Left identity

Left identity is an axiom which states that return a >>= f is identical to f a.

To see what that means, let’s say we have a function f:

f = -> (x) { Success(x ** 2) }

There are two ways to call use the function:

Call it using plain Ruby
Wrap an argument into a monad and pass the function to #bind

The law says that those are equal:

Success(5).bind(&f) # => Success(25)
f.(5) # => Success(25)

Voilà! That’s it. Putting the value in the default context (Success) and feeding it to a function is the same as applying the function to the value.

What it means:

there’s nothing special about #bind – it’s just a fancy method call
if you need to use a monadic function, you don’t need to wrap the argument into a monad

Right identity

Right identity states that m >>= return is the same as m.

That means that if we have a Result object and try to bind it to a #Success, the operation won’t change anything.

Success(2).bind(&method(:Success))
# => Success(2)

Success(2).bind(&Dry::Monads::Success)
# => Success(2)

Failure(2).bind(&method(:Success))
# => Failure(2)

I haven’t figured out the practical value of this yet. If you have any ideas, send me an email at igor@morozov.is.

Associativity

The fanciest of the three, associativity axiom states that (m >>= f) >>= g and m >>= ( \x -> f x >>= g) are the same.

The trickiest part for me was \x -> f x >>= g, which turned out to be an anonymous function which accepts x and has a body f x >>= g.

This is how the Ruby equivalent of the law would look like:

# prerequisites

m = Success(2)

f = -> (x) { Success(x ** 2) }
g = -> (x) { x < 50 ? Success(x) : Failure(:number_too_large) }

# (m >>= f) >>= g

(m.bind(&f)).bind(&g)
# => Success(4)

# m >>= ( \x -> f x >>= g)

m.bind do |x|
  f.(x).bind(&g)
end # => Success(4)

To put it the other way: if you have a chain of computations, it doesn’t matter how you nest them – the result would always stay the same.

Recap

A monad is a powerful construct from category theory which can be used as mathematically sound result objects. In Ruby, dry-monads is the de-facto standard gem, which gives us the Result (Either), Maybe, Task, Try and List monads.

To be called a monad, the data type must conform to three axioms called “monad laws”:

Left identity: wrapping a value into a monad and binding it to a function is the same as applying the function to the value.

Right identity: feeding a monadic value to a default constructor doesn’t do anything.

Associativity: you can nest your computations and binds however you like.

While those laws have little to no practical value for a casual user, reading about the principles behind it all might help you join the world of functional programming and category theory. Definitely helps me!

Cheers!

References

Do Notation in Ruby: Railway Oriented Programming

2018-05-27T10:25:00+00:00

Railway oriented programming is a design pattern which helps us handle errors in our applications. Instead of relying on exceptions, we design our data and functions in a specific way. Since applications are essentially just a combination of steps, we’ll make some design decisions about those steps and their structure:

There is a Result type, which can be either a Success or a Failure
Success and Failure are practically containers with different data
Steps accept Result and return Result
Once a step returns Failure, we stop further execution

I want to emphasize that Result is just an alternative name for the Either monad. Railway Oriented Programming comes from functional programming, so it is tightly related to the usual FP concepts like monads, composition, and many others. However, you don’t need to have an extensive knowledge of monads to use ROP in your code. In this article, I’ll show you how to write railway-oriented code in Ruby.

ROP with dry-transaction

There is a dry-transaction gem which provides a DSL to build railway-oriented business transactions. The core part of the gem is dry-monads which provides the Result type and tools to work with it.

To create a railway-oriented operation, we’ll need to do a few things:

Create a class and include Dry::Transaction
Define a few methods that return either Success or Failure
Use step adapters to chain those methods together

Then, we can instantiate the class and pass any input to the #call method.

Here’s how it looks like:

class MyOperation
  include Dry::Transaction
  include Dry::Monads

  step :validate
  step :log
  step :persist

  def validate(data)
    if data.valid?
      Success(name: data.name, age: data.user_age)
    else
      Failure("something went wrong")
    end
  end

  def log(name:, **rest)
    print("User name is #{name}")

    Success(name: name, **rest)
  end

  def persist(name:, age:)
    ...
    # some business logic here
    ...
    Success(name: name, age: age)
  end
end

MyOperation.new.call(...)
# ^ can return either
# Success(name: ..., age: ...)
# or Failure("something went wrong")

As you can see, we use class-level step method to compose the validate, log and persist methods. Failure returned from validate halts the further execution.

Pros of the approach:

It’s plain Ruby
It allows us to reuse steps
It works!
Chained methods don’t need to unwrap the input

Cons of the approach:

The DSL has a weaker control over the program’s flow — we can’t have conditions unless we add a special step
The Result object that we pass around keeps accumulating data and becomes enormous, so we have to use **rest in our function signatures
Database transactions were hard to implement until around step came around. Still awkward, though

dry-monads to the rescue

Since dry-transaction is based on dry-monads, we could probably build something ourselves, right?

Result has a few methods to help us chain those monads:

bind applies unwrapped Success value to the block, which should return a Result object. No-op on Failure
fmap is similar to bind, but wraps the returned value into Success
or is similar to bind, but only applies Failure values
or_fmap is similar to or, but wraps the returned value into Success
tee does the same thing as bind, but returns input if the result is a Success
success? and failure? tell us which kind of Result it is
value_or extracts the value from Success or returns fallback value

This is how the same example would look like using raw monads:

class MyOperation
  include Dry::Monads

  def call(data)
    validate(data).bind(method(:log)).bind(method(:persist))
  end

  def validate(data)
    if data.valid?
      Success(name: data.name, age: data.user_age)
    else
      Failure("something went wrong")
    end
  end

  def log(name:, **rest)
    print("User name is #{name}")

    Success(name: name, **rest)
  end

  def persist(name:, age:)
    ...
    # some business logic here
    ...
    Success(name: name, age: age)
  end
end

The differences:

Plain methods instead of a DSL
Better control over flow of our application: more ways to add branching

However, there are some disadvantages to the approach

Having to use #method is hideous. Solution? Callable objects
We still have to pass all parameters to each function
Complex logic gets awkward as we add more steps to the chain
It doesn’t halt execution if a function returned a Failure, so we’ll have to work around that

Since 1.0.0.beta1 of dry-monads, there’s a solution to the problems: do notation.

Do notation in Ruby

When we work with monads and Result in particular, we have to constantly bind them together. When we introduce complex logic with conditions, we end up with nested binds, and those are hard to work with. Do notation provides an alternative to bind, which also flattens the code.

So we can replace this:

def call(data)
  validate(data).bind(method(:log)).bind(method(:log))
end

With that:

def call(data)
  validated_data = yield validate(data)

  yield log(validated_data[:name])
  persist(validated_data) # <= You don't need to `yield` the last expression
end

or even that:

def call(data)
  validated_data = yield validate(data)

  log(validated_data[:name])
  persist(validated_data)
end

Sounds cool, right? We had to use bind to chain those operations. Now we can just yield the steps that can fail and keep the code flat.

This is how our operation looks with do notation:

class MyOperation
  include Dry::Monads
  include Dry::Monads::Do.for(:call)

  def call(data)
    validated_data = yield validate(data)

    log(validated_data[:name])

    persist(validated_data)
  end

  def validate(data)
    if data.valid?
      Success(name: data.name, age: data.user_age)
    else
      Failure("something went wrong")
    end
  end

  def log(name)
    print("User name is #{name}")
  end

  def persist(name:, age:)
    ...
    # some business logic here
    ...
  end
end

The core points:

A new mixin: Dry::Monads::Do.for(:call)
yield halts the execution if the function returns Failure
No need to unwrap the monad: yield does it for us
log and persist no longer need to return Result as they don’t affect the flow
We don’t have to stick to declarative style anymore
No need to yield the last expression — Ruby handles it for you
Last expression must return Result

Performance

The reason I wrote the article is that I wanted to benchmark do notation and compare its performance against dry-transaction.

The questions I wanted to answer:

Is do-notation faster than dry-transaction?
What are the performance differences between happy and not-so-happy paths?
What kind of performance drop do we have as we add more steps?

So I wrote a simple benchmark to test those things. Design decisions:

No IO or loops
Simple arithmetics is good enough
Objects behave like pure functions

The algorithm I tested looks as follows:

Multiply input by 2
If the result is greater than 100, return an error
Add 2

Total: 3 steps.

Benchmark output:

Warming up --------------------------------------
  do-notation: happy    33.809k i/100ms
do-notation: failure    14.274k i/100ms
  transaction: happy     5.878k i/100ms
transaction: failure     5.867k i/100ms
Calculating -------------------------------------
  do-notation: happy    387.914k (± 1.4%) i/s -      1.961M in   5.056134s
do-notation: failure    152.445k (± 1.7%) i/s -    770.796k in   5.057752s
  transaction: happy     59.981k (± 3.0%) i/s -    299.778k in   5.002999s
transaction: failure     60.327k (± 1.5%) i/s -    305.084k in   5.058375s

Comparison:
  do-notation: happy:   387913.7 i/s
do-notation: failure:   152445.2 i/s - 2.54x  slower
transaction: failure:    60327.4 i/s - 6.43x  slower
  transaction: happy:    59981.0 i/s - 6.47x  slower

So what do we see:

dry-transaction performance isn’t really affected by failures
do notation becomes approximately 2.5 times slower if we get a Failure
Do notation is over six times faster than dry-transaction

Heavier benchmark

Alright, so we had a benchmark that worked with three steps that could theoretically return Failure. But real-world apps are way more complex than that. So I decided to add more steps and see what happens.

Algorithm:

Multiply input by 2
Add 2 three times
If the result is greater than 100, return an error
Add 2 four times

Total: 9 steps.

Benchmark output:

Warming up --------------------------------------
  do-notation: happy    10.384k i/100ms
do-notation: failure     9.282k i/100ms
  transaction: happy     2.084k i/100ms
transaction: failure     2.083k i/100ms
Calculating -------------------------------------
  do-notation: happy    108.311k (± 1.3%) i/s -    550.352k in   5.082157s
do-notation: failure     89.917k (± 6.9%) i/s -    454.818k in   5.086821s
  transaction: happy     21.047k (± 2.1%) i/s -    106.284k in   5.052038s
transaction: failure     21.047k (± 1.5%) i/s -    106.233k in   5.048585s

Comparison:
  do-notation: happy:   108310.5 i/s
do-notation: failure:    89917.5 i/s - 1.20x  slower
  transaction: happy:    21047.4 i/s - 5.15x  slower
transaction: failure:    21047.1 i/s - 5.15x  slower

So what do we see here:

Happy path is not that much faster than not-so-happy path

That’s because happy path still has to evaluate the remaining steps. It takes time.
dry-transaction still shows similar performance for both outcomes
dry-transaction is five times slower than do notation

Facts & conclusion

Railway Oriented Programming is a way to gracefully handle errors in your application
You can use dry-monads and dry-transactions to build railway-oriented services
Functions can return either Success or Failure, which form the Result monad
Dry-transaction provides a DSL for railway oriented programming
Use bind, fmap, or and or_fmap to build railway-oriented code in Ruby
Use do notation to have a better control over your program’s flow
Do notation is way faster and more flexible than dry-transaction
This approach is framework-agnostic: works with Rails, Hanami, Sinatra, dry-web-roda

Also, there’s a visible lack of documentation for dry-monads, so if you decide to give it a try, you are welcome to contribute!