Creating a DSL with Ruby

After completing the first version of a website for a friend, I dedicated some time to extract an API client I had developed for connecting with dlocalGo’s API. dlocalGo is a payment processor built by dlocal, a company like Stripe, but that operates mainly in South America and other emerging markets.

Since this was my first time creating an open-source gem and I wasn’t on a rush to publish it, I took the liberty to play around with some Ruby metaprogramming features while at it. After playing around for a while, I realized that creating my own mini-DSL to easily add support for new endpoints with little-to-no code was easier than I initially expected.

I leveraged Ruby’s metaprogramming capability, particularly the define_method, which allows for dynamic definition of new methods. Down the road, I also ended up digging into other Ruby features such as class_eval, instance_eval, multiline strings with <<-, Module’s lifecycle and more. This also encouraged me to dig into some of ActiveSupport’s APIs and discovered some awesome helper methods like class_attribute and how ActiveSupport::Concern actually works

v0.1

When starting the website we only needed to support one-off payments and refunds, so I created a simple class with a method per endpoint:

module DlocalGo
  class Client
    # ...

    def create_payment(params = {})
      uri = "/v1/payments"
      # ...
    end

    def get_payment(payment_id)
      uri = "/v1/payments/#{payment_id}"
      # ...
    end

    def create_refund(params = {})
      uri = "/v1/refunds"
      # ...
    end

    # ...
  end
end

This approach soon proved inadequate as we needed to incorporate subscriptions and recurring payments. By following the same approach we would’ve ended up with 20 different repetitive methods and a bloated class with 600+ lines of code.

On our way to v1.0 🏎️

The main idea was to create a class method that when being called would generate a method for a specific endpoint based on its URI, HTTP method and response model. It should look something like:

module DlocalGo
  class Client
    # ...

    endpoint :create_payment, uri: "/v1/payments", verb: :post, dto_class: Responses::Payments

    # ...
  end
end

# Usage

client = DlocalGo::Client.new
response = client.create_payment({param1: "1", param2: "2"})
# response is a Responses::Payment object

This is where define_method comes in handy.

---

I already had experience combining class methods and define_method, but it was always inside Ruby on Rails and using concerns (ActiveSupport::Concern). So when I first tried the following approach in the "dlocal_go" gem I realized "class_methods do ... end" and "included do ... end" don’t work! 😭

# This approach doesn't work with plain Ruby

module EndpointGenerator
  class_methods do
    def endpoint(method, uri:, verb:, dto_class:)
      define_method(method) do |params = {}|
        # ...
      end
    end
  end
end

Encountering “method_missing” errors led me to discover that these methods were part of ActiveSupport::Concern. This prompted me to consult the ActiveSupport documentation and delve into the source code for included and class_methods.

It’s easy to see in their documentation that class_methods defines class methods from a given block (see docs), but how does it work behind the scenes?

You can navigate to the gem’s source code by using gem unpack, gem open or installing extensions in your favorite IDE (although it might not be necessary if you use something like RubyMine). Here you’ll find the definition to class_methods

module ActiveSupport
  module Concern
    def append_features(base)
      # ...
      base.extend const_get(:ClassMethods) if const_defined?(:ClassMethods)
      # ...
    end

    def class_methods(&class_methods_module_definition)
      mod = const_defined?(:ClassMethods, false) ?
        const_get(:ClassMethods) :
        const_set(:ClassMethods, Module.new)

      mod.module_eval(&class_methods_module_definition)
    end
  end
end

It checks if there’s a constant named ClassMethods defined, if so it keeps it, otherwise it creates a new empty ClassMethods module, then evaluates the block we passed inside the ClassMethods module. Finally ActiveSupport::Concern has its own implementation of the append_features method, a method that gets called when a module is included (see docs). Here, it “extends” base (the class where you included the Module) with all of methods inside the ClassMethods module.

This all makes sense especially when we look how a typical module looks with plain ruby:

module M
  def self.included(base)
    base.extend ClassMethods
  end

  module ClassMethods
    # ...
  end
end

The method included is similar to append_features (see docs).

It’s clear that ActiveSupport::Concern gives us a nicer and cleaner API to achieve the same goal. So after getting a better grasp of how modules and their lifecycle works, we can create the first version of our EndpointGenerator Module:

module EndpointGenerator
  def self.included(base)
    base.extend(ClassMethods)
  end

  module ClassMethods
    def endpoint(method, uri:, verb:, dto_class:)
      define_method(method) do |params = {}|
        # ...
      end
    end
  end
end

Parsing the response

🎉 Now we can take a closer look at parsing a succesful response, the first version of the response objects (v0.1) looked more or less like this:

module DlocalGo
  module Responses
    class Payment
      RESPONSE_ATTRIBUTES = %i[id amount etc..].freeze

      attr_reader(*RESPONSE_ATTRIBUTES)

      def initialize(response)
        RESPONSE_ATTRIBUTES.each do |attribute|
          instance_variable_set("@#{attribute}", response.send(attribute))
        end
      end
    end
  end
end

If a response object had an association it would look something like this:

require_relative "subscription_plan"

module DlocalGo
  module Responses
    class Subscription
      RESPONSE_ATTRIBUTES = %i[id country etc...].freeze

      attr_reader(*RESPONSE_ATTRIBUTES)

      def initialize(response)
        (RESPONSE_ATTRIBUTES).each do |attribute|
          instance_variable_set("@#{attribute}", response.send(attribute))
        end

        @plan = SubscriptionPlan.new(OpenStruct.new(response["plan"]))
      end
    end
  end
end

There was clear repetition in all of the response objects, so I thought it would be great if by just defining the attributes and associations with a clean API similar to ActiveRecord’s one, all of them could be populated automatically. Something that would look similar to this:

require_relative "subscription_plan"
require_relative "client"

module DlocalGo
  module Responses
    class Subscription < Base
      has_attributes %i[id country etc..]

      has_association :plan, SubscriptionPlan
      has_association :client, Client
    end
  end
end

To achieve this, we needed a way to “save” all of the associations data (attribute name and its corresponding response class) somewhere when has_association was called.

There was a specific method ActiveSupport::Support gives us that I saw all around Rails codebase: class_attribute. It declares a class-level attribute and creates the getter/read and setter/write methods for us. With that in mind, I created a Responses::Base class, which all Response objects will inherit from, and created a new ReponseParser module, in which I planned to include all of the “magic” 🪄

require_relative "response_parser"

module DlocalGo
  module Responses
    class Base
      include ResponseParser
    end
  end
end

module DlocalGo
  module Responses
    module ResponseParser
      def self.included(base)
        base.class_eval { class_attribute :response_attributes, :response_associations }
      end
    end
  end
end

Not sure what class_eval does? Here’s a short and interesting article from Stanford that explains class_eval and instance_eval

Now we need to add the ClassMethod module with the methods that will define our second mini-DSL (has_attributes and has_association)

module DlocalGo
  module Responses
    module ResponseParser
      def self.included(base)
        base.class_eval { class_attribute :response_attributes, :response_associations }

        base.extend(ClassMethods)
      end

      module ClassMethods
        def has_attributes(attributes)
          self.response_attributes ||= []
          self.response_attributes << attributes

          class_eval { attr_reader(*attributes) }
        end

        def has_association(attribute, klass)
          self.response_associations ||= {}
          self.response_associations[attribute] = klass

          class_eval { attr_reader attribute }
        end
      end
    end
  end
end

We now save all the information needed to parse the response inside the response_attributes and response_associations class attributes. The only missing piece is to override the initialize method, so we can call a new method that will iterate on the response_attributes and response_associations and populate each one of them.

module DlocalGo
  module Responses
    module ResponseParser
      def self.included(base)
        base.class_eval { class_attribute :response_attributes, :response_associations }

        base.extend(ClassMethods)

        base.class_eval <<-CODE, __FILE__, __LINE__ + 1
          def initialize(response)
            assign_attributes(response)
            assign_associations(response)
          end
        CODE
      end

      module ClassMethods
        def has_attributes(attributes)
          # ...
        end

        def has_association(attribute, klass)
          # ...
        end
      end

      private

      def assign_attributes(response)
        # ...
      end

      def assign_associations(response)
        # ...
      end
    end
  end
end

You might be wondering what does <<- stand for 🤔. It’s just a multiline string delimited by the CODE keyword. __FILE__ and __LINE__ + 1 are there for debugging purposes, you can read more about it in Ruby’s official documentation.

You’ll see this approach in various places inside the Ruby on Rails codebase, class_eval not only works with blocks but also with strings, so instead of defining a method inside of a method (doing def initialize inside self.included(base)), we use class_eval and use a string instead of a block.

Conclusion

That’s a wrap! We’ve explored an extensive range of topics, from ActiveSupport::Concern internals to the lifecycle of modules, along with define_method, class_eval, and multiline strings. We’ve also looked at helper methods like class_attribute 😄.

So what might be next?

• Since we ended up including ActiveSupport as a dependency in order to use the class_attribute method, we might as well refactor these modules into cocerns 😅.

• Maybe creating another gem similar to ActiveResource, that would allow you to turn any Ruby class into a HTTP client by just adding endpoint calls like we do inside DlocalGo::Client

• I also plan to create a small rails engine in the future were you’ll be able to get a simple dlocalGo dashboard with all the payments, subscriptions, and transactions just by mounting it inside your Ruby on Rails app.

Don’t know what a rails engine is? That might be a whole new blog post, meanwhile, I encourage you to read the official docs that exaplains it amazingly