Blog (rails) - Fred Wu's Tech

I Accidentally Some Machine Learning - My Story of A Month of Learning Elixir

2016-07-23T18:33:29.000000Z

About a month ago I was in-between jobs - I had two weeks to rest up, recharge and get ready for my new job. So I thought, I should use those two weeks to learn something new.

Years ago I briefly looked into Elixir when it was first released to the wild, at the time I wasn’t interested in picking it up due to its syntax similarity to Ruby, despite their vastly different underlying semantics. I love Ruby, and it’s been my weapon of choice for the past 6-7 years, so when it came time for me to learn something new, I naturally wanted to learn something a bit more different than Ruby, syntax-wise.

Fast-forward a few years, I am more mature and open-minded, and are now in a position to welcome Elixir and to embrace the Ruby-like syntax as well as the functional programming mindset with open arms.

Learning Elixir

Given the strong influence of Ruby in Elixir, I can’t help but get nostalgic about learning it by reading another great book by Dave Thomas: Programming Elixir.

Dave Thomas’ original Programming Ruby was instrumental in the success of the Ruby programming language and its communities in the west, and it certainly has helped me to greatly widen my exposure to not only the wonderful world of Ruby but object-oriented programming in general as a PHP developer.

Learning Elixir has been really fun, not only does it share the same developer-friendliness championed by Ruby, it also does so with remarkable strength in concurrency thanks to BEAM (Erlang’s VM).

Many people are drawn to Elixir due to its Ruby influence, its functional and immutability nature, and its Actor-based concurrency model. I would however like to call out one of my favourite features of Elixir that sometimes gets overlooked, and that is how you could write unit tests using ExUnit.DocTest.

Elixir Makes Writing Unit Tests Effortless

Python programmers have been writing doctests for years, but Ruby due to not having an equivalent standard library, has never had writing doctests taken off in its community. I’m glad Elixir has.

Take a look at the code snippet below:

defmodule Stemmer.Step0 do
  @doc """
  ## Examples

      iex> Stemmer.Step0.trim_apostrophes("'ok")
      "ok"

      iex> Stemmer.Step0.trim_apostrophes("o'k")
      "o'k"

      iex> Stemmer.Step0.trim_apostrophes("'o'k'")
      "o'k"
  """
  def trim_apostrophes(word) do
    word
    |> String.replace_prefix("'", "")
    |> String.replace_suffix("'", "")
  end
end

The three test cases given will actually be tested by ExUnit, provided you ask it to do so as below:

defmodule Stemmer.Step0Test do
  use ExUnit.Case, async: true

  doctest Stemmer.Step0
end

You may think having doctests is not a big deal, what I can say is that I am really enjoying the fact that unit tests can be written with minimal friction and high visibility (as they sit with the implementation, rather than within a big test suite with a sea of files). Besides, not all of us practice TDD religiously or 100% of the time, and when I don’t or can’t do TDD, having doctests ensures I don’t forget adding test cases. :)

Toy Robot in Elixir

The best way to learn something new is to practice it as you learn it. Instead of diving straight into a complicated system, or to write a hello world program, I thought I would dig out a code test and re-implement it in Elixir.

The code test is the rather infamous Toy Robot test. As an interviewer, I must have reviewed a few hundreds of these tests, most of which in Ruby. I know what a good OO solution looks like, so naturally I was looking forward to “rethink” the problem and have a crack at it using Elixir.

Here is the result: https://github.com/fredwu/toy-robot-elixir

As a learner, I wanted to practice as many language features as possible, so included in the Toy Robot test code I tried:

Different data structures such as Struct and List
Pattern matching
Data piping (|>)
Agent for managing states
Macros for validation rules

I kept the test code small and nimble on purpose to illustrate the readability of a highly expressive language. I may not have exercised all the language features (such as supervisors and protocols), but at that point I felt I could start the Elixir journey with a big smile on my face already.

Learning Phoenix

I have to admit, a big part of the reason that got me started on learning Elixir is to build a side project. Initially I wanted to simply use Ruby and Rails because I can be very productive using them, but in the end I decided on learning Elixir and Phoenix, because most side projects fail, and when they do, how much learning can you extract out of those experiences? My answer to this question, is to learn a new language, a new framework and most importantly a new programming paradigm to drastically increase the amount of learning I could gain.

But please, before you jump on Phoenix, learn Elixir properly first! Over the years I’ve seen far too many cases of people jumping on Rails without understanding the language features of Ruby…

To learn Phoenix, the Programming Phoenix book is pretty much the bible on this subject aside from the official guide, and it is written by Chris McCord, the author of Phoenix, Bruce Tate, and Jose Valim, the author of Elixir.

As an experienced Rails developer, it only took me a day or two to go through the book (I largely skimmed the chapters on Channels as I don’t intend to build a real-time app).

Now, as a Ruby developer, the biggest pain point when working on Rails or other sizeable Ruby MVC projects for me, is ActiveRecord. I got so frustrated to the status quo I even attempted to fix it.

Fortunately, in Elixir and Phoenix we have Ecto. One of my favourite features of Ecto is the concept of Changeset.

Take a look at the code snippet below:

defmodule MySecretApp.User do
  use MySecretApp.Web, :model

  schema "users" do
    field :username, :string
    field :email, :string
    field :password, :string, virtual: true
    field :encrypted_password, :string

    has_one :profile, MySecretApp.Profile

    timestamps()
  end

  def changeset(struct, params \\ %{}) do
    struct
    |> cast(params, [:username, :email])
    |> validate_required([:username, :email])
    |> validate_length(:username, min: 3, max: 20)
    |> validate_format(:username, ~r/\A[a-zA-Z0-9_]+\z/, message: "alphanumeric and underscores only")
    |> validate_format(:email, ~r/@/)
    |> unique_constraint(:username)
    |> unique_constraint(:email)
  end

  def creation_changeset(struct, params \\ %{}) do
    struct
    |> changeset(params)
    |> password_changeset(params)
  end


  defp password_changeset(struct, params) do
    struct
    |> cast(params, [:password])
    |> validate_required([:password])
    |> validate_length(:password, min: 8, max: 200)
    |> encrypt_password
  end

  defp encrypt_password(changeset) do
    case changeset do
      %Ecto.Changeset{valid?: true, changes: %{password: password}} ->
        put_change(changeset, :encrypted_password, Comeonin.Bcrypt.hashpwsalt(password))
      _ ->
        changeset
    end
  end
end

The changset/2 function can be used when a user needs to be updated, whereas the creation_changeset/2 is to be used only when a user is first created. Sure, you can achieve similar result in Rails by using custom validators, but the fact that this practice is enforced by the library and the framework, is encouraging.

One other thing that I’ve seen in larger Rails apps, is the leaky abstraction of view-level logic, they typically sit in controllers, helpers (which are globally available) or worse, models. Phoenix in this case follows what Hanami, Trailblazer and many other frameworks do: it introduces a “view model” layer.

Something along the lines of:

defmodule MySecretApp.UserView do
  use MySecretApp.Web, :view

  def full_name do
    "#{title} #{first_name} #{last_name}"
  end
end

In a nutshell, Phoenix is just like Rails, but less magical (in a good way) and faster. :) Like the Elixir eco-system, most libraries and concepts feel like their ruby/rails counterparts, but more refined.

There are a few popular Hex packages if you’re familiar with their ruby counterparts:

	Elixir	Ruby
Code analysis	credo	rubocop
Testing	espec	rspec
Browser testing	wallaby	capybara
Test coverage	excoveralls	simplecov
Test factory	ex_machina	factory_girl
Authentication	guardian	devise

And the list goes on and on… Be sure to check out Awesome Elixir for more community curated Elixir libraries.

Learning Machine Learning

As I started building the foundation of my side project, you know, the usual user management and session management, etc, etc, by chance I came across mentions of Bayesian inference.

One thing led to another, very soon I started looking into varies different machine learning algorithms such as Naive Bayes and Random Forest. These are all subjects I had never come across or even heard of before, as I have no strong mathematics, statistics or computer science background. I was however, intrigued and inspired nonetheless, and wanted to employ some machine learning in my side project.

And the best way to understand and learn about machine learning algorithms? You guessed, is to write one! After some research, I came to the conclusion that Naive Bayes is one of the simplest to implement, is great for text classification which is useful for my side project, and has good accuracy given its simplicity and fast speed.

Introducing Simple Bayes

So, after a few train rides to and from work, on my newly purchased Macbook (yes, the one with only a single USB-C port), I built a Naive Bayes library: Simple Bayes.

As of writing, this is the only Naive Bayes library in Elixir that supports the following features:

Naive Bayes algorithm with different models
- Multinomial
- Binarized (boolean) multinomial
- Bernoulli
No external dependencies
Ignores stop words
Additive smoothing
TF-IDF
Optional keywords weighting
Optional word stemming via Stemmer

Let’s see it in action shall we? :)

SimpleBayes.init
|> SimpleBayes.train(:ruby, "I enjoyed using Rails and ActiveRecord for the most part.")
|> SimpleBayes.train(:ruby, "The Ruby community is awesome.")
|> SimpleBayes.train(:ruby, "There is a new framework called Hanami that's promising.")
|> SimpleBayes.train(:ruby, "Please learn Ruby before you learn Rails.")
|> SimpleBayes.train(:ruby, "We use Rails at work.")
|> SimpleBayes.train(:elixir, "It has Phoenix which is a Rails-like framework.")
|> SimpleBayes.train(:elixir, "Its author is a Rails core member, Jose Valim.")
|> SimpleBayes.train(:elixir, "Phoenix and Rails are on many levels, comparable.")
|> SimpleBayes.train(:elixir, "Phoenix has great performance.")
|> SimpleBayes.train(:elixir, "I love Elixir.")
|> SimpleBayes.train(:php, "I haven't written any PHP in years.")
|> SimpleBayes.train(:php, "The PHP framework Laravel is inspired by Rails.")
|> SimpleBayes.classify("I wrote some Rails code at work today.")
# => [
# ruby: 0.20761437345986136,
# elixir: 0.08101868169313056,
# php: 0.019047884912605735
# ]

As you can see, provided with reasonable training data, Naive Bayes can work extremely well.

Introducing Stemmer

Something I discovered as I was building Simple Bayes, is something called stemming. Let’s see another example:

SimpleBayes.init(stem: false)
|> SimpleBayes.train(:apple, "buying apple")
|> SimpleBayes.train(:banana, "buy banana")
|> SimpleBayes.classify("buy apple")
# => [
# banana: 0.057143654737817205,
# apple: 0.057143654737817205
# ]

Oops, the probabilities of the sentence “buy apple” of being apple and banana are the same, that’s not good, as we know “buying” and “buy” should mean the same thing. This is where stemming comes in handy. Let’s enable stemming and run the example again:

SimpleBayes.init(stem: true)
|> SimpleBayes.train(:apple, "buying apple")
|> SimpleBayes.train(:banana, "buy banana")
|> SimpleBayes.classify("buy apple")
# => [
# apple: 0.18096114003107086,
# banana: 0.1505149978319906
# ]

Stemming in this case correctly identified “buy” and “buying” as the same thing (they both have the stemmed root of “buy”).

How did I include stemming in Simple Bayes you wonder? Let me introduce you to Stemmer - as of writing this is the only Porter2 stemmer available in Elixir. :)

Elixir and BEAM may not be known for their raw performance, but compared to a similar Porter2 stemmer implemented in pure ruby, my Elixir version runs about five times faster (under 5s to stem 29000+ words, compared to 25s with the ruby version).

The Learning Continues…

I’ve only been writing Elixir for a month, and there are still heaps to learn and to practice. So far though, I have to say my experience with both Elixir and Phoenix has been fantastic - I get the same satisfying and pumped feeling I got when I was learning Ruby and Rails years ago.

If you haven’t checked out Elixir, I strongly encourage you to do so, after all, the world seems to be moving in the concurrency direction at an increasingly rapid speed, including Ruby 3x3. :)

]]>

Datamappify - A New Take on Decoupling Domain, Form and Persistence in Rails

2013-06-27T12:09:00.000000Z

This post is about the ruby library we are building - Datamappify, please go check it out.

At Locomote we are building a relatively large web application using Rails. Before we began to lay the foundation, we knew very well that if we wanted the project to be maintainable we had to architect the system with extra care and attention. More specifically, we can’t rely on simply using ActiveRecord which combines behaviour and persistence as our domain models.

We began our search for something that would help us decouple our application from the domain layer down to the form handling. We’ve found a couple of gems that are close to what we were after - Curator, Minimapper, Edr and later on Reform. They are all wonderful gems but unfortunately none of them has everything we need.

Here are the things we need:

Decouple domain logic and data persistence.
Decouple models and forms.
Support ActiveRecord (or at the very least, ActiveModel) so we can still use many of the awesome gems like Devise, SimpleForm and CarrierWave.
Support attributes mapped from different data sources (e.g. remote web services from third-party vendors).
Support lazy loading so that attributes stored on remote data sources will not get triggered upon loading the entities.

All things considered, we bit the bullet and started working on Datamappify.

Before I go into details and examples, here is an extremely simplified overview of Datamappify’s architecture:

As you can see, Datamappify is loosely based on Repository pattern and Entity Aggregation pattern.

Datamappify has three main design goals:

To utilise the powerfulness of existing ORMs so that using Datamappify doesn’t interrupt too much of your current workflow. For example, Devise would still work if you use it with a UserAccount ActiveRecord model that is attached to a User entity managed by Datamappify.
To have a flexible entity model that works great with dealing with form data. For example, SimpleForm would still work with nested attributes from different ORM models if you map entity attributes smartly in your repositories managed by Datamappify.
To have a set of data providers to encapsulate the handling of how the data is persisted. This is especially useful for dealing with external data sources such as a web service. For example, by calling UserRepository.save(user), certain attributes of the user entity are now persisted on a remote web service. Better yet, dirty tracking and lazy loading are supported out of the box!

You can read more about Datamappify in the project’s README. For this blog post I will focus on how we use Datamappify in our Rails application. We are still early days in our application development and Datamappify still has quirks and issues, but I am hoping this post will illustrate some of the key benefits of Datamappify.

Getting Started

Getting started with using Datamappify is really easy. We simply include it in the Gemfile:

gem 'datamappify'

To keep things organised, we put entities and repositories in their respective directories under app:

Entities

The reason we wanted Datamappify to utilise existing ORMs like ActiveRecord, is so that we could still use gems like Devise.

So, we have a UserAccount model that handles authentication:

class UserAccount < ActiveRecord::Base
  devise :database_authenticatable,
          :recoverable, :rememberable, :trackable, :validatable
end

UserAccount has one and only one purpose - user account authentication. Other user behaviours would be contained in either the User entity itself or service objects. Speaking of the User entity, it looks like this:

class User
  include Datamappify::Entity

  attr_accessor :user_account

  attribute :account_username, String
  attribute :account_email, String
  attribute :first_name, String
  attribute :last_name, String
  attribute :activated, Boolean, :default => true

  attributes_from Contact, prefix_with: :work

  validates :first_name, presence: true
  validates :last_name, presence: true

  references :agency

  def full_name
    "#{first_name} #{last_name}"
  end
end

We include Datamappify::Entity in the User class to make it an entity. We set the :user_account accessor is so that we could attach the UserAccount object onto the entity.

The attribute DSL is from Virtus - we get attribute type coercion for free, awesome!

attributes_from is a DSL provided by Datamappify - it essentially “mounts” all the attributes from another entity, in this case the Contact entity, which looks like this:

class Contact
  include Datamappify::Entity

  attribute :email, String
  attribute :phone_number, String
  attribute :fax_number, String

  validates :phone_number, presence: true
  validates :fax_number, presence: true
end

All the attributes and validation rules from Contact are now “mounted” on User. attributes_from Contact, prefix_with: :work is equivalent to:

attribute :work_email, String
attribute :work_phone_number, String
attribute :work_fax_number, String

validates :work_phone_number, presence: true
validates :work_fax_number, presence: true

Validation DSL is provided by ActiveModel::Validations.

references :agency is a convenient DSL provided by Datamappify. It is equivalent to:

attr_accessor :agency

attribute :agency_id, Integer

Now that we have entities, we need a way to retrieve and store them. For that we need repositories.

Repositories

Here is the user repository:

class UserRepository
  include Datamappify::Repository

  for_entity User

  default_provider :ActiveRecord

  map_attribute :account_username, 'ActiveRecord::UserAccount#username'
  map_attribute :account_email, 'ActiveRecord::UserAccount#email'

  map_attribute :work_email, 'ActiveRecord::Contact#email'
  map_attribute :work_phone_number, 'ActiveRecord::Contact#phone_number'
  map_attribute :work_fax_number, 'ActiveRecord::Contact#fax_number'
end

Similarly to an entity, we include Datamappify::Repository to make the class a repository. We specify for_entity to link the repository to an entity, and default_provider to use a specific data provider for unmapped attributes.

Unmapped attributes are the ones not specified in map_attribute, in this case they are first_name, last_name and activated. Unmapped attributes are actually automatically mapped by Datamappify, so the user repository essentially does this:

map_attribute :first_name, 'ActiveRecord::User#first_name'
map_attribute :last_name, 'ActiveRecord::User#last_name'
map_attribute :activated, 'ActiveRecord::User#activated'

The first argument of map_attribute is the name of the attribute from the User entity (e.g. :first_name).

The second argument is a string containing three things:

ActiveRecord is the data provider.
::User is the ActiveRecord model class.
#first_name is the ActiveRecord attribute from the model class.

Even though the User entity is a representation of a user on the domain level, the underlying data structure does not necessary have to be. As you can see from the user repository example, we are mapping :account_username and :account_email to the UserAccount ActiveRecord model we’ve seen before. And we have a bunch of contact details attributes mapped to the Contact ActiveRecord model.

The database schema therefore looks like this:

create_table "contacts", force: true do |t|
  t.string "email"
  t.string "phone_number"
  t.string "fax_number"
  t.integer "user_id"
end

create_table "user_accounts", force: true do |t|
  t.string "username", default: "", null: false
  t.string "email", default: "", null: false
  t.string "encrypted_password", default: "", null: false
  t.string "reset_password_token"
  t.datetime "reset_password_sent_at"
  t.datetime "remember_created_at"
  t.integer "sign_in_count", default: 0
  t.datetime "current_sign_in_at"
  t.datetime "last_sign_in_at"
  t.string "current_sign_in_ip"
  t.string "last_sign_in_ip"
  t.integer "user_id"
  t.datetime "created_at"
  t.datetime "updated_at"
end

create_table "users", force: true do |t|
  t.string "first_name"
  t.string "last_name"
  t.boolean "activated"
  t.integer "agency_id"
end

Note that because the Contact entity is mounted on the User entity, we need a foreign key user_id in the contacts table to link them.

Data providers

Because we are allowed to specify a data provider (i.e. ActiveRecord) for each attribute, we can map attributes to entirely different data providers! For instance, we could have:

map_attribute :first_name, 'ActiveRecord::User#first_name'
map_attribute :last_name, 'ActiveRecord::User#last_name'
map_attribute :activated, 'Sequel::UserActivation#activated'

Now the activated attribute is mapped to the UserActivation Sequel model. This powerful feature would allow us to develop data providers that communicate with remote web services. :)

Repository APIs

Datamappify provides a bunch of APIs for retrieving and storing data. These APIs are being developed as needed during our application development.

For instance, to find a particular user entity by ID:

user = UserRepository.find(1)

To get all the users:

users = UserRepository.all

To search for users:

users = UserRepository.find(:first_name => 'Fred', :activated => true)

To save a user:

UserRepository.save(user)

To delete a user:

UserRepository.destroy(user)

There are some limitations for certain APIs, you can read more about them here.

When saving an entity, because Datamappify has an internal pool for tracking dirty attributes, only those dirty attributes will get saved.

Also, Datamappify supports attribute lazy loading, all we have to do is to tell our entity to become lazy aware:

class User
  include Datamappify::Entity
  include Datamappify::Lazy
end

Once an entity is lazy aware, a repository will inject the lazy loading mechanism onto the entity when it retrieves such entity (via find).

Both repositories and entities support inheritance, again, you may read more about them in the project’s README.

Putting Things Together With A Controller

Remember we have the UserAccount for authentication? So how does that work? Well, here’s the piece of code in our application controller:

class ApplicationController < ActionController::Base
  before_filter :authenticate_user_account!

  private

  def current_user
    unless current_user_account.blank?
      @current_user ||= UserRepository.find(current_user_account.user_id)
    end
  end

  helper_method :current_user
end

Not too different from the normal Devise workflow hey? ;)

Forms

Web applications typically have lots of forms - ours is no different. It turns out, Datamappify can help build form objects too!

Here’s the view portion of one of our forms (we use Slim templates):

# our standard form layout

= simple_form_for submission_target, url: submission_path, signed: true do |f|

  = yield(f)

  .form-actions
    .pull-left
      - if archivable? && resource.persisted?
        = link_to 'Archive', '#', method: :delete, class: 'btn-archive'

    = f.submit 'Save'
    = link_to 'Cancel', cancel_path

# actual form content

= render layout: 'forms/resource' do |f|
  h3 Account Details
  .row-fluid
    .span4 = f.input :account_type, collection: BankAccount::ACCOUNT_TYPES.invert
    .span4 = f.input :account_code
    .span4 = f.input :account_name
  .row-fluid
    .span4 = f.input :account_number
    .span4 = f.input :bank_name
    .span4 = f.input :bank_branch
  .row-fluid
    .span4 = f.input :bank_bsb, label: "Bank BSB"
    .span4 = f.input :activated, as: :boolean

  h3 Branch Details
  .row-fluid
    .span4 = f.input :branch_address_line_1, label: "Address 1"
    .span4 = f.input :branch_address_line_2, label: "Address 2"
    .span4 = f.input :branch_address_line_3, label: "Address 3"
  .row-fluid
    .span4 = f.input :branch_state, label: "State"
    .span4 = f.input :branch_postcode, label: "Postcode"
    .span4
      = f.input :branch_country_code, label: "Country" do
        = f.country_select :branch_country_code, ['au', 'us', 'gb'], value: 'au'
      - f.add_signed_fields :branch_country_code
  .row-fluid
    .span4 = f.input :branch_phone_number, label: "Phone number"
    .span4 = f.input :branch_fax_number, label: "Fax number"

  h3 Merchant Details
  .row-fluid
    .span4 = f.input :merchant_number
    .span4 = f.input :merchant_name
    .span4 = f.input :merchant_address
  .row-fluid
    .span4 = f.input :merchant_biller_code, label: "Biller code"
    .span4 = f.input :merchant_bpay_method, collection: Merchant::BPAY_METHODS.invert, label: "BPAY method"

  h3 Payment Details
  .row-fluid
    .span4 = f.input :payment_number
    .span4 = f.input :payment_merchant, label: "Merchant description"
    .span4 = f.input :payment_reference_type, collection: PaymentAccount::REFERENCE_TYPES.invert, label: 'Reference type'

As you can see, it’s a form containing details from bank account, branch address, branch contact information, merchant and payment. And indeed we do have bank_accounts, addresses, contacts, merchants and payment_accounts tables in our database. Yet, the form still remains flat-structured and submits via simple_form_for.

This is thanks to our BankAccount entity and BankAccountRepository repository:

class BankAccount
  include Datamappify::Entity
  include Concerns::Archivable

  ACCOUNT_TYPES = {
    'general' => 'General',
    'trust' => 'Trust'
  }

  attribute :account_type, String
  attribute :account_code, String
  attribute :account_name, String
  attribute :account_number, String
  attribute :bank_name, String
  attribute :bank_branch, String
  attribute :bank_bsb, String
  attribute :activated, Boolean, default: true

  attributes_from Address, prefix_with: :branch
  attributes_from Contact, prefix_with: :branch
  attributes_from Merchant, prefix_with: :merchant
  attributes_from PaymentAccount, prefix_with: :payment

  references :agency

  validates :account_type, Mos::Validation.in(ACCOUNT_TYPES)
  validates :account_code, Mos::Validation::STANDARD_TEXT
  validates :account_name, Mos::Validation::STANDARD_TEXT
  validates :account_number, Mos::Validation::STANDARD_TEXT
  validates :bank_name, Mos::Validation::STANDARD_TEXT
  validates :bank_branch, Mos::Validation::STANDARD_TEXT_OPTIONAL
  validates :bank_bsb, Mos::Validation::BSB
  validates :merchant_bpay_method, Mos::Validation.presence_depend_on(:merchant_biller_code)
  validates :payment_reference_type, Mos::Validation.presence_depend_on(:payment_number)
end

class BankAccountRepository
  include Datamappify::Repository

  for_entity BankAccount

  default_provider :ActiveRecord

  map_attribute :branch_address_line_1, 'ActiveRecord::Address#address_line_1'
  map_attribute :branch_address_line_2, 'ActiveRecord::Address#address_line_2'
  map_attribute :branch_address_line_3, 'ActiveRecord::Address#address_line_3'
  map_attribute :branch_state, 'ActiveRecord::Address#state'
  map_attribute :branch_postcode, 'ActiveRecord::Address#postcode'
  map_attribute :branch_country_code, 'ActiveRecord::Address#country_code'

  map_attribute :branch_phone_number, 'ActiveRecord::Contact#phone_number'
  map_attribute :branch_fax_number, 'ActiveRecord::Contact#fax_number'
  map_attribute :branch_email, 'ActiveRecord::Contact#email'
  map_attribute :branch_website, 'ActiveRecord::Contact#website'

  map_attribute :merchant_number, 'ActiveRecord::Merchant#number'
  map_attribute :merchant_name, 'ActiveRecord::Merchant#name'
  map_attribute :merchant_address, 'ActiveRecord::Merchant#address'
  map_attribute :merchant_biller_code, 'ActiveRecord::Merchant#biller_code'
  map_attribute :merchant_bpay_method, 'ActiveRecord::Merchant#bpay_method'

  map_attribute :payment_number, 'ActiveRecord::PaymentAccount#number'
  map_attribute :payment_merchant, 'ActiveRecord::PaymentAccount#merchant'
  map_attribute :payment_reference_type, 'ActiveRecord::PaymentAccount#reference_type'
end

Pretty clean and straightforward. What do you think?

Datamappify - Keeping Your Domain Layer Sane

Datamappify’s concept isn’t new, but there simply isn’t anything in the ruby community that solves everything Datamappify tries to solve.

I hope that this post has not only introduced you to Datamappify, but has also made you think about how to make your application’s domain layer more decoupled.

Thanks for reading, and if you have any feedback for Datamappify please get in touch with me!

]]>

ActiveRecord and DB Migration Ate My Model Attributes!

2013-02-27T00:12:00.000000Z

Update: You might also want to check out reset_column_information.

So a few days ago we started seeing the following errors on our Jenkins builds (swapped with fictional model and attribute names):

NoMethodError:
  undefined method `attack_power=' for #

attack_power is a new attribute we recently added to the Ironman ActiveRecord model.

I was baffled, as the table column is clearly there but ActiveRecord couldn’t see it.

This weird behaviour is confirmed by debugging the model:

Ironman
# => Ironman(id: integer, created_at: datetime, updated_at: datetime)

Ironman.column_names
# => ["id", "created_at", "updated_at"]

And by debugging the schema:

ActiveRecord::Base.connection.structure_dump
# => "CREATE TABLE `ironmans` (\n `id` int(11) NOT NULL AUTO_INCREMENT,\n `created_at` datetime NOT NULL,\n `updated_at` datetime NOT NULL,\n `attack_power` int(11) NOT NULL,\n PRIMARY KEY (`id`)\n) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;\n\n"

So apparently somehow ActiveRecord ate our attack_power attribute!

After some struggling and head-scratching, finally I’ve discovered that during the migration if we call the Ironman class, then any subsequent attribute changes to the class will not be recognised by ActiveRecord.

It turns out, because we run our test suites by invoking rake tasks:

Rake::Task['spec:something'].invoke
Rake::Task['spec:something_else'].invoke

And we manually reset our database beforehand too within the same Rakefile:

Rake::Task['db:reset'].invoke

ActiveRecord will cache its attributes as soon as a model class (Ironman) is called during the migration.

The fix? Simple! Simply use another process to run the database reset:

system 'rake db:reset'

]]>

API Taster: Visually Test Rails Application API

2012-07-02T04:59:00.000000Z

Like a lot of places, at Locomote we are building a platform that is API-based. As much as I like having comprehensive test suites, I often feel the need to manually test API endpoints to see exactly what the responses are.

Tools such as Postman solves part of the issue: they allow us to quickly test API endpoints without messing with cURL.

But as a lazy developer, I want more. ;)

I want something that:

automatically generates API endpoints from Rails routes definition
defines input params as easy as defining routes
has input params that can be shared with test factories

And so API Taster was born. Please check it out to see how you can use it.

]]>

[Rails Tip] Render views outside of Controllers or Views

2012-06-20T09:09:00.000000Z

Ever wondered how you could utilise the render method outside the context of Rails controllers and views? If you wonder why anyone would do that. Well, imagine you are building an awesome form builder, you need to output and/or store rendered partials in the buffer. How do you do that?

For example, what if you want to do this in your view?

<%=raw Awesome::FormBuilder.new(some_options).html %>

You could do something like this:

module Awesome
  class FormBuilder < AbstractController::Base
    include AbstractController::Rendering
    include ActionView::Context
    include ActionView::Helpers::CaptureHelper

    # set the view paths from your engine or from your application root, i.e. Rails.root
    self.view_paths = Awesome::Engine.root.join('app/views')

    def initialize(params)
      flush_output_buffer
      @_buffer = ''
      add_to_buffer(params)
    end

    def html
      @_buffer
    end

    private

    def add_to_buffer(params)
      # some logic to add rendered content to @_buffer
    end
  end
end

The idea is to mixin the render method, but also ensuring the view buffer is correctly reset with flush_output_buffer.

Hope that helps. :)

]]>

Slim, a Fast and Lightweight Rails Template Engine!

2010-10-17T21:40:20.000000Z

For the past few weeks I have started contributing to a small project - Slim.

Slim is a fast, lightweight templating engine for Rails 3. It has been tested on Ruby 1.9.2 and Ruby/REE 1.8.7. Slim is heavily influenced by Haml and Jade.

Andrew Stone who is the author of the project has posted a quick update on the latest feature additions to Slim. Please go check it out.

The source code of Slim is available on Github.

]]>

A Speedier Rails App using Rails 3.1 + Arel 2.0

2010-09-29T00:51:00.000000Z

Before (Rails 3.0.1pre stable branch + Arel 1.0.1):

After (Rails 3.1.0 master branch + Arel 2.0.0dev master branch:

Thanks to the awesome work done by Aaron Patterson (@tenderlove) and others. :-)

UPDATE:

After Aaron Patteron’s tweet, I ran the tests again on Rails 3.0.1pre stable branch + Arel 2.0.0dev master branch, and the result blew my mind:

]]>

`bundle: command not found` or `Could not find RubyGem bundler (>= 0)` During Capistrano Deployment? No Problems!

2010-09-16T05:59:00.000000Z

Don’t you just hate it when you get the following errors during a Capistrano deployment?

bundle: command not found
Could not find RubyGem bundler (>= 0) (Gem::LoadError)

In fact, even if you don’t use bundler, you might still get errors like this:

rake: command not found
Could not find RubyGem rake (>= 0) (Gem::LoadError)

It turns out this has something to do with the $PATH and $GEM_HOME variables.

So here’s the quick fix.

PermitUserEnvironment yes

Don’t forget to restart ssh:

/etc/init.d/ssh restart

Now, log in as the deployment user, and create ’~/.ssh/environment’ with the following content:

PATH=/usr/local/rvm/gems/ruby-1.9.2-p0/bin:/bin:/usr/local/rvm/rubies/ruby-1.9.2-p0/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
GEM_HOME=/usr/local/rvm/gems/ruby-1.9.2-p0

* The above paths are for your reference only, obviously you need to work them out for your server environment. The only thing you need to make sure is that the GEM_HOME’s path matches one from the PATH.

Now, to verify this all work, you may use cap shell to start a new shell session and try out your commands.

]]>

[Rails] RailsConfig, A New Iteration of AppConfig for Rails 3

2010-08-11T01:50:14.000000Z

If you haven’t already been using AppConfig in your Rails project, well, you should!

Jacques Crocker has recently released his new version of the original AppConfig - RailsConfig. I was invited to join the development of this new tool, so make sure you go check it out. :-)

]]>

[Rails] Inherited Resources Views Now Supports Rails 2.x

2010-08-07T07:35:00.000000Z

I have just pushed a commit that added Rails 2.x compatibility to Inherited Resources Views. Please give it a spin! :-)

P.S. I’ve only tested it on Rails 2.3.8.

]]>