Blog


Mar

Advanced topics in Ruby FFI


Short primer: what is FFI?

This article is not a tutorial on the basics of FFI. However, if you’ve never heard of FFI before, I’d like to wet your appetite before continuing on.

FFI is an alternative to writing C to use functionality locked within native libraries from Ruby. It allows you to explain, with an intuitive Ruby DSL, which functions your native library contain, and how they should be used. Once the functionality of your native library is mapped out, you can call the functions directly from Ruby.

Furthermore, gems using ffi do not need to be compiled, and will run without modifications on CRuby, JRuby and Rubinius! In practice there could be small differences between the platforms in the behaviour and usage of FFI, and if you find any you should report them to the Ruby FFI issue tracker so it can be dealt with.

As far as basic tutorials on using FFI, your best resource is the FFI wiki. It also has a list of projects using FFI, which is your second best resource on learning how to use FFI.

Aliasing with typedef

If we look at the header for a function from libspotify:

SP_LIBEXPORT(sp_error) sp_session_player_prefetch(sp_session *session, sp_track *track);

Naively mapping this to FFI we’ll need:

enum :error, [ … ]
attach_function :sp_session_player_prefetch, [ :pointer, :pointer ], :error

Unfortunately, we lost two pieces of valuable information here. Both sp_session and sp_track are types that occur many times in the library. When we look at the ruby implementation, there is no hint whatsoever of what type the two pointers should be of.

It does not need to be like this. Using typedef we can name our parameters, and bring back the information that we lost in our translation.

typedef :pointer, :session
typedef :pointer, :track
enum :error, [ … ]
attach_function :sp_session_player_prefetch, [ :session, :track ], :error

Functionality of our method does not change, but implementation is now slightly more clear and maintainable.

Specializing in attach_function

C libraries do not follow Ruby naming conventions, which makes sense since they’re not written in Ruby. However, bindings written with Ruby FFI are in Ruby and will be called from Ruby, so they should have the look and feel of Ruby.

Attach function allow you to call it in two ways:

attach_function :c_name, [ :params ], :returns, { :options => values } # 1
attach_function :ruby_name, :c_name, [ :params ], :returns, { :options => values } # 2

Using the first form will create your Ruby methods with the same name as your native library’s functions. Using the second form allows you to rename the bound method, giving it a more expected final name.

Native libraries you bind with FFI will have naming conventions of their own. For example, OpenAL will prefix it’s functions with al or alc, and camel case. libspotify will prefix it’s functions with sp_. Apart from removing the suffix, and snake_casing the function name, we want the Ruby method to be named similarly. We could repeat ourselves for every method:

attach_function :open_device, :alcOpenDevice, [ :string ], :device
attach_function :close_device, :alcCloseDevice, [ :device ], :bool

But remember! When you use FFI, you extend the FFI::Library inside a module of your own. This also means you can override the attach_function call, without your specialized version leaking to the outside world. By overriding attach_function we can avoid unnecessary noise in our FFI bindings.

def self.attach_function(c_name, args, returns)
    ruby_name = c_name.to_s.sub(/\Aalc?/, "").gsub(/(?\<\!\A)\p{Lu}/u, '_\0').downcase
    super(ruby_name, c_name, args, returns)
end

attach_function :alcOpenDevice, [ :string ], :device # gets bound to open_device
attach_function :alcCloseDevice, [ :device ], :bool # gets bound to close_device

This does not end here. After calling super inside attach_function you have the option of further specializing the newly bound method. You could implement automatic error checking for every API call, or alter the parameters based on native library conventions, and more. Just remember that the added complexity should be worth the savings.

FFI::Structs as parameters

Structs in FFI can be used as parameters, and is by default equivalent to specifying a type of :pointer.

class SomeStruct < FFI::Struct
end

attach_function :some_function, [ SomeStruct ], :void
# equivalent to:
attach_function :some_function, [ :pointer ], :void

callback :some_callback, [ SomeStruct ], :void
# equivalent to:
callback :some_callback, [ :pointer ], :void

I’d like to bring forth an alternative for your referenced struct parameters, namely FFI::Struct.by_ref. It behaves very similarly to the above, with the important difference in that it type-safety built-in!

attach_function :some_function, [ SomeStruct ], :void
some_function FFI::Pointer.new(0xADDE55) # this is possibly unsafe, but allowed

attach_function :some_function, [ SomeStruct.by_ref ], :void
some_function FFI::Pointer.new(0xADDE55) # BOOM, wrong argument type FFI::Pointer (expected SomeStruct) (TypeError)
some_function SomeOtherStruct.new # BOOM, wrong argument type SomeOtherStruct (expected SomeStruct) (TypeError)

Further more, if you use FFI::Struct.by_ref for your callback parameters or function return values, FFI will automatically cast the pointer to an instance of your struct for you!

callback :some_callback, [ SomeStruct.by_ref ], :void
attach_function :some_function, [ :some_callback ], :void

returned_struct = some_function(proc do |struct|
  # struct is an instance of SomeStruct, instead of an FFI::Pointer
end)

attach_function :some_other_function, [ ], SomeStruct.by_ref
some_other_function.is_a?(SomeStruct) # true, instead of being an FFI::Pointer

Keep in mind, that on JRuby 1.7.3, FFI::Struct.by_ref type accepts any descendant of FFI::Struct, and not only instances of YourStruct. See https://github.com/jruby/jruby/issues/612 for updates.

Piggy-back on Ruby’s garbage collection with regular FFI::Structs

If we take a look again at the above code with SomeStruct as return value.

attach_function :some_other_function, [ ], SomeStruct.by_ref

In some libraries, the memory for the pointer to SomeStruct returned from some_other_function is expected to be managed by us. This means we’ll most likely need to call some function free_some_struct to specifically free the memory used by SomeStruct when the object is no longer needed. Here’s how it would be used:

begin
  some_struct = some_other_function
  # do something with some_struct
ensure
  free_some_struct(some_struct)
end

Unfortunately, if we pass some_struct somewhere else beyond our control, we must be able to trust that the new guardian of some_struct calls free_some_struct in the future, or we will have a memory leak! Oh no!

Fear not, for FFI::Struct has a trick up it’s sleeve for us. Have a look at this.

class SomeStruct < FFI::Struct
  def self.release(pointer)
    MyFFIBinding.free_some_struct(pointer) unless pointer.null?
  end
end

attach_function :some_other_function, [], SomeStruct.auto_ptr

With the above binding code, some_other_function still returns an instance of SomeStruct. However, when our object is garbage collected FFI will call upon SomeStruct.release to free the native memory used by our struct. We can safely pass our instance of SomeStruct around everywhere and to everyone, and safely remember that when the object goes out of scope and Ruby garbage collects it, FFI will call upon us to free the underlying memory!

Related to this, you should look into FFI::ManagedStruct and FFI::AutoPointer if you have not already.

Writing our own data types

class Device < FFI::Pointer
end
attach_function :some_function, [ ], Device

Subclassing FFI::Pointers is a convenient way of working with pointers from native libraries less generic. Using the above code, when we call some_function we’ll receive an instance of Device, instead of the FFI::Pointer we would get if we specified the return value as a :pointer.

If objects in our native library are not pointers we can’t do what we’ve done above. For example, in OpenAL there’s a concept of audio sources, but they are represented by an integer, and not a pointer. Passing arbitrary integers around is not a nice practice, so what you could do is wrap the source in an object for further use.

class Source
  def initialize(id)
    @id = id
  end
  attr_reader :id
end

typedef :int, :source
attach_function :create_source, [], :source
attach_function :destroy_source, [ :source ], :void

# Usage
source = Source.new(create_source)
destroy_source(source.id)

While the code above is not bad, we could do much better by utilizing something in FFI called DataConverters. DataConverters are a way of writing code that tells FFI how to convert a native value to a ruby value and back. By doing this, we could have FFI automatically wrap source above in an object, making it completely transparent to the developer using the library.

class Source
  extend FFI::DataConverter
  native_type FFI::Type::INT

  class << self
    # `value` is a ruby object that we want to convert to a native object
    # this method should return a type of the native_type we specified above
    def to_native(value, context)
      if value
        value.id # in our case, we convert a Source to an int
      else
        -1 # if value is nil, we represent a `no source` value as -1
      end
    end

    # `value` is a type of the native_type specified above, we should return
    # a ruby object we wish to pass around in our application
    def from_native(value, context)
      new(value)
    end

    # this is needed when FFI needs to figure out the native size of your native type
    # for example, if you want to generate a pointer to hold something of this type
    # e.g. FFI::MemoryPointer.new(Source) # <= requires size to be defined and correct
    def size
      FFI.type_size(FFI::Type::INT)
    end

    # this method is a hint to FFI that the object returned from to_native needs to
    # be kept alive for the native value in the object to remain valid, so that if we
    # return an object that automatically frees itself on garbage collection, ffi will
    # prevent it from being garbage collected while it’s still needed, mainly useful
    # for to_native methods that allocate memory
    def reference_required?
      false
    end
  end

  def initialize(id)
    @id = id
  end

  attr_reader :id
end

attach_function :create_source, [], Source
attach_function :destroy_source, [ Source ], :void

source = create_source # an instance of Source, created through Source.from_native!
source.id # => the native value
destroy_source(source) # converts source to native value through Source.to_native!

You could do this to all types, even pointers. Even more, you are not constrained to only doing type conversion in to_native and from_native — you could perform validation, making sure your values have the correct type, length, or what ever you may need!

If you’d like some more example of custom types, I’ve written down a few in this gist: https://gist.github.com/elabs-dev/41c27fdb0a007ad4cac6

Implementing type safety

Do you remember what I mentioned earlier about FFI::Struct.by_ref automatically giving us some kind of type safety, preventing us from shenanigans where somebody sends invalid values to native functions? We can implement the very same kind of type safety ourselves for all types, by overriding to_native in our DataConverters.

# A to_native DataConverter method that raises an error if the value is not of the same type.
module TypeSafety
  def to_native(value, ctx)
    if value.kind_of?(self)
      super
    else
      raise TypeError, "expected a kind of #{name}, was #{value.class}"
    end
  end
end

We could now mix the above module into our own custom data types from the previous chapters.

# Even if we have another object that happens to look like a Source from our previous chapter,
# by having a #value method, we now won’t allow sending it down to C unless it’s an instance of
# Source or any of it’s subclasses.
Source.extend(TypeSafety)

# Remember Device from earlier? It’s a descendant of FFI::Pointer. Now all parameters of type Device
# will only accept instances of Device or any of it’s subclasses. All else results in a type error.
Device.extend(TypeSafety)

Duck-typing is very useful in Ruby, where raising an exception is the worst thing that can happen when we try to call a method on an object that does not respond to such a method. However, when interfacing with C libraries, passing in the wrong type will segfault your application with little information on what went wrong. Using this TypeSafety module, we can catch errors early, with a useful error message as a result.

Final words

Personally I really like using FFI. It’s a low-pain way of writing gems that use native libraries, and if you set your types up properly, not having a compiler that type-checks your code won’t be so bad. If you can work with native libraries through the means of FFI instead of writing a C extension, by all means do. Even if you intend on writing a C extension, using FFI can be a quick way of exploring a native API without wiring up C functions and data structures together with the Ruby C API.

Something that FFI excells at, in comparison to writing a C extension, is handling asynchronous callbacks from non-ruby threads in C. FFI can save you a lot of headache in that area.

Thank you.

References

Mar

Handle secret credentials in Ruby On Rails


This blog post aims to lay out a simple and concrete strategy for handling sensitive data in your Ruby On Rails applications, and to explain the importance of such a strategy.

Never, ever check them into source control

Even if your project is closed source and your trusted colleagues are the only ones with access, you never know when a freelancer or consultant might be joining the project. Even if that never occurs, how do you keep track of all the locations where that repository is checked out? Who knows on how many hard drives your company's credit card transaction secret API key might be stored. What happens when someone with a weak login password forgets their laptop on the bus or at the airport?

Also note that it's not always as simple as removing secrets after the fact, especially with version control. It's usually impossible to do this without drastically changing your entire project's history!

Do it right

For a long time, we've been using YAML files to store our application configuration. It's easy to manage and can be configured for different Rails environments. These YAML files could look like the following:

config/app.yml:

development: &defaults
  awesomeness_score: 3
  host: "localhost:3000"
  s3_bucket: "example-development-us"

production:
  <<: *defaults
  host: "example.com"
  s3_bucket: "example-production-us"

test:
  <<: *defaults

config/app_secret.yml.example:

  development: &defaults
  aws_access_key_id: ""
  aws_secret_access_key_id: ""

production:
  <<: *defaults

test:
  <<: *defaults

config/app_secret.yml:

development: &defaults
  aws_access_key_id: "ACTUAL-ID-WOULD-GO-HERE"
  aws_secret_access_key_id: "ACTUAL-SECRET-WOULD-GO-HERE"

production:
  <<: *defaults

test:
  <<: *defaults

Only the first two files would be checked in to source control, and the application's README would instruct developers to cp config/app_secret.yml.example config/app_secret.yml and fill in the gaps from the company keychain.

To make sure we never check in the secrets by mistake, we ignore the app_secret.yml file:

.gitignore:

# ...
/config/app_secret.yml

We then use the econfig gem written by Jonas Nicklas to easily merge them together:

Gemfile

# ...
gem "econfig", require: "econfig/rails"

config/application.rb

# ...
module YourApp
  extend Econfig::Shortcut
  # ...
end

Now we can access any configuration variable and secret credential:

YourApp.host # => "localhost:3000"
YourApp.aws_secret_access_key_id # => "ACTUAL-SECRET-WOULD-GO-HERE"

Deploy

When you deploy the application, you must manually manage the secrets on the server(s).

Capistrano

If you deploy with Capistrano, you'll want to place the app_secret.yml in your /shared folder. Once that's done, it can be copied to each release with symlink task:

deploy.rb

# ...
namespace :config do
  desc "Symlink application config files."
  task :symlink do
    run "ln -s {#{shared_path},#{release_path}}/config/app_secret.yml"  
  end
end

after "deploy", "config:symlink"

Heroku

If you're deploying your application where you don't have file access, such as Heroku, you're better off storing this kind of information in ENV. The econfig gem has built in support for this and a few other storage backends, but that's another blog post.

Conclusion

With this method, we now have a clear separation of sensitive and non-sensitive data. There's no risk of checking in any sensitive data, since we have only one place to put it all and it's hidden from source control. Data access within the application hasn't changed, and we no longer have to concern ourselves with how sensitive it is.

We can now be sure that giving access to a repository does not imply giving access to other systems.

Epilogue

If you have any feedback on how the blog post can be improved, or if you spot any errors, please let me know by posting a comment below!

Nov

Simple authorization in Ruby on Rails apps


Here at Elabs, we've been using CanCan for authorization in a number of applications. Ryan Bates managed to build an authorization system which is both simple and powerful. A step away from the bloated role based system available at the time, yet more sophisticated than simply tacking on methods on ActiveRecord models.

Over time though we've come against a few grievances with CanCan.

  • Ability files quickly become too big to manage, and there is no built in strategy for splitting up abilities across multiple files.
  • Even worse, there is no natural way to structure ability files. We usually resort to comments to divide the file into sections for different models.
  • All ability rules need to be evaluated for every request. While not a huge performance hit, it seems like a built in wastefulness.

And finally: at the time of writing, CanCan has 128 open issues, 28 open pull requests. Important functionality in the gem is broken, and attempts to fix it through pull requests are ignored. The test suite depends on ActiveRecord < 3.1 and won't even run with later versions of ActiveRecord, unless someone fixes this, we don't actually know if CanCan works at all with newer versions of AR.

In a recent project we worked on, we were running against bugs in CanCan which forced us to run a forked version, and we were fighting against an ability file which was growing out of control. We decided that we needed a new way to approach the problem.

Back to basics

We really like CanCan's simple approach. The ability file isolates all authorization logic, and it leaves you free to handle authorization however you want to. You are free to grow your authorization system from a single user role to whatever complexity you need. We were intent on keeping this flexibility.

We wanted something simpler though. Something which we can implement without really needing a library at all. We wanted to have full control over how the authorization system works.

We took inspiration from objectify and Bryan Helmkamp's excellent blog post 7 Patterns to Refactor Fat ActiveRecord Models among others and pared the whole thing down to creating a plain Ruby class for each domain model.

We call these classes policies and we put them in app/policies. They might look like this:

class PostPolicy
  attr_reader :user, :post

  def initialize(user, post)
    @user = user
    @post = post
  end

  def create?
    user.admin? or not post.published?
  end
end

Using these classes from the controller is fairly easy:

def create
  @post = Post.new(params[:post])
  raise NotAuthorizedError unless PostPolicy.new(current_user, @post).create?
  if @post.save
    redirect_to @post
  else
    render :new
  end
end

This works quite nicely, but unfortunately it's a lot more code to write in the controller. Controllers are un-DRY enough as it is; we need to make this easier. It's simple enough to introduce a helper method for fetching a policy for a given record:

def policy(record)
  "#{record.class}Policy".constantize.new(current_user, record)
end

Now we can simplify our create method somewhat.

def create
  @post = Post.new(params[:post])
  raise NotAuthorizedError unless policy(@post).create?
  …
end

We can easily wrap this pattern in another method:

def authorize(record)
  raise NotAuthorizedError unless policy(record).public_send(params[:action] + "?")
end

And we end up with this:

def create
  @post = Post.new(params[:post])
  authorize(@post)
  …
end

That looks a lot closer to what we have in CanCan.

This pattern works fine for 6 of the 7 restful actions, but what about #index?

CanCan can automatically construct a query based on the permissions you have specified, as long as the hash based syntax is used, anyway. Unfortunately we've found this magic to be error prone and sometimes insufficient. We really want to use scopes for this, but we don't want those to pollute our model objects. Again, taking inspiration from Bryan's blog post, we create a class for this:

class PostPolicy < Struct.new(:user, :post)
  class Scope < Struct.new(:user, :scope)
    def resolve
      if user.admin?
        scope
      else
        scope.where(:published => true)
      end
    end
  end

  …
end

Usage for this from the index action is also fairly easy:

def index
  @posts = PostPolicy::Scope.new(current_user, Post.scoped).resolve
end

Again, we can simplify this with a helper method:

def index
  @posts = policy_scope(Post.scoped)
end

Both the policy and policy_scope method are especially useful in views. We can do things like this:

<% policy_scope(@category.posts).each do |post| %>
  <li>
    <h2><%= post.title %></h2>
    <p><%= link_to "Edit", [:edit, post] if policy(post).edit? %></p>
  </li>
<% end %>

Our views are kept quite nice and DRY, just like with CanCan.

We have bundled up these helpers in a very simple gem we're calling Pundit. It has a few more tricks up its sleeves, but basically it does exactly what this post outlines. We found that we could replace CanCan with this pattern very effectively. The resulting code is simpler, easier to understand and easier to test.

Conclusion

While we do think that Pundit has been useful to us, there is a bigger takeaway from this. We had a problem, and we threw a large library with a complicated DSL at the problem, and as the old saying goes, we now had two problems. Sometimes the simpler solution is better. Sometimes it makes sense to leverage Ruby, over creating your own mini language.

Spend some time reconsidering the dependencies you have in your application and whether they are actually helping you, or if you're spending more time fighting them than you're getting out.

Mar

Working with time zones in Ruby on Rails


Rails provides great tools for working with time zones but there's still a lot of things that can go wrong. This blog post aims to shed some light on these gotchas and provide solutions to the most common problems.

The one that probably has tricked me the most times is the fact that Rails fools you to believe it got you all covered all the time (pardon the pun). Don't get me wrong. I want Rails to do as much work for me as possible. But I've learnt the hard way that I can't get away with not knowing when and how Rails is helping me. Another gotcha is the fact that you have more time zones in play than you might first believe. Consider the following: db, server, dev machine, system configured, user specific configured and the browser.

Configure your Rails app

So what tools do we have at our disposal as Rails developers? The most important one is the config.time_zone configuration in your config/application.rb file. ActiveRecord will help you convert from and to (which the documentation fails to explain) UTC and the time zone of your choice. This means that if all you're doing is having users post times through a form and use Active Record to persist it you're good to go.

Processing time information

So what about actually doing something with the time information before persisting it? That's when it becomes tricky.

Parsing

When parsing time information it's important to never do it without specifying the time zone. The best way to do this is to use Time.zone.parse (which will use the time zone specified in config.time_zone) instead of just Time.parse (which will use the computer's time zone).

Work with Numerical and ActiveRecord attributes

Method calls like 2.hours.ago uses the time zone you've configured, so use these if you can! The same thing is true for time attributes on ActiveRecord models.

post = Post.first
post.published_at #=> Thu, 22 Mar 2012 00:00:00 CDT -05:00

ActiveRecord fetches the UTC time from the database and converts it to the time zone in config.time_zone for you.

Date vs Time

Time has date information but Date does NOT have time information. Even if you don't think you care you might realize that you do sooner then later. Be safe and use Time (or DateTime if you need support for times very far from the present).

But let's say you're stuck with a Date that you need to treat as a Time, at least make sure to convert it to your configured time zone:

1.day.from_now # => Fri, 03 Mar 2012 22:04:47 JST +09:00
Date.today.in_time_zone # => Fri, 02 Mar 2012 00:00:00 JST +09:00

Never use:

Date.today.to_time # => 2012-03-02 00:00:00 +0100

Querying

Since Rails know that your time information is stored as UTC in the database it will convert any time you give it to UTC.

Post.where(["posts.published_at > ?", Time.current])

Just be sure to never construct the query string by hand and always use Time.current as the base and you should be safe.

Working with APIs

Supplying

Building a web API for others to consume? Make sure to always send all time data as UTC (and specify that this is the case).

Time.current.utc.iso8601 #=> "2012-03-16T14:55:33Z"

Read more about why iso8601 is advisable here: http://devblog.avdi.org/2009/10/25/iso8601-dates-in-ruby/

Consuming

When you get the time information from an external API which you don't have control over you simply need to figure out the format and time zone it's sent to you with. Because Time.zone.parse might not work with the format you receive you might need to use:

Time.strptime(time_string, "%Y-%m-%dT%H:%M:%S%z").in_time_zone

This assumes time_string a iso8601 formated string. strptime will throw a very unintuitive error complaining on the format argument when in reality the problem is that the time string's format mismatches the format template argument. in_time_zone defaults to use the Rails configured time zone.

Why there's no strptime method on Time.zone when there's a parse beats me.

Working with multiple user time zones

Many systems needs to support users entering and viewing time information in a variety of time zones. To achieve this you need to store each user's time zone (probably just one of the time zone string names found in rake time:zones:all). Then to actually use that time zone the most common pattern is to simply create a private method in your ActionController and run it as an around filter.

around_filter :user_time_zone, :if => :current_user

def user_time_zone(&block)
  Time.use_zone(current_user.time_zone, &block)
end

This will do the same thing as config.time_zone but on a per request basis. I still recommend to change the default config.time_zone to a time zone that is a good default for your users. (Thank you Matt Bridges for pointing out the potential problems with using a before_filter instead of an around_filter.)

Testing

All the above is something that your tests should catch for you. The problem is that you as the user and your computer as the development server happen to reside in the same time zone. This is rarely the case once you push things to production.

There is Zonebie, a gem that helps you deal with this. I haven't had time to try it out myself yet, but it looks promising. If you find this to be overkill, at least make sure that your tests run with Time.zone set to another time zone than the one your development machine is in!

Cheat Sheet

DOs

2.hours.ago # => Fri, 02 Mar 2012 20:04:47 JST +09:00
1.day.from_now # => Fri, 03 Mar 2012 22:04:47 JST +09:00
Date.today.in_time_zone # => Fri, 02 Mar 2012 22:04:47 JST +09:00
Date.current # => Fri, 02 Mar
Time.zone.parse("2012-03-02 16:05:37") # => Fri, 02 Mar 2012 16:05:37 JST +09:00
Time.zone.now # => Fri, 02 Mar 2012 22:04:47 JST +09:00
Time.current # Same thing but shorter. (Thank you Lukas Sarnacki pointing this out.)
Time.zone.today # If you really can't have a Time or DateTime for some reason
Time.current.utc.iso8601 # When supliyng an API (you can actually skip .zone here, but I find it better to always use it, than miss it when it's needed)
Time.strptime(time_string, "%Y-%m-%dT%H:%M:%S%z").in_time_zone # If you can't use time.zone.parse

DON'Ts

Time.now # => Returns system time and ignores your configured time zone.
Time.parse("2012-03-02 16:05:37") # => Will assume time string given is in the system's time zone.
Time.strptime(time_string, "%Y-%m-%dT%H:%M:%S%z") # Same problem as with Time#parse.
Date.today # This could be yesterday or tomorrow depending on the machine's time zone.
Date.today.to_time # => # Still not the configured time zone.

Epilogue

I hope you've learned something from this post. I sure did while writing it! If you have any feedback on how it can be improved, or if you spot any errors, please let me know by posting a comment below!

Ruby and Rails version

This article was first written in March 2012. Back then Rails 3.2 was the new hot and as you all know a lot happens in Rails-land in two and a half years and will continue to do so. I will do my best to keep the article accurate and up to date with the latest versions of Rails. If you spot anything that is reported deprecated or not working please let me know in the comment section below!

  • Article publish date: 2012-03-20
  • Article last updated: 2014-11-04
  • Last verified Rails version: 4.1.7
  • Last verified Ruby version: 2.1.4p265

There is a git repository which you can clone:

git clone git@github.com:ramhoj/time-zone-article.git
cd time-zone-article
bundle install
rake db:create:all db:migrate db:test:prepare
rspec spec/

The Rails application is running on the version defined above and has been verified to work under the described Ruby version above. If you want to make sure things are working in the version of Rails or Ruby that you're using please fork the repository and make the necessary adjustments and run the test suite. If you want more in-debt, hands-on of the examples this repository's test suite aims to help with this too.

Changelog

See the git repository's commits.

Mar

Working with time zones in Ruby on Rails


Rails provides great tools for working with time zones but there's still a lot of things that can go wrong. This blog post aims to shed some light on these gotchas and provide solutions to the most common problems.

The one that probably has tricked me the most times is the fact that Rails fools you to believe it got you all covered all the time (pardon the pun). Don't get me wrong. I want Rails to do as much work for me as possible. But I've learnt the hard way that I can't get away with not knowing when and how Rails is helping me. Another gotcha is the fact that you have more time zones in play than you might first believe. Consider the following: db, server, dev machine, system configured, user specific configured and the browser.

Configure your Rails app

So what tools do we have at our disposal as Rails developers? The most important one is the config.time_zone configuration in your config/application.rb file. ActiveRecord will help you convert from and to (which the documentation fails to explain) UTC and the time zone of your choice. This means that if all you're doing is having users post times through a form and use Active Record to persist it you're good to go.

Processing time information

So what about actually doing something with the time information before persisting it? That's when it becomes tricky.

Parsing

When parsing time information it's important to never do it without specifying the time zone. The best way to do this is to use Time.zone.parse (which will use the time zone specified in config.time_zone) instead of just Time.parse (which will use the computer's time zone).

Work with Numerical and ActiveRecord attributes

Method calls like 2.hours.ago uses the time zone you've configured, so use these if you can! The same thing is true for time attributes on ActiveRecord models.

post = Post.first
post.published_at #=> Thu, 22 Mar 2012 00:00:00 CDT -05:00

ActiveRecord fetches the UTC time from the database and converts it to the time zone in config.time_zone for you.

Date vs Time

Time has date information but Date does NOT have time information. Even if you don't think you care you might realize that you do sooner then later. Be safe and use Time (or DateTime if you need support for times very far from the present).

But let's say you're stuck with a Date that you need to treat as a Time, at least make sure to convert it to your configured time zone:

1.day.from_now # => Fri, 03 Mar 2012 22:04:47 JST +09:00
Date.today.in_time_zone # => Fri, 02 Mar 2012 00:00:00 JST +09:00

Never use:

Date.today.to_time # => 2012-03-02 00:00:00 +0100

Querying

Since Rails know that your time information is stored as UTC in the database it will convert any time you give it to UTC.

Post.where(["posts.published_at > ?", Time.current])

Just be sure to never construct the query string by hand and always use Time.current as the base and you should be safe.

Working with APIs

Supplying

Building a web API for others to consume? Make sure to always send all time data as UTC (and specify that this is the case).

Time.current.utc.iso8601 #=> "2012-03-16T14:55:33Z"

Read more about why iso8601 is advisable here: http://devblog.avdi.org/2009/10/25/iso8601-dates-in-ruby/

Consuming

When you get the time information from an external API which you don't have control over you simply need to figure out the format and time zone it's sent to you with. Because Time.zone.parse might not work with the format you receive you might need to use:

Time.strptime(time_string, "%Y-%m-%dT%H:%M:%S%z").in_time_zone

This assumes time_string a iso8601 formated string. strptime will throw a very unintuitive error complaining on the format argument when in reality the problem is that the time string's format mismatches the format template argument. in_time_zone defaults to use the Rails configured time zone.

Why there's no strptime method on Time.zone when there's a parse beats me.

Working with multiple user time zones

Many systems needs to support users entering and viewing time information in a variety of time zones. To achieve this you need to store each user's time zone (probably just one of the time zone string names found in rake time:zones:all). Then to actually use that time zone the most common pattern is to simply create a private method in your ActionController and run it as an around filter.

around_filter :user_time_zone, :if => :current_user

def user_time_zone(&block)
  Time.use_zone(current_user.time_zone, &block)
end

This will do the same thing as config.time_zone but on a per request basis. I still recommend to change the default config.time_zone to a time zone that is a good default for your users. (Thank you Matt Bridges for pointing out the potential problems with using a before_filter instead of an around_filter.)

Testing

All the above is something that your tests should catch for you. The problem is that you as the user and your computer as the development server happen to reside in the same time zone. This is rarely the case once you push things to production.

There is Zonebie, a gem that helps you deal with this. I haven't had time to try it out myself yet, but it looks promising. If you find this to be overkill, at least make sure that your tests run with Time.zone set to another time zone than the one your development machine is in!

Cheat Sheet

DOs

2.hours.ago # => Fri, 02 Mar 2012 20:04:47 JST +09:00
1.day.from_now # => Fri, 03 Mar 2012 22:04:47 JST +09:00
Date.today.in_time_zone # => Fri, 02 Mar 2012 22:04:47 JST +09:00
Date.current # => Fri, 02 Mar
Time.zone.parse("2012-03-02 16:05:37") # => Fri, 02 Mar 2012 16:05:37 JST +09:00
Time.zone.now # => Fri, 02 Mar 2012 22:04:47 JST +09:00
Time.current # Same thing but shorter. (Thank you Lukas Sarnacki pointing this out.)
Time.zone.today # If you really can't have a Time or DateTime for some reason
Time.current.utc.iso8601 # When supliyng an API (you can actually skip .zone here, but I find it better to always use it, than miss it when it's needed)
Time.strptime(time_string, "%Y-%m-%dT%H:%M:%S%z").in_time_zone # If you can't use time.zone.parse

DON'Ts

Time.now # => Returns system time and ignores your configured time zone.
Time.parse("2012-03-02 16:05:37") # => Will assume time string given is in the system's time zone.
Time.strptime(time_string, "%Y-%m-%dT%H:%M:%S%z") # Same problem as with Time#parse.
Date.today # This could be yesterday or tomorrow depending on the machine's time zone.
Date.today.to_time # => # Still not the configured time zone.

Epilogue

I hope you've learned something from this post. I sure did while writing it! If you have any feedback on how it can be improved, or if you spot any errors, please let me know by posting a comment below!

Ruby and Rails version

This article was first written in March 2012. Back then Rails 3.2 was the new hot and as you all know a lot happens in Rails-land in two and a half years and will continue to do so. I will do my best to keep the article accurate and up to date with the latest versions of Rails. If you spot anything that is reported deprecated or not working please let me know in the comment section below!

  • Article publish date: 2012-03-20
  • Article last updated: 2014-11-04
  • Last verified Rails version: 4.1.7
  • Last verified Ruby version: 2.1.4p265

There is a git repository which you can clone:

git clone git@github.com:ramhoj/time-zone-article.git
cd time-zone-article
bundle install
rake db:create:all db:migrate db:test:prepare
rspec spec/

The Rails application is running on the version defined above and has been verified to work under the described Ruby version above. If you want to make sure things are working in the version of Rails or Ruby that you're using please fork the repository and make the necessary adjustments and run the test suite. If you want more in-debt, hands-on of the examples this repository's test suite aims to help with this too.

Changelog

See the git repository's commits.