Blog


Dec

Has Microsoft become altruistic or is cooperation simply better than competition?


The market economy prides itself on its competitive nature. Through competition we’re promised better and cheaper products and services. I believe this to be true in countless cases. I also believe that we need to be better at considering both the benefits and the inefficiencies that the competitive model brings. As with any dogma it can do just as much harm as good unless constantly being evaluated from a case-to-case perspective.

Retrieved from http://www.teslamotors.com/models/photo-gallery

Consider for example what companies do to protect their investments in research and development. Companies are constantly suing each other over patents and spending billions on lobbying to make IP-laws stricter and more far-reaching. We buy this concept because we believe competition is more effective than cooperation. It’s been this way for so long that it’s hard to even imagine the alternative. Who knows what kind of cars we would have if all the car manufactures cooperated in their RnD. Maybe we would all have been driving the Tesla Model S ten years ago, maybe we would still be stuck in a Trabant, who knows?

Given this blog post being on a software development firm’s blog you’ve probably already figured out where this is going. Open Source is a large-scale, unconditional, more or less all-in choice of cooperation before competition, and it works! 59% of all web sites run on open source web servers. 95% of smart phones run on open source technology. 82% of users access the Internet using a web browser built on an open source foundation. With Microsoft’s recent release of the .NET framework as open source pretty much all major programming environments are now open source.

Retrieved from http://www.flickr.com/photos/gforsythe/8174197748/ Illustration by Giulia Forsythe

In my experience open source projects tend to be numerous and diverse, allowing the user to choose a project that suits their needs. Often times they have large and helpful communities, answering questions in time zones all around the globe. Open source products are often free, and thus available to more users. Simply giving more people access to others’ creations can spawn innovation that would be lost in a closed environment.

As a software consultant I know that trust is key. When there is lack of trust there is legal overhead, misinformation and static control structures that prevent agility. These projects tend to be more expensive and deliver inferior results. Open source requires a whole bunch of trust. Put yourself in Microsoft’s shoes. Would you find it easy to “give away” technology that you’ve spent billions developing? You’d need to put your trust in people you’ve never met, hoping you will get back as much as you’ve given.

Open source in the software industry might be the most obvious example of trusting others to do something great(er) together. But in fact our entire civilization requires us to trust in each other. Without trust currency would be hugely inconvenient, e-commerce would be impossible and we would have no taxes to pay for services such as law, defense and education.

I don’t believe every sector would benefit from unconditional cooperation but I’m certain that there are vast possibilities to create better, cheaper, fairer, and safer products and services if we start trusting each other more!

Which sectors do you think would benefit from replacing competition with cooperation? How can the software industry benefit from even more cooperation? Or have we gone too far? Use the comment section below!

Mar

How to write a bug report


This is a blog post I've wanted to write for a really long time. Every couple of days I get an issue reported to one of my open-source projects which looks something like this:

Capybara doesn't work

It's totally broken. I tried everything. Why doesn't it work? Please fix it for me!

Okay, maybe it's not quite that bad. As you might guess, such a bug report is completely useless, it's just a burden to everyone involved in the project to have to tackle this kind of stuff. Worse than that, the person who posted this isn't going to get the help they need, and their possibly very legitimate issue is not going to get fixed.

Every time I receive one of these I wish there were some kind of source I could point these people to, so they might learn how to contribute with insight to open-source projects, instead of being a burden. This is that source.

Closed source

This stuff only applies to open-source, free as in beer and speech projects. Closed source projects are a different game, though some of this advice may still be useful.

The holy grail of bug report templates

This is where you should start. I can't remember which project I first saw this template on, so unfortunately I can't give credit where it's due, suffice to say I didn't invent it and am not claiming credit for it.

Answer these three very simple questions:

  • What did you do?
  • What did you expect to happen?
  • What happened instead?

Just by answering these three questions, you're already doing better than most bug reports out there.

If you encounter an error, any kind of error, make sure to include a stack trace. Those are immensely useful. Also be sure that the stack trace is complete, some frameworks such as Rails or test frameworks like RSpec and Cucumber tend to filter the stack trace. In RSpec and Cucumber, run your tests with -b to get a full stack trace. If you can't get a stack trace, make sure to expand upon the third question as much as possible.

A bug report isn't a question

If you're asking why something broke, if you're wondering what the cause of a particular behaviour is, you aren't actually writing a bug report, you're asking a question.

Asking questions is good, and I encourage you to reach out when you need help and have exhausted all other options. Asking questions on an issue tracker however is bad. Issues need to be closed, they need to be resolved; a question cannot really be resolved. It can be answered, but then that answer needs to be acknowledged and accepted and so on. There is a lot of process involved in closing each issue and it takes time away from the maintainers to focus on more important things.

Most open source project have a mailing list, use that for asking questions. If there isn't one, use StackOverflow or a more generic mailing list. If you've posted to one of these and you didn't get an answer in two weeks or so, investigate again and see if you can't find the answer yourself. If you can find it, make sure to answer your own question in the forum you asked it, so that others who have the same question in the future may find your answer through the almighty Google. If all of that's failed, that's when you're allowed to open an issue for a question.

When you send a bug report, you should be able to at least speculate at what the cause might be. Saying feature X broke doesn't help at all. Most open source projects of decent size are used by thousands of people, they have massive, comprehensive test suites. The most basic functionality in the project very likely isn't broken. So even if you have a legitimate issue, it likely happened under a specific set of circumstances. You need to pare away possible causes until you are left with the most minimal set of conditions needed to replicate the issue. Which brings us to:

Replication

While it isn't strictly necessary for a bug report, it's immensely helpful to have some way to replicate the issue. It's especially helpful to have an executable way of replicating an issue.

I generally dislike sample applications, such as Rails applications for this. There are too many possible areas where breakage could occur, and it takes too long for me to understand how all the pieces fit together. If you need a complete sample application, you likely don't understand the issue well enough yet to send a good bug report. Pare it down further until you truly understand what components cause the problem.

The absolute best way is of course to send a failing test case. You definitely don't need to actually fix the problem, though of course we appreciate it if you do. Understanding a code base can be troublesome, so it's cool if you don't want to spend the time to understand how best to resolve an issue. But if you do spend the time to write a test case, you reduce the time it takes the maintainers to solve your problem by an order of magnitude. Seriously.

A well formatted email

If you're sending a message to a mailing list, be aware that reading code or stack traces inline in an email message is quite painful. Either keep the code samples very short or link to a pastie/gist or something. Reading long code samples without syntax highlighting and possibly with automatically wrapped lines is quite painful.

Debugging

Debugging is a central part of software development. You cannot get by as a programmer without being good at debugging. Use the skills you use to solve problems in your applications when you encounter what you perceive as bugs in open source projects. Debugging really is a simple process:

  1. Isolate the cause
  2. Fix it

Where 99.8%¹ of the time is spent on point 1. If you don't do the first point, you are effectively asking someone else to do most of your work for you. For free.

Be polite

The people who write open-source projects do not owe you anything. They've invested a lot of time, which you get for free, but they are under no obligation to keep providing that time to you, or to provide more of their time just because you demand it.

The most common form of this I see are the emails I get to my personal email account from people asking for help. Asking me for help in a private forum such as email is asking for a handout. It's a form of begging. Please don't do it. I hate getting these emails, and I hate having to reply that I don't answer questions sent to me privately.

Some people believe that instead, you owe authors for the software they provide, but I don't really believe that either. I think the only thing you do owe them is to treat them with the same respect and politeness that you would extend to anyone else.

So no one owes each other anything really, and that's cool.

Don't be disappointed by the way, if it takes a while for your issue to get a response. It's cool if you ping the author after a couple of weeks if nothing has happened. Sometimes one can lose motivation, or become engrossed in something else.

Summary

Be polite. Be explicit. Be as knowledgeable about the causes of your issue as you can be. Answer these following questions:

  • What did you do?
  • What did you expect to happen?
  • What happened instead?

¹ totally made up number

Jan

Why Serenade.js?


Yesterday I opened up the repo for Serenade.js to the public, and discussion ensued on Twitter and Hacker News. A lot of people are asking why we would need yet another client side framework, so I thought I would write down why I think Serenade offers something we haven't really seen yet.

MVC

A lot of the popular client side frameworks today claim to follow the MVC pattern. But the way the MVC pattern was originally envisioned is very different from what any client side MVC framework is doing today. This paper outlines the MVC pattern used by the Smalltalk-80 interface, developed at Xerox PARC. It's that old, and still very relevant.

The paper outlines the responsibilities of the MVC triad as follows:

Views: Present data from the model and update it if it changes, notify controllers of user interaction events.
Controllers: React to user interaction events by instructing the model to perform certain actions.
Models: Handle business logic and persistence, notify the view of any changes to the data.

A notable difference to how most MVC frameworks today handle this is that it is often the controller's responsibility to present data to the view, and make sure that that data is up to date, giving the controller much bigger responsibility than it would otherwise have. Oftentimes the controller is also responsible for rendering the view, and as in Backbone, for placing it in a particular location on the page.

By clearly defining the role of the controller as only reacting to user events, we can isolate it from a lot of concerns. We are following the single responsibility principle, and as an added bonus, we've made it possible to test our controllers without even involving the DOM.

I tried to follow the Smalltalk-80 style of MVC very closely in Serenade. The almost unexpected side effect of this was that neither models, nor controllers needed any kind of special logic. In Ember.js for example, all objects you want to use inherit from Ember.Object. In Serenade you can use any JavaScript object as a controller or a model, which means you can use it in an existing application or in just a fragment of an application without having to bend your entire app around it.

Templates

The template language in Serenade.js is very carefully constructed so that binding data to both attributes and text is possible, without a lot of additional code in the templates, and without any additional markup. Consider the following example:

h1[title=@name] "His name is " @name

The resulting markup from this would look like this:

<h1 title="Jonas">His name is Jonas</h1>

No additional markup at all! And yet, when I update name to "Peter", the markup just magically changes to this:

<h1 title="Peter">His name is Peter</h1>

And it does this without changing anything that doesn't need to change. The text "His name is" for example is not affected at all by this update.

I came to the conclusion that doing this is only really possible by writing a purpose built template language.

Let's look at this:

ul#comments
  - collection @comments
    li @title

Which would generate something like this:

<ul id="comments">
  <li>Hello</li>
  <li>Awesome</li>
</ul>

Just adding another comment by pushing it onto comments post.comments.push({ title: 'It works' }) would append it at the end:

<ul id="comments">
  <li>Hello</li>
  <li>Awesome</li>
  <li>It works</li>
</ul>

With no additional markup, with nothing explicit needed to make this dynamically update. It's all simple, clean and efficient. Without writing a custom template language this would simply not have been possible.

In contrast, frameworks like Ember and Batman need to add a lot of additional markup for similar functionality, and it's much more difficult for them to do these kinds of things without unexpected side effects.

jQuery

I'm not a huge fan of jQuery. It does what it does very well, and I use it on a majority of projects, but sometimes I want to use something else. I really intensely dislike the fact that pretty much all client side frameworks depend on jQuery. It stifles innovation because it makes it impossible for competitors to have a chance. I wanted to use MooTools for a project recently. It would have been a better fit. But I couldn't, because my framework forced me to use jQuery, and I didn't want to use more than one library.

As far as I can see, Batman is the only framework that is usable without jQuery, using Batman.Solo. Kudos to the Batman.

Serenade has no dependencies. It achieves everything it does by using normal DOM manipulation. This means we probably have to give up on ever supporting IE6, since it has too many weird quirks to work around, but all other browsers in use today, yes even IE7, work without needing a lot of tweaking.

And as an added bonus, if you are using jQuery, just call Serenade.useJQuery() before rendering any views and Serenade will use jQuery for event bindings, instead of its own custom code.

Clear point of entry

In the end we want to use frameworks client side to dynamically manipulate the DOM. I found that most frameworks didn't start at this point though. There are routers, controllers, dispatchers, and all other kinds of things that you would need to add in order to actually get to the point where DOM nodes are created.

Serenade turns this whole concept on its head. We start off by generating DOM nodes. No routers, no dispatch, nothing. Just render a view and insert it into the DOM:

var element = Serenade.view('h1 "Hello world"').render();
document.appendChild(element);

Want to put some data into this?

var model = { name: 'Jonas' }
var element = Serenade.view('h1 "Hello" @name').render(model);
document.appendChild(element);

As you can see you can use Serenade like a template engine. In fact you could use it together with a framework like Backbone, just to render templates. You could specify your Backbone views as the controller in Serenade, and bind events from the view. Cool stuff. Serenade is extremely flexible in how you can use it, and that's because it has a single, simple, point of entry.

Small

Serenade is just 9k gzipped, so it doesn't require a huge download. By comparison, Ember is 37k gzipped. That's more than four times the size! Plus it depends on jQuery, which is another 31k.

But aside from being small in file size, Serenade is also small in concept. It tries to solve one single problem, abstracting away DOM manipulation, and very little else. The model layer abstractions it provides are just helpers to simplify the communication between models and views.

The one other feature Serenade provides is caching, which I've found to be incredibly hard to implement and maintain, and really wanted the framework to take of for me. Transparently and automatically caching stuff in local storage is a huge win, especially for mobile applications.

JavaScript-y

Some of the comments on HackerNews complained about the use of CoffeeScript and especially its classes. I think that's kind of unfair. Serenade is written in CoffeeScript and internally makes a lot of use of CoffeeScript classes. But you're not forced to use them in your application at all.

Let's first clear up that CoffeeScript classes are nothing else than fancy syntax for JavaScript's prototypal inheritance, and the fact that they are called classes, is just calling them what prototypal inheritance is used for in 99% of cases.

As I mentioned before, you can use any JavaScript object with Serenade. The first few examples in the README show how to extend regular JavaScript objects with Serenade.Properties, so that you can get dynamic view bindings without using any constructors. This was very important to me. I didn't want to force people to use constructors if they didn't want to. I didn't want to impose how to create objects at all. Using Serenade.Model is definitely the path of least resistance, but you don't have to use it if you don't want to.

I think that Serenade is a lot more conservative in pushing you towards a class-based way of writing your apps than any framework currently out there. Maybe this doesn't come across very well in the documentation, I'll happily accept suggestions for improvements.

Testability

Unit tests are still unpopular in JavaScript land, but for those that do love them, Serenade makes it incredibly easy to test your application logic. Since templates are logic-less, unit testing them is unnecessary. Instead you can focus on testing controllers and models. Since neither of them interact with the DOM at all, you don't need the DOM to test them, which means you can test them completely in isolation. That's a huge advantage.

Conclusion

Serenade is not radically different from other options out there, I am well aware of that. But the devil is in the details. I hope that people who have had experience writing Backbone or Spine apps or have used other frameworks, will appreciate how simple Serenade is, and yet how it solves a lot of problems which are incredibly annoying and difficult and tedious to solve otherwise. It's not a revolution, it's an evolution.

Nov

Solving Cucumber's Problems


Cucumber took the Rails community by storm a couple of years ago. For the first time, we had an easy way of excercising the full stack of our applications. Many people didn't even realize that behind the scenes there was another library, Webrat, doing all the hard work. Cucumber became the de-facto way of writing end-to-end tests in Ruby.

When I wrote Capybara, it was mostly to improve the experience of writing Cucumber features. Over time however, with the arrival on the scene of Steak and similar approaches, people realized that the Capybara API was quite efficient at driving acceptance tests on its own and began using it with just plain RSpec or another testing framework.

And all was well, case closed, right?

Over the last year, I kind of abandoned cucumber, to write acceptance tests in plain Ruby, Capybara and RSpec. Throughout this experience, I have tried to keep an open mind. My verdict is: I don't buy it. In the beginning, it was fantastic, the overhead of Cucumber was gone, we were insanely productive. But over time, cracks appeared. As the projects grew larger, the tests became more and more difficult to maintain. I have since tried to figure out why this is.

Imagine the scenario of creating a task with a particular title, in Capybara, this might look something like this:

visit(new_tasks_path)
fill_in('Title', :with => 'Buy milk')
click_button('Create')

Simple enough. Imagine that we do this a lot, now we want to abstract this. Also quite convenient:

create_task(:title => 'Buy milk')

That looks good. Now imagine that this task is attached to a milestone:

milestone = create_milestone('name' => '1.0')
create_task(:title => 'Buy milk', :milestone => '1.0')
create_task(:title => 'Drink milk', :milestone => '1.0')

That's quite okay too, but what if this is a common pattern, a milestone with multiple tasks?

create_milestone('name' => '1.0', :tasks => ['Buy milk', 'Drink milk'])

That looks fantastic!

Here's the problem though: no one ever builds this abstraction. There is so much overhead involved in implementing the create_milestone method, that in practice, it's simply not done. It's certainly not done for the first acceptance test that could have used it. And herein lies the whole crux of the problem: the default behaviour for acceptance tests in Ruby is to be unnecessarily verbose, and you have to constantly fight this behaviour in order to write maintainable tests.

It is in abstracting these kinds of common patterns that Cucumber shines. In fact, this abstraction is probably still at too high a level for Cukes. If cukes are written like this:

Given there is a milestone called "1.0"
And there is a task called "Buy milk" for the milestone "1.0"
And there is a task called "Drink milk" for the milestone "1.0"
When I visit the homepage
And I click "Milestones"
And I click "1.0"
Then I should see "Buy milk"
And I should see "Drink milk"

Then you are not gaining any benefit from cucumber at all. You really want something like this:

Given I am looking at a milestone with the tasks "Buy milk" and "Drink milk"
Then I should see "Buy milk" and "Drink milk"

In my experience, it's very difficult to write tests at this level of abstraction with Ruby and a lot easier to write them with Gherkin, the language that cucumber features are written in.

Problems

Still, going back to Cucumber after being in Ruby land for a while, I encountered a number of problems. These are the same problems that are mentioned by many abandoning cucumber for plain Ruby.

  1. Having a separate test framework is annoying
  2. Mapping steps to regexps is hard
  3. Cucumber has a huge, messy codebase
  4. Steps are always global

I have written a new library, which I believe solves these problems.

Turnip

Turnip parses Gherkin feature files and runs them in RSpec. You run your feature files the exact same way you would run a normal spec file, and they are automatically run when you run your RSpec suite. So to run a feature file with Turnip, you would do something like:

$ rspec spec/acceptance/view_milestone.feature

Steps are implemented with strings instead of regexps, like this:

step "there is a task called :name" do |title|
  Task.create(:title => title)
end

It still allows for some variation in natural language by allowing a pseudo syntax for optional letters or alternative words:

step "there is/are :count monster(s)" do |count|
end

Just like Markdown, we're aiming for something which follows the natural conventions of writing text, instead of using the more arcane regexp syntax. The idea is to cover the 90% use case very well, instead of allowing every possible variation.

Turnip was written just to solve this particular, rather simple problem. There is no support for other programming languages, no wire protocol, it doesn't have its own runner or formatters or anything. Its only dependencies are rspec and gherkin.

In Turnip, steps can be local by scoping them to tags:

steps_for :interface do
  step "I do it" do
    click_link('Do it')
  end
end

steps_for :database do
  step "I do it" do
    Do.it!
  end
end

Now just tag the scenarios with the @interface and @database tags and you have different behaviour for the same step in different scenarios.

@interface
Scenario: do it through the interface
  When I do it

@database
Scenario: do it through the database
  When I do it

Conclusion

I don't know if Turnip solves the problems that Cucumber has. I don't know if Cucumber is the right solution for you. I do believe that Cucumber has a lot of benefits which the hivemind of this community has too easily dismissed this past year or so. I have tried to separate the ideas of Cucumber from its implementation. Try it out and see if you like the result!