Primary maintainer of React. @floyodphone
This two-part blog post tells the story of my venturing into Clojure. To get a better grasp of the language, I wanted to move beyond solving programming puzzles and build something tangible in the browser. Omingard is a Solitaire-like HTML5 card game built with Om, a ClojureScript interface to Facebook's React.
In this first part, “My Way into Clojure”, I'll provide some background on why I built Omingard and introduce the concepts behind Clojure. What's so fascinating about functional programming in general, and Clojure in particular, and why was the appearance of Om a game changer for me?
In the upcoming second part, “Building a Card Game with Om”, we'll look at how I built Omingard. What are the rules of the game, and what role do React, ClojureScript, Om, Leiningen, Garden, and Google Closure Tools play in its implementation? We'll also take a detailed look at the concepts behind Om, and how it achieves even faster performance than React.
When working with Ruby on Rails a lot, it feels liberating to leave behind the shackles of the framework ("your code calls a library, a framework calls your code"), and do some coding with hand tools. Don't worry, I still think “Rails is cool”. Clojure is just also cool - and probably even cooler (1, 2, 3, 4, 5) - but I'm getting ahead of myself.
I assume this post will introduce most of its readers to several new concepts and solutions at once. The stack of discussed technologies is quite impressive and, from top to bottom, as follows:
I'm well aware that this is a lot to take in at once. This post is not meant to be a tutorial but is an opinionated experience report trying to introduce you to Clojure's core concepts. My aim is to broaden your horizon and whet your appetite for new, even alien technologies.
LISPers proudly call their language “alien technology”, meaning it's too good to be true. Superior technology from another planet handed to us by the gods.
But LISP, Clojure, and many of the concepts we'll be talking about are predominantly alien in the sense of foreign to most programmers.
The good thing is I've been there already and hope you let me be your tour guide on a trip through the Clojureverse :)
I've been interested in LISP and functional programming for a while but never got very much beyond browsing some books, playing with demo code, or dropping a couple
reduces in Ruby or Underscore.js.
Clojure is a vibrant modern LISP that lends itself very well to functional programming. It first entered my radar in 2012, when Rich Hickey held the keynote at Rails Conf: “Simplicity matters”. He defines simplicity and distinguishes it from its false friend easiness as follows:
“Simple” means one thing, “easy” another.
Simple is the opposite of complex. A thing is simple if it has no interleaving, if it has one purpose, one concept, one dimension, one task. Being simple does not imply one instance or one operation: it's about interleaving, not cardinality. Importantly, this means that simplicity is objective.
Easy is the opposite of hard, or difficult. A thing is easy if it's near to hand, if it's easy to get at (location), if it's near to our understanding (familiarity) or skill set or if it's within our capabilities. This means that ease is relative.
Speaking English is dead easy for me, but that doesn't mean that speaking English is intrinsically simple. I find French quite difficult. Little French children speak French all the time, [...] It's easy for them, it lies near to them.
(taken from Jonathan M. Lange's notes for “Simple made easy”)
Rails developers need to be particularly wary of mistaking easy things for simple. Rich warns that “Programmers know the benefits of everything and the tradeoffs of nothing.” (“Simplicity matters”, slide 10).
gem install devise is damn easy. But the code it installs is incredibly complex. The same applies to
rails new my_blog - do you know what you ship?
Not meaning to start another Γιγαντομαχία, but look at this:
Programmer convenience is not everything and putting it first can be dangerous.
Another thing is that you need a working Ruby environment to run the command above. Rails beginner events show time and again how tedious this can be - easy for me -vs- easy for you.
Marvin Minsky gives another nice metaphor for the difficulty of telling simple and easy things apart:
“A computer is like a violin. You can imagine a novice trying first a phonograph and then a violin. The latter, he says, sounds terrible. That is the argument we have heard from our humanists and most of our computer scientists. Computer programs are good, they say, for particular purposes, but they aren't flexible. Neither is a violin, or a typewriter, until you learn how to use it.”
Marvin Minsky, “Why Programming Is a Good Medium for Expressing Poorly-Understood and Sloppily-Formulated Ideas”, quoted after Structure and Interpretation of Computer Programs, “Preface to the First Edition”.
Some things are inherently complex - your business processes, for example. The thing we should try to avoid or rein in as much as possible is incidental complexity and Rich argues that languages like Ruby suffer from it and thus bring along unnecessary complexity:
We can make the same exact software we are making today with dramatically simpler stuff. Really radically simpler languages, tools, techniques, approaches. Radically simpler than Ruby, which seems really simple. Why aren't we?
(Rich Hickey. 2012. “Simplicity matters”. Rails Conf. http://youtu.be/rI8tNMsozo0?t=19m24s)
He presents the following comparison of complex and simple tools:
Watch his talk to learn more.
What took me by surprise was that variables made the list. What could possibly be complex about one of the fundamental building blocks of the languages I've used so far? The problem is that variables are places whose content can change any time. This is especially excruciating in concurrent programming.
In Clojure, you use
let bindings, which locally bind a value to a symbol within a scope they create. The value the symbol refers to cannot be changed - it's as if you built a language with constants instead of variables as the default. Mutable storage (variables) also exist but they are an explicit exception. To learn more about this check out “The Value of Values”.
Rich's talks on programming principles are brilliant and very inspiring. They're ripe with etymological derivations. Rich quips that all he needs to prepare his talks is a dictionary :) His concept analyses are deep and unique - reminiscent of Heidegger's etymological arguments. One major reason I chose Clojure is to better follow Rich's thinking. If people like Rich were to leave Clojure in a couple of years, I'd be happy to follow them to other languages. He's one of the most inspiring thinkers in applied programming I know.
After I'd watched the keynote, I read the first few chapters of one or two Clojure books - there were already several to choose from, which always leaves you with the nagging question which one's best for you... (here are my recommendations anyway). And despite the fascination with Rich's ideas, I soon put Clojure aside again. It was hard to find the concepts he was talking about materialise in the books I was reading. The books' authors aren't to blame, at least it's the same in many other programming languages: books presenting one language feature after the other. They are fine when you know you have to struggle through them for a course or your job, but when you mostly read them after work with high expectations, they can easily become disappointing.
On the one hand, I was looking for a more “dessert-first” approach to Clojure - give me something more exciting than trifle puzzles, something shiny I can still grasp as a beginner and that connects with Rich's talk. On the other, reading about Clojure was confusing when your background's another language and you cannot suppress asking yourself how certain features compare across languages.
“Anyone could learn LISP in one day, except that if they already knew Fortran, it would take three days.” — Marvin Minsky
Brian Marick's book Functional Programming for the Object-Oriented Programmer was helpful and is recommended. Another great recommendation came from my friend Max Weber, who runs our local Clojure User Group. He told me to get my head around Clojure's ideas first and that talks would probably help me to do that faster than books. Rich Hickey is the Socrates of Clojure - he teaches more by talking than writing :) I did both in parallel anyway, but watched more than I read :) When a new environment is alien and confusing, pressing on and immersing yourself even more into it is definitely the right thing to do.
After I'd read and heard enough to about Clojure's foundations, I decided it was time to get my hands dirty and gather some practical experiences. However, I had no idea what to build with Clojure, and hands-on tutorials I could relate to were scarce. Web Development with Clojure hadn't been written yet and the learning curve for setting up web apps and a development environment looked steep. ClojureScript - Clojure that runs in browsers - was still in its infancy.
The overarching design principle behind Clojure is a striving for simplicity. Here are Clojure's other core features:
Of these features, we'll take a closer look at LISP in the next section. In the second blog post, we'll discover why immutable persistent data structures make Om even faster than React and how they allow efficient undo functionality.
LISP is worth learning for a different reason — the profound enlightenment experience you will have when you finally get it. That experience will make you a better programmer for the rest of your days, even if you never actually use LISP itself a lot.
“How To Become A Hacker”, Eric S. Raymond
LISP is the second-oldest programming language still in use. It's been around since 1958 and is far from dead. It used to be the preferred language for AI programming. LISP is often wrongly portrayed as one language while it's actually a very heterogeneous family of languages. Most LISPs are functional, but some are object-oriented; most are dynamic, but there are also typed ones etc.
It's usually part of LISPers' education to write their own LISP implementation and new variants keep cropping up. Racket is one of the most exciting recent ones and last week Timothy Baldridge released the Clojure-inspired Pixie, written in Python.
There's no other language that has so many dialects. What connects them is their syntax.
Here's a trivial example of Clojure code:
(+ 7 (* 3 4)) ;; => 19
And here's the same in Ruby:
3 * 4 + 7
Obviously, Ruby's syntax is more familiar and easier for us since it's the notation learned in countless math lessons. LISP uses so-called prefix notation, where function names always come before their parameters - even in calculations. The Ruby example uses infix notation.
The interesting thing here is that the Clojure code is structurally equivalent to the abstract syntax tree (AST) that Ruby's interpreter has to construct from the Ruby code. Ruby's interpreter has to know about operator precedence to build the correct AST, whereas the Clojure compiler has the AST straight away and then only has to process nested parentheses. LISP's syntax is extremely simple - yet takes some small effort from the programmer to get used to (remember the violin quote above?).
Here's a function call in Clojure:
And the same in Ruby:
The only difference here really is that in Clojure, the parentheses have moved outside, wrapping the function's name and the parameters, whereas in Ruby they only wrap the parameters. If you look back at our maths example, you can now see that
(+ 1 1) has the same structure as
(my_function "Hey!"), whereas in Ruby, some functions use prefix notation (
my_function("Hey!")) while others use infix notation (
1 + 1). The Ruby interpreter has to know about a lot more special cases and is hence more complex than a LISP interpreter.
But there is more. So far we've only seen that Clojure's syntax is more uniform than Ruby - no operator precedence, no mix of infix and prefix notation, no need to build an AST. Another thing about Clojure code is that it consists purely of Clojure data structures. Everything in parentheses is a Clojure list.
(+ 1 2)
is actually a list with the members:
In a homoiconic language, metaprogramming is much easier. When your code is data, you can manipulate it just like any other data - you can call functions like
map etc. to manipulate your code. Clojure has a very powerful macro system, which gives its users super powers. In many languages, if you wanted to have
unless you'd have to wait for the language maintainers to update the compiler. In Clojure, you just write a macro. In “Beating the Averages” Paul Graham called LISP his “secret weapon”. It's recently been argued that if LISP were a weapon, it'd be a shiv ;)
“Lisp is a programmable programming language.”
— John Foderaro
If code is data, why do we still store it in text files then? Chris Granger takes the idea of code as data to its logical conclusion in “Programming inside a Database. Thoughtbot Podcast”.
My impression is that programmers most easily dismiss Clojure because of its syntax. Programmer-convenience-first thinking (“it has to be easy to use or we won't use it!”) and an unfamiliar, hard-to-read syntax don't go very well together. But this is really just judging Clojure by its cover.
When you're used to Ruby's clean syntax, Clojure may look like a step backwards. I thought the same initially, but then learned why Clojure chose LISP syntax - simplicity and macros. I have to say that, first of all, you'll get used to the brackets and prefix notation, and secondly it's become questionable to me whether Ruby's clean syntax is worth what you're losing in terms of reasoning about your language and metaprogramming elegance.
Hardcore LISPers have misgivings about Clojure because it runs on the Java Virtual Machine (JVM). Stanislav Datskovskiy wrote two posts in this vein: “Thumbs Down for Clojure” and “Of Weighty Matters, or Thumbs Still Down for Clojure”, and asserts that Clojure is “A Lisp which barfs Java stack traces”. Implementing a dynamic LISP on the JVM is no small feat and the Clojure community's aware of it. Better stack traces still rank high on the community's whish list.
Granted, Clojure's implementation is complex, but so is Ruby's. Usually I don't have to deal with my language's implementation details and Clojure's success in large projects like Room Key or Daily Mail show that we can greatly benefit from the many simplifications it brings.
What Stanislav and many other LISPers are yearning for is a comeback of LISP machines. If we wanted a cleaner Clojure implementation we should start with cleaner hardware, then write a cleaner replacement for the JVM (same cross-platform capabilities!?, same performance optimisations!?), as well as all its libraries... we'd be busy at least for another decade and required backing from several large companies. Clojure's implementation may not be ideal, but we can use it today and it gives software engineers a much cleaner and simpler language to work with.
Stanislav's criticism is legitimate, but he's holding Clojure to a very high, idealist standard - something you usually don't have the luxury to do in applied programming. Otherwise, what's keeping us all from doing more Haskell?
Many who aren't scared off by Clojure's syntax leave the room when they hear that it isn't object-oriented and they can no longer model things with objects. Aren't objects the best way to model the world!? George Lakoff wouldn't agree.
The problem with OOP is that objects are encapsulated data bundled together with methods that manipulate that data. Encapsulation is bad in two regards. Firstly, sharing methods between objects becomes unnecessarily tedious. And secondly objects' hidden state (instance variables in Ruby) make reasoning about your code extremely complex. Calling
my_object.some_method(5) can return different results when
some_method depends on the object's mutable internal state. In functional programming, functions always return the same result for the same input - they're referentially transparent. In OOP we usually also write referentially opaque methods, which means you cannot replace all method calls with the same argument with the value the function returns when it's first being called without changing the program's behaviour.
Rich Hickey, Emperor of State
While OOP will certainly remain the dominant way of doing programming for the near future, there's an increasing interest in functional programming and Clojure in particular:
Ok, I've decided it's finally time to learn Clojure. Too many smart people I know are using it. Suggestions for a good place to start?— Patrick Dubroy (@dubroy) October 18, 2013
I should go to clojure conferences even if I don't use Clojure. I see too many smart people going. #ClojureWest— James “Jimmy” Long (@jlongster) March 18, 2013
Lawrence Krubner's article “Object Oriented Programming is an expensive disaster which must end” is a good source to learn about the shortcomings of OOP. The great thing about the Clojure community is that they're not on a crusade against OOP and know that FP isn't a silver bullet either. They happily import OOP's best features into Clojure where it makes sense: Clojure's multimethods are polymorphism ported to functional programming and Stuart Sierra's Component framework uses stateful objects in Clojure. Even more on Clojure's OOP influences here: “Functional OOP, Clojure style”.
My interest in LISP and Clojure lay dormant for a while until an episode of the German podcast Geekstammtisch with Moritz Heidkamp appeared in October 2013: “Vom Tellerwäscher zum LISP-Entwickler” (“From dishwasher to LISP developer”). It is an exciting round trip through various LISPs and the ideas behind them, introduces Chicken Scheme, and reports from 2013's EuroClojure. My curiosity in Clojure and functional programming was reawakened - but the podcast only made Clojure's alternatives more appealing and I still didn't know how to get started building something meaningful. Luckily, this was to change shortly thereafter.
We've reached the end of part 1. Very glad you made it to the end :) I hope you've enjoyed the trip so far. We met Clojure, a dynamic LISP dialect that runs on the JVM. We found its syntax the most confusing at first. But with Rich Hickey distinction between simplicity and easiness in mind we saw that LISP syntax is actually strikingly simple and elegant and drives Clojure's powerful macro system.
In the next part, we'll see how immutable persistent data structures make Clojure easier to reason about and we'll explore how they power Om's features distinguishing it from React. Om is a ClojureScript library built on top of React that is even faster than React itself and has an efficient undo feature - all thanks to immutable persistent data structures.
Stay tuned, best follow me on Twitter to hear when PART 2: “Building a Card Game with Om” gets published: @wakkahari.
Feeling impatient? Then check out the slides of my talks on Om:
Thank you for the great feedback so far! Amazed this post even reached Cognitect CEO Justin Gehtland:
I'm quite enjoying reading Paul's take on his Clojure journey: http://t.co/riZZNGBP5l— jgehtland (@jgehtland) November 14, 2014
Clarification on Haskell
The bits where I'm picking on Haskell above are tongue-in-cheek. I called it "useless" and linked the word to a video where Simon Peyton Jones, one of Haskell's core contributors, himself puts Haskell into the "safe but useless" corner of a diagram. The interesting thing is that he's trying to reach safe and useful heaven from Haskell, whereas languages like Ruby etc. are coming from the "useful but unsafe" corner where C presides and try to make it safer.
As would have become clear in part 2's "The Road ahead" section, Haskell is on my list of things to learn next. Even if it may be useless - jokes aside, chances are most of us won't be writing a lot of Haskell for their professional work -, there's a lot to learn from Haskell. Maybe I'll write "My Way into Haskell" in a year or two.
I'm calling Om a JS-MVC only to put it into perspective for readers unfamiliar with the intricacies of the JS-MVC field. Om does not follow the MVC pattern.
React stands out even from our lax definition, since all it offers is the “V” in MVC. However, the concepts it introduced to just this third of MVC are so interesting and simplify view rendering so vastly, that it is making large waves. It's become popular to replace the view layer in other JS-MVC frameworks with React (Angular with React, Backbone with React) and Ember is planning to port some of React's ideas. For the purposes of building Omingard, React was more than enough - even though it's not a full-blown MVC framework.