Elixir for Humans Who Know Python

This article has nothing to do with Joyyo, a service that lets you video chat with visitors to your website so you can make more sales and get better feedback. Yes, Joyyo is written in Elixir.

Hi there! My name is Rich. Some of you might already know me from some of my Python projects and talks I've given at Python and server-less conferences.

Recently, I've been growing a bit unhappy with the direction that Python has taken, so I've been trying out other languages, looking for something which expands upon the things I love about Python but jettisons the things I don't like about it.

I've spent the past year or so working in a language called Elixir, building games, web services and AI projects, and I've been enjoying it very much.

I wrote this article to share some of what I've learned so far so that other Python developers who want to give Elixir a try can hit the ground running. I'll start with a high-level overview about the benefits and capabilities of Elixir and then dive into some practical examples, common patterns, and how to overcome snags and oddities that you might find along the way. It assumes a little bit of knowledge of Python, but even if you're not familiar with Python, I think you'll still find it useful.

This article is quite long, so if you want to skip my opinions and get right to Elixir, click here to skip to that.

Why I'm Excited About Elixir

Just to get it out of the way, Elixir is not "Python++." Linguistically, Elixir is more of a descendant of Ruby than it is a descendant of Python. It's not white-space significant, there's often more than one right way to solve a problem, and control logic requires the use of a do keyword, which I've always found a turn-off, but it's not difficult to get used to. The reasons I like Elixir are not because it's a Python 2.7 taken in a different direction - I like it because Elixir enables me to build with new capabilities at a speed that I don't think would be possible with Python in its current state.

The main reason to be excited about Elixir isn't actually the language itself, but rather a web development framework, Phoenix. Phoenix is a batteries-included web framework, similar to Django. It isn't quite as featureful as Django - there are no Users or Admin - but it gives you 80% of what you'll need out of the box, and provides some new things that you never knew you needed before, but once you try them, you'll never want to look back.

The most significant of those benefits is LiveView. I think LiveView-like functionality (known generally as "Live Apps") will be the basis for a "fourth age" of web-development, and I'm very excited about it.

A Brief Detour Into The Different Eras of Web Development

The first age of web development was static files served through a webserver. It was new, exciting, chaotic and fun, but very primitive. Most of you are probably too young to remember this era by now.

The second age of web development was in languages like Perl, PHP, Ruby and Python, where a client sends HTTP requests to URLs, and the server returns dynamic content rendered as a static resource. This was a very fun era to be a developer, but applications were limited in how complex and featureful they could be, and there were a fair inefficiencies from re-sending a whole page just to reflect a single updated value.

In next phase of web development, browsers rendered applications using a client-side JavaScript framework like React, Angular or Vue, which then make HTTP requests through a separate API, which are then rendered dynamically inside of the JS framework. This enabled more complex applications to emerge, starting with GMail, but exploded the amount of work required, as now applications required a hefty about of development in JavaScript on the front-end in addition to the the back-end work maintaining an API.

Live Apps

Live Apps combine the best of two worlds, enabling developers to build complex, dynamic applications without the need for any JavaScript or a dedicated API. A client requests an initial page load, which the server renders and maintains a local understanding of, and then makes a follow-up request to establish a WebSocket connection. Client side interactions with the application are then passed through the WebSocket and changes to the state are returned and the DOM is dynamically updated accordingly, without any API interaction or custom JavaScript required, as all of the logic is kept on the server-side.

This is where the capabilities of Elixir over Python become really apparent. Concurrency in Python has been notoriously tricky since the early days, and more recent advancements like await/async still seem hacky and cumbersome. In Elixir, however, concurrency is at the heart of the system.

Spawning and maintaining a new process is essentially cost-free, meaning that even a small server can house millions of concurrent processes. This is because Elixir runs on top of the Erlang BEAM VM, which has been used for telephony applications for decades, illustrated most wonderfully in Erlang, The Movie, and has famously powered WhatsApp and Facebook Chat.

What this means in practice is that Phoenix LiveView applications maintain a separate process for every website visitor. This would be a ridiculous, mind-melting prospect in Python, but it's a non-issue in Elixir - that's simply how things work. It doesn't require any configuration or plumbing to get going, it just works like that right out of the box, and it's awesome. You can even add new servers in a different region at any time, and the application distributes the load accordingly.

Writing Elixir applications feels a bit more like writing a "system" than it does an application the way you're probably used to. Tasks which would normally require complexity like messaging queues and external caches become just another part of your application. As a small example of what I mean, I wrote an application which has different game lobbies. In the last era of web development, you might have an API endpoint api/games/state?=waiting which clients would poll, and the server would return them. If 10,000 people are trying to find a game at once, that would mean 10,000 database queries every few seconds to fetch accurate data. Instead, it was trivial to write a process which would make that query, and then propagate the result to the clients via their LiveView subscriptions, requiring only a single database request rather than 10,000.

The Elixir Language

Now that I've hopefully enticed you into wanting to learn a bit more, let's get into the nitty gritty of the language.

Data Types

Data types in Elixir are mostly what you'd expect - integers, floats, booleans, lists, tuples, strings (UTF-8 encoded, thank the lord), and anonymous functions, which all behave like you'd expect. There are no "dictionaries", but there are "maps" which behave much the same way.

There's also one important type you might not have encountered before: the atom. Atoms are like special string constants whose values are their own names. You'll see them as words prefixed with a :, like :apple. Under the hood, false, true and nil are technically atoms, but as a convenience you don't need the : just for those. Atoms aren't scary, you'll see them all over the place as they're used extensively for the pattern matching features we'll explore later, and you'll grow to love them very quickly.

Functionalism

Elixir is a "functional" language. Though this might conjure scary thoughts of bearded wizards casting spells full of parentheses, I think the schism between functional and object-oriented programming has been wildly over-blown, and if you're comfortable lambdas in Python, you shouldn't have any issues at all with Elixir. One hangup you might encounter is that all data is immutable. In practice, this mostly means that you'll writing new_thing = Thing.do(thing) rather than thing.do(). This actually ends up being a relief, as in Python it's often unclear if a function will modify an object or return a modified version, leaving the original in tact. Life is simpler when this distinction doesn't exist.

What this allows for is some wonderful syntactic sugar that Elixir calls pipes, just like in UNIX shells. The pipe operator, |>, passes the output of a function to the first argument of the next function. So, instead of writing something like:

You have the option to write:

The readability benefit is clear - you don't have to read the code "backwards" to understand what it really does.

You aren't forced to write code this way, but it becomes very convenient once you get used to it, and you'll see it used almost everywhere in code that you'll read, so it's good to be familiar with this style. It also pairs very nicely with IO.inspect(), which you'll want to use for print-debugging, as it returns its input, so it can be used inside of a long pipe, like so:

Parentheses are also optional for functions with single arguments, so this can also be written like:

Although I actually don't like this style, as it can make it unclear what's a function and what's a variable.

Overall, I think pipes are excellent, and you'll soon miss them when you have to program in languages that don't have them.

A Note on Data Immutability

I'd like to briefly clarify what I said about "all data being immutable" with a quick example. Consider this Python code:

Notice that when we change the value of x, the output of of the print function will change as well. Now let's see the same example in Elixir:

The behavior is different - the output doesn't change. This is because when the print function was defined, x was pointing to 10 in memory. Making x point to another value in memory later doesn't change the value that was held when the print function was defined.

In practice, it's not a problem you encounter all too often, but it is something to be mindful of if you are seeing some behavior you didn't expect.

Control Flow

One of the trickier differences between Python and Elixir is control flow. For starters, there is no return statement, so you can't break out of your function or loop early, which will seem very counter-intuitive initially. Functions always return the result of their last statement as their result. The effect of this is that you will end up writing a larger number of smaller functions for reasons which will be come more clear in the next section. There's also no while statement.

Similarly, since everything is a function that returns its last statement, that means that control statements like if do as well. So, it's possible to write something like:

By convention, function names ending with a ? mark return a boolean. Functions ended with a ! raise an exception upon failure. Strings are interpolated with #{}. I suppose that was a silly example, but you get the idea.

Because of the data immutability I mentioned earlier, writing loops is a bit different. You can't do classic C-style for loops with i++ since i is immutable. Python programmers should be familiar with generators, which work the same way in Elixir:

However, many times when you're reaching for a loop, it's because you want to iterate over some data and update it. In that case, you'll want to reach for the Enum.map and Enum.reduce functions.

Enum.map applies a function to every value in a data structure and Enum.reduce produces an accumulated value for a function applied to every value in a data structure.

This might seem intimidating at first, but it becomes intuitive and convenient very quickly. There are also other useful functions of a similar nature in the Enum and Map modules in the standard library.

Pattern Matching

For cases which are more complex than a binary if statement, you can use the case statement. This brings us to the next major Elixir concept: pattern matching. This concept goes deep into the heart of Elixir - so much so that = isn't technically an assignment operator, but rather a match operator. If that doesn't make sense yet, that's okay, hopefully it will become clear by example. So, let's look at the case operator:

This example shows that we can create highly readable control flow without the need for messy if-and statements. The _ or anything prefixed with _ represents an unused variable, in this case the condition that nothing else matches.

The idea of pattern matching extends further than just control flow and applies to function definitions as well. For instance, this example from the Elixir documentation:

This shows a few things to take note of. The function executed actually matches on the contents of the arguments passed in. When those things are variables, it can match on those things conditionally. When those variables don't match, the function is never executed at all.

You'll this pattern a lot when creating your LiveView applications, like when handling different types of info events from another process:

ETS

Another very cool thing I feel compelled to mention is ETS - Erlang Term Storage, which will let your store and retrieve in-memory data from anywhere in your application. It's like having a built in Redis server! It's great for caching data or for anything you'd need to share between your processes.

Simple! ETS is incredibly handy and I use it all the time, particularly for tracking process IDs and dynamic configuration. I use it as an in-memory store, but you can easily configure it to use disk storage instead.

GenServer

One common use of ETS is to keep track of the process IDs of any processes you might be spawning. I mentioned how vital these processes are in Elixir, so let's see an example.

Most of the time, you'll be using GenServer, or generic server processes. These are a special type of process which implement standard functions you'll need for a client/server model. Here, we see a simple Key/Value store.

Then, to start it with an empty Map:

And to interact with it:

If we store that PID in ETS, then we can retrieve it from anywhere else in the system and use the process at any time from anywhere. Cool! If we're running it inside of Phoenix, we can also start it at runtime by adding {KeyValue, %{}}, to our application.ex, which defines all of the processes which start when the application launches.

Phoenix LiveView

Okay, now you've gotten a sense of the language, let's take a look at what you're really here for - Phoenix LiveView. If you just want a traditional web application, you don't have to use Phoenix with LiveView if you don't want to, but if you're reading this article then you probably do want to use LiveView, so we're going to dive straight in to it.

I'll skip most of the the installing-Elixir stuff (just use your system's package manager), but I'll quickly say that Elixir ships with its own build tool called mix, which will act like pip and django-admin for our purposes. Package management seems a bit more sane than with pip, everything just gets installed to a local deps directory, so there's no need to deal with virtual environments. All the packages live on a service called Hex, which is like PyPI.

As we mentioned before, a LiveView application maintains a lightweight WebSocket connection to each client, every interaction with the page goes through a websocket, which automatically returns with a diff that is automatically applied to the client's page without a page reload.

We're going to build a very simple lightswitch app, so let's start a new project by running mix phx.new light_switch.

A Simple Example

In the file lib/light_switch/live/light_live.ex, put:

and in lib/light_switch/live/light_live.heex (yes, templates can live in the same directory), put:

When we run the server and open our browser, we'll see a some buttons that will allow us to toggle the status of the light on and off. But what's going on here?

The first thing you'll see is the mount function. This is where the page is first loaded. In a LiveView, this function is actually executed twice - once when the basic HTTP page is loaded, and the once again when the WebSocket connection is initialized. (If you need to do different behaviors for each phase, use connected?(socket) to see if the WebSocket connection is established or not.)

The state of our interface is handled by assigning values to the socket, called "assigns". These values can be assigned at any time, so whenever we send a :noreply back with an updated socket, that change will be reflected in the interface.

Next we see some handle_event functions. These are invoked from the client, in this case from phx-click events on buttons, but they can come from lots of other sources as well and are handled in the same way.

Templates

Just like the Django Template Language, Phoenix has its own template language. In fact, it's got a few: Eex, Leex and Heex. This is honestly kind of annoying, as Heex is pretty new and some older tutorials you might find will only use Leex, which has a slightly different syntax. Heex is the future, so just use Heex everywhere and forget about the others.

At first glance, Heex seems a lot less featureful than the Django Template Language, but it's ultimately more powerful since it can execute arbitrary Elixir but the Django Template Language can't execute arbitrary Python.

In our template, we see some ordinary HTML mixed with some Heex. Unlike Django templates, where only a subset of commands can be executed inside {{ }} tags, any Elixir code can be put inside <%= %> tags, or { } tags on DOM element properties. Variables assigned to your sockets will be accessible by prefixing an @.

So, in this example, we see the light state and two buttons, one of which will be disabled depending on the state of bulb_on. When we press the On button, phx-click="on" is executed and handle_event("on", _value, socket) is executed on the server. We update the value assigned to the socket, which then gets sent back to the client and the values on the interface are updated.

Template Inheritance and Components

In Django, it's very common to have nested templates which extend each other. I've found that this approach doesn't translate very well to Phoenix, and it's best to forget about extending templates at all. Instead, it's better to build reusable "Components" and use those where possible. So, rather than having a "DashboardBase" template that all of your sub-pages extend, it's better to build TopBar and SideBar components, and to use those where needed inside of a scaffold.

For instance, if we want to show a user in a list of users, we'd make a render_user component:

(Note that ~H is called a "sigil" - it just a way to say in our code that the following string is Heex.)

We'd use in a template like this:

Not only does this make it easier to re-use code all over your application, it's actually more performant for very long lists.

Ecto

One of Django's primary strengths is the Django ORM, the object relational mapper, which is used to translate calls between Python objects and items in a database table. In Elixir/Phoenix, you'll be using an ORM called Ecto to handle your models and database calls.

Now we'll be able to use the Repo - which is created at runtime to provide our interface to the database - to fetch our objects, like so: ted = Repo.get_by(User, username: "ted").

You'll also see a changeset function. Changesets provide a pipeline for validating and manipulating data before it's stored in the database. From the documentation:

Changesets define a pipeline of transformations our data needs to undergo before it will be ready for our application to use. These transformations might include type-casting, user input validation, and filtering out any extraneous parameters. Often we'll use changesets to validate user input before writing it to the database. Ecto repositories are also changeset-aware, which allows them not only to refuse invalid data, but also perform the minimal database updates possible by inspecting the changeset to know which fields have changed.

This changset is used by our create_user function, which we'd use like User.create_user(%{username: "ted", bulb_on: false}). If any of the fiends in validate_required aren't available, our user won't be created.

Migrations

One of the more annoying things about Phoenix coming from Django is the lack of django-admin makemigrations. Ecto makes you define migrations on your own. There is a mix function to make the file for you, but you have to define the changes manually. So, we'd run something like mix ecto.gen.migration add_users_table. It'll make the file for us, but we'll have to define the contents ourselves:

It's annoying but not too bad. One thing to watch out for is foreign keys, where your migration will need to define a field like add :user_owner_id, references(:users) but the object schema definition drops the _id, so it's just belongs_to :user_owner, User.

More Advanced Queries

Of course, can write more complex queries than just fetching by a value. Somewhat annoyingly, there are a two different ways of doing that - "keyword-based" and "macro-based". A keyword-based query looks like this:

and a macro-based query looks like this:

It's the same thing under the hood, I don't know why they did this. I think keyword-based queries are more common in examples, so maybe just use them.

PubSub and Presence

One extremely useful thing that Phoenix provides is an internal PubSub mechanism, which allows a process - which for a LiveView, corresponds to a user's connection - to subscribe and publish to a stream of events. So in the mount function of our light-switch viewer, we can..

Then, we can send information on this channel to anybody subscribed to it like this:

and receive information by defining a function like this:

Now if we have <%= @switch_count => in our template, the user's interface will be updated in real time whenever a a switch count message comes over the PubSub.

Phoenix also provides a special case called Presence which operates the same way, but provides some handy extra information related to user connections. This makes it perfect showing, for instance, how many users are currently browsing a page.

Other Things I Should Mention

We're getting close to wrapping up here, but here's a quick dump of things I should mention but couldn't fit anywhere else.

• Different functions can have the same name but take a different number of arguments. This is called "arity". For instance, String.split/2 has an arity of two. You get used it it.

• There is a REPL, it's called iex. It's okay.

• Strings are joined like this: "Hello " <> "World"

• Lists are joined like this: [1,2,3] ++ [4,5,6]

• and are subtracted like this: [1,2,3,4,5,6] -- [2,4,6]

• Rather than import pdb; pdb.set_trace(), there's require IEx; IEx.pry(), but you'll have to make sure that you call your server with iex -S mix phx.server rather than just mix phx.server.

• If you use IceCream for debugging, there's an IceCream for Elixir. It's also called IceCream.

• Functions can have default arguments with \\, like so: def funky(options \\ []). However, this can be annoying if you have multiple keyword arguments, so you can this pattern instead:

In Conclusion

Okay, well there you have it, a brain-dump of Elixir for Python programmers!

There's plenty more I'd like to discuss, so if there's interest, I'll do a follow-up post covering things like releasing, deploying, clustering, and scaling, as well anything else I've forgotten that I should have included here.

If you want to see a LiveView application in action, try Joyyo!

A Joyyo is a little snippet of JavaScript that you put on your website that lets you video chat with your visitors. In your Joyyo dashboard, you'll see a Live interface of all of your website's visitors in real-time, and you'll be able to filter and sort them all without reloading the page. When you see a high-value sales opportunity, press the button to start a video call with that visitor and start selling!

It's totally free to try for 30 days, and it's only one-click to unsubscribe if it doesn't pay for itself.

Cheers!