10 Lessons from Decade with Erlang

10 Lessons from Decade with Erlang

The year was 2008. I had a steady job as a .NET developer. Then I read an ad from a company that was looking for developers with knowledge of Erlang… or functional programming in general and I applied.

I had learned a bit of Haskell in college and I loved it but I was not even remotely close to having experience in functional programming . Nevertheless, something told me that it was the right path.

Totally unprepared but ready to just improvise and see , I arrived at Novamens for an interview. I met Juanjo there but, more importantly, I met Erlang!

And that moment changed my life .

Erlang was celebrating just 10 years of open source back in 2008 :’)

Higher-order Constructs

Erlang as a language is pretty simple, with few types, few keywords and a set of very basic operations. Those are the building blocks of huge systems and you should totally learn and understand them well.

But you should also learn to build abstractions on top of that. Think of things like higher-order functions, list comprehensions, OTP behaviors, libraries like sumo_rest and others. They all encapsulate shared knowledge that makes your life as a developer easier by removing the repetitive parts and letting you focus only on the specific stuff you need just for your system.

Things like message passing with ! and receive , recursion over lists, parsing xml manually, etc. should be scarcely used in large systems. You should instead use the proper libraries, frameworks o syntax (e.g. OTP, list comprehensions, xmerl, etc.). And if you find yourself writing similar things over and over again, you should consider abstracting that generic pieces to a library.

Use higher-order constructs (libraries, frameworks, tools) instead of building everything from scratch. If there is no higher-order construct yet, build one.

Find more about this on this in this article I wrote atErlang Solutions blog.

Opaque Data Structures

Software development ( as Hernán describes it in Spanish ) can be seen the process of building computable models of reality, particularly in the early stages of development when you’re designing your system.

Those models include representations of the entities that exist in the real world. In OOP one would use objects for that. In other functional languages, like Haskell, we would use types. But Erlang has a pretty narrow set of types and, in principle, you are not allowed to define your own ones.

So, what to do? You have to combine those types to represent your entities (for instance using tagged tuples, records, etc.). But that gets messy pretty quickly.

That’s why I recommend using Opaque Data Structures, instead. ODSs are modules with opaque exported types and all the logic needed to manage them. They expose a functional interface for others to consume without worrying about the internal representation of the types.

Use Opaque Data Structures to represent your entities.

Learn more on this topic in these two talks I gave (one at EFLBA2017 and the other at CodeBEAM SF 2018)…

Test Driven Development

This is not particular to Erlang, TDD is a great methodology to create software in general. I will not go over its virtues here, but I will say that Erlang makes working with TDD very very easy.

For instance, here is an example of an assignment I provide my students when teaching them recursion:

-<strong>module</strong> my_lists.

-<strong>export</strong> [test/0].

test() ->
  [] = my_lists:run_length([]),
  [{a,1}] = my_lists:run_length([a]),
  [{a,1}, {b,1}] = my_lists:run_length([a, b]),
  [{a,2}] = my_lists:run_length([a, a]),
  [{a,2}, {b,1}, {a,1}] = my_lists:run_length([a, a, b, a]),
  …
  ok.

That module compiles (thanks to its dynamic nature, Erlang compiler won’t blame me for not having a run_length/1 function defined there) but when you try to run it…

1> c(my_lists).
{ok,my_lists}
2> my_lists:test().
** exception error: undefined function my_lists:run_length/1
     in function  my_lists:test/0 (my_lists.erl, line 4)
3>

There you go, in pure TDD fashion, I’m prompted to define run_length/1 now.

See how easy it is? And for more complex systems you have tools like Common Test that work exactly like that test/1 function above , using pattern-matching to determine if a test passes or fails.

In my mind, there is no excuse not to work this way when building systems in Erlang.

Develop your systems incrementally using Test Driven Development .

Meta-Testing

Meta-Testing, as theInakos d̶e̶f̶i̶n̶e̶d̶ borrowed from Hernán , is the practice of writing tests to validate particular properties of your code instead of its behavior . In other words, the idea is to check your code with tools like dialyzer, xref, elvis, etc. as part of your tests or continuous integration processes .

If you start using dialyzer, xref or elvis once your project is mature… you’ll have to spend a lot of time trying to detangle the cryptic meaning of dialyzer warnings. And don’t forget…

Dialyzer is never wrong. A warning emitted by dialyzer means there is a bug somewhere.

Dialyzer may not warn you about some problems, but if it emits a warning (confusing as it may be) that means you have an issue , somewhere. Maybe it’s not where the warning is reported, but you do have something to fix.

Now, deciphering what dialyzer found when you run it for the first time on a codebase with tens of thousands of lines of code can be challenging. But, if you run dialyzer on your code from the very first day, and you keep your code warning-free, whenever you get a warning it can only be something you just changed and that’s far easier to debug.

Use dialyzer , xref, and elvis in your projects constantly and consistently .

Start using those tools as soon as you start developing your system.

Katana Test (as explained in the link above) will make that extremely easy for you if you use common test. You just need to add a suite like the one below and that’s it. In fact, this one is usually the first suite I add to all my projects.

-<strong>module</strong>(your_meta_SUITE).

-include_lib("mixer/include/mixer.hrl").
-mixin([ktn_meta_SUITE]).

-<strong>export</strong>([init_per_suite/1, end_per_suite/1]).

init_per_suite(Config) -> [{application, your_app} | Config].
end_per_suite(_) -> ok.

Test Speed

Large systems tend to have even larger batteries of tests. As good practices go, you generally run all those tests at least once for every pull request and/or before every deploy.

That’s all good, but if kept unattended, those large batteries of tests will start making your everyday development cycle longer a longer. The goal of test completeness (i.e. covering as much functionality of your system as possible with tests) should be balanced with test speed. And that is not an easy thing to do.

Keep your tests running smoothly and fast .

In this article you’ll find some useful techniques to achieve that balance.

Behaviors

Behaviors live at the core of OTP, yet time and again people struggle with them. If you come from OOP land, probably somebody already told you that behaviors are like interfaces .

While that’s generally true, it hides a lot of complexity and it sometimes leads to some false beliefs that will needlessly complicate your code.

Invest time in understanding how the behaviors you use work and how to define and use your own ones .

Behaviors are great, they’re very powerful and yet extremely simple. You can learn more about them and unlock their whole potential with these articles I wrote a while back: Erlang Behaviors… and how to behave around them .

Tools

Erlang/OTP comes with many useful but somewhat hidden gems that will help you in your everyday life as an Erlang developer and boost your productivity. You should learn how to use them.

From .erlang and user_default to dbg , xref , and observer . Have you checked the sys module? What about erlang:system_info/1 or et ? There are many hidden gems that can make your life easier.

Learn about all the tools that Erlang/OTP already provides to work better and avoid reinventing the wheel .

You can find 10 of these things in this article I wrote forPluralsight.

No Debugging

Coming from the OOP world, one of the first things I tried to use when I started working with Erlang was the debugger . Turns out, that’s not a great idea when you’re dealing with a concurrent and distributed programming language.

Don’t get me wrong, the debugger works really well and it’s very powerful, but it will mostly just help you debugging sequential code. Debugging big systems and applications with it is… cumbersome at best.

Do not debug , inspect and trace instead.

On the other hand, Erlang/OTP comes with tons of tools to make tracing and inspecting systems easier. You can find several of them in this article byDimitris Zorbas. And on top of the ones provided by Erlang/OTP itself, you have amazing libraries like redbug and recon .

Engage with the Community

This is easily the most common advice I ever gave to anybody who asks me about Erlang: Engage with the Community .

The community is not huge (we know each other by first name, asMonika Coles pointed out not so long ago), but it’s proactive and always helpful.

When you start working in Erlang it’s not uncommon to feel lost, many things are new and others are just unique for someone who never used the language before. Since the main benefits of Erlang are experienced when you build large systems, the initial steps can be challenging. Everyone in the community knows that (we’ve all been through those things) and we’re here to help.

Join the community and don’t hesitate to ask for help .

Join the mailing lists . Join the Erlang Slack . Find uson IRC. Find a meetup near you .

Have fun!

The most important lesson of all: The fact that you might end up building massive high-reliable backend systems with Erlang (and those seem really serious to me), doesn’t mean you can’t just have fun with it!

Erlang, being designed with fault-tolerance and concurrency in mind from day 0, allows you to worry less on those things and more on the interesting aspects of what you’re building. That’s The Zen of Erlang .

Enjoy your time working with this amazing language !

Besides that, if you just want to play with Erlang, you can test your knowledge of the language with BeamOlympics or challenge your friends to a game of Serpents …

Or… you can build a team and win some awesome prizes at SpawnFest , too!

Thank you!

Finally, I want to use this chance to thank everybody who joined me, pushed me and guide me through this path. In no particular order…

• Cony and Tato (my wife and kid), Rosa (my mom, a VB6 programmer ), Andrés (my brother) and the rest of my family.
• J uanjo, Pachi, Esteban and Guille from Novamens.
• Chad, Martina,Marcelo, Stephanie , iraunkortasuna , Marcos , Juan , Harenson , Carlos , Hernán , unbalancedparentheses ,Felipe and so many many otherInakos.
• Monika , Michal ,Robert, Francesco,Csaba, Magdalena , Claudio ,Nicolas and the ones I’m forgetting to mention here fromErlang Solutions.
• Kevin, Pablo, Mike,Miriam and all the amazing folks fromAdRoll.
• Mariano,Manuel, Fred,Kenji,Joe, Mark,José, the otherJoe :sunglasses:,Frank, and all the other friends from the Erlang community around the world.
• All the followers of this blog and all the people that read whatever I write wherever I write it — You rock, folks! :)

:memo:

Do you want to contribute to Erlang-Battleground ? We’re accepting writers! Sign up for a Medium account and get in touch with me (Brujo Benavides) or join the Inaka Community .

:coffee:️

As usual, you can buy me a coffee .

You are also invited to join the Inaka Community .