José Valim tells us about Elixir
I saw you on Twitch, maybe, like, a week ago, a week or two ago, and I was like, hey, I recognize that person. Yeah. Exciting to see you starting to stream. Is that something you're getting into?
José:Yeah. So I actually did. I think my first venture into Twitch was in 2018 for Advent of Code. Mhmm. And I did that for a while, and it was nice.
José:But I never really I think this is probably, like, my biggest streak. And and I and I think, like, part of part of part the reason why is because I spent today a lot of time, like, reviewing people's codes. And, like, there is a company. It's a small company. We're not even 10 people.
José:But I I'm spending less time coding. Mhmm. And, like, the most time I spend coding nowadays is when I'm twitching because I can just, like, sit out. Like, I'll do here. I'll sit here and I'll code with everybody, but it's just for coding.
Dax:Yeah. It's funny because people have very different experiences. Like, some some of the people that I know that stream, they're the one that get the least amount of work done because they're so distracted by the chat and, like, by people watching. But it's pretty productive for me as well. I I it's similar, like I have so much that I do, like I'm juggling a bunch of different things.
José:So It's funny because the projects that I was working on so far, I I probably already streamed like ten days. And in the I I only did Elixir for real, like, in the eightieth or nine if day. So all the time before first, I was trying to do stuff with ChatGPT, so it was basically prompt engineering. And then it was Rust and and a little bit of c. So it was fun, like, because those are not things that I have experienced, so it was actually helpful doing that with everybody.
José:Because people who actually know Rust and actually know c, they were they were help helping out. I at some point, I was even joking that it was like the equivalent of Twitch Plays Pokemon but for programming because I was just hearing what people say, they're like, okay. I'll try this. I'll try that.
Dax:It is awesome when you have a decent sized audience and there's people in there that know stuff better than you. It's like that combined with ChadGBT, it's like a it's like a superpower. It's just you don't you're just like a body that's that other people are controlling. Yeah. Just for a little bit of context, I for the past, I guess, two to three years, I've been in the serverless space and specifically has brought me to back end JavaScript, and everything I've been doing has been in back end JavaScript.
Dax:And most people know me from that, but I'm still relatively new to it because I was spending the previous four to five years in the elixir world and as someone coming from another ecosystem into the JavaScript world, I think I just have a different perspective on like, just the type of tools I'm used to, the things that I had available before that I no longer do. So I've been trying to talk to more people that are from other ecosystems, like I had someone out from the Laravel ecosystem a few months ago, because I think the JavaScript back end world, I feel like the bar is very low in terms of what is considered like, okay, these are good tools. But I think that's mostly because most of the people in that ecosystem just haven't seen kind of what's available in others. So just getting more exposure to the other stuff that's out there, I think will help make the jobs of ecosystem better. I just feel like we just it's not that there's any technical limitation.
Dax:It's almost like a cultural thing like we don't know that we can make stuff better. I feel like we don't look outside of our our bubble as much, so yeah, I was excited to talk to you. I have a bunch of questions and we'll just kind of run through them and it'll be a casual conversation. Did I talk to you because you know you bootstrapped a new ecosystem from zero, like roughly zero. I think there's a lot of interesting insight in in hearing about that journey and and the kinds of specific details there.
Dax:Does that make sense?
José:Perfect.
Dax:Cool. Jose created a language called Elixir. When was that, like, thirteen years ago? Something like that?
José:Twelve. Almost thirteen. Yeah. Yeah. The the initial prototype, it it was, like, twelve, thirteen years ago, but the version that became Elixir today was eleven to 12.
José:It was a year later, kinda.
Dax:Nice. Yeah. So I wanna hear, like, how did that even happen? Right? Because creating a new I mean, people create new frameworks, and that's, like, a little bit more common.
Dax:You've heard of that. But that's usually sitting on top of an existing language and ecosystem. But so creating a brand new language, how did that happen? What gave you the confidence that it was viable? Because, like, you know, creating the language is a lot of work.
Dax:Like, a lot of things needs to happen before it's even adopted or gets traction. So, yeah, I would love to hear about that journey.
José:So always started. So before I worked on Elixir, I was member of the Rails framework, the Rails framework core team. And even back then, we were starting to hear a lot about concurrency. Mhmm. Like, we're starting to get, like, the first machines with multiple cores, and then people wanted to deploy those things using all the cores in the machine.
José:But it was not really feasible, And we were working and improving those things over time. And today, like to say that, like, everything that you do in your machine, it should be using all cores. Right? Like, because even your wristwatch today, it have, like, two or four cores. Right?
José:So we say a lot, oh, like, something is slow. This tool is slow. Like you know? And then you look at it. My tests are running slow.
José:And then when you look at it, it's just using one core in your machine, which so there is a chance that things can get, like, four times faster. The resources are there, but for some reason, we are not using it. So but that's so back then, that's when it was starting. And I remember there was one bug report that I still remember because it was, like I think it was very significant for me because somebody said, look. We are running rails in production.
José:And when we have a spike in traffic, so, you know, a lot of people are using at the same time, they have all the scaling. So auto scaling would start a new instance of Rails. And when that new instance was booting, it would get hours that were booting. Somewhere in Rails boot process, things were failing there are many requests at the same time. And and then when you get a bug report like this, like, your first thing is, like, how I'm even going to reproduce this.
José:Right? Because it's, like, such a specific scenario. And it's related to concurrency. It means that when when Rails is initializing, requests are coming in. There's something happening somewhere about two requests come at the same time that makes thing fail.
José:So, like, how I'm going to reproduce it? How I'm going to to tackle this? How I'm going to fix this? And even if I do all those things, like, how can I be sure that I tested and fixed it properly? So that was kind of, like, one of the things that made me think, if concurrency, so running software on all the cores in your machine as efficiently as possible, is going to become an important thing, we I I need to have better tools.
José:So it always started for me as, like, trying to find better tools for for the problems that I was facing, which I think really resonates, like, with what you were saying at the beginning of always, like, trying to look outside. And and that's how I started doing. And then so I I put myself in this journey, and I like to say that there were two points in this journey for that, like, changed how I approach software. And the first one is functional programming, and it means a lot of things to different people. But to me, what it means is that if you get a lot of the concurrency issues that you have in in o o or imperative languages, is that so for example, in that case, I would have, like, two requests or two threads trying to change the same place in memory.
José:So try to mutate an object or something like that. And then because we're trying to do at the same time, something would get corrupted. If you're lucky, you get exception. If you're unlucky, you get a segmentation fault, and the whole thing crashes. And the thing about functional programming and, again, it means different things to people.
José:But to me, it's really about instead of changing things in memory, we always transform things. So if I have a list with 10 elements, want to delete one, now I get a list with nine elements. So we have ways of making that efficient. But but for me, like, this was, like, this was, like, a big change because there are ways that we can solve problems, and sometimes you, like, you solve a problem by putting a tape on top. But if you can solve the problem by removing the problem altogether, then it sounds, like, very exciting.
José:So I was like, okay. This is great, and I want to learn more about this thing. Like, I was I was convinced. And it it would also help with a lot of like, sometimes when you're writing code, you know, in the usual languages, like JavaScript, Java, Python, c. Right?
José:Like, when you call a method, for example, you pass you pass something to that. And that thing can change the arguments in ways that you don't expect because you can always change anything. So and then I was like, well, that problem is lost removed. Everything's going to be more explicit about how data goes in, how data comes back. So for me, it was it was an easy sell.
José:And I was like, okay. I want to learn more. And then I started learning more about functional programming languages, functional runtimes. And eventually, I found the Arling Virtual Machine. So Elixir runs on the Arling Virtual Machine, and the Arling Virtual Machine was it was designed by Ericsson, which is a tech telecommunication company.
José:And what is really interesting is that we go back, like, eleven and twelve years ago. Like, everybody was talking about concurrency. It was in everybody's radar. So if you see, like, the programming languages that were coming out at the time, like, Go, Clojure, they all had a concurrency story to them. Right?
José:But Airlang does not did not only have a concurrency story because concurrency is about, well, I want to use all the resources, all the cores in my machine as efficient efficiently as possible. But if we're working on a problem, eventually, like, if that problem starts to grow, you cannot solve the problem only using a single machine. You have to use multiple machines. Right? And Airline not only did concurrency, but it also did the next step.
Dax:Mhmm.
José:It did the distribution of things and how you coordinate work across multiple machines. So and it not and it did that, like, three decades ago.
Dax:Yeah.
José:So for me, I I was I was amazed. I was like, not only it has solved the the the next problem, it, like, it was one step ahead, but it is one step ahead for, like, three decades. So I I I really fell in love with their language machine. And at the beginning, I was like, okay. I I want to learn more about their language machine, so I need a project.
José:Mhmm. I am a impatient learner. I like, if you give me a book, I read 10 pages, and I wanna do something, and then I drop the book. You know? So I was like, I need something to do.
José:And I was like, I'll try to create a programming language. I don't have a CS background, so I was like, I'm going to learn two things at once or maybe learn nothing at once. But I was like, okay. I'll create programming language. I'll learn AirLang.
José:And and after I did that with time, I felt like, hey. I think there is really an opportunity for something here, for a programming language that expose their learning virtual machine in different ways and that solves those problems that their learning virtual machine, again, has been solving for a really long time. And just one more thing related to this is that the web with time has become really, really closer to, like, telecommunication systems because in fact, we get telecommunication system, everything is connected and information is going and going back all of the time. Right? And and that's what is the web today.
José:We are connected to our phones, information comes goes. We have IoT devices. So it's a lot of data going through, and I thought, like, this this technology is going to be fantastic for that. And that's pretty much
Dax:Yeah.
José:That's pretty much the summarized version.
Dax:No. That that that makes sense. I it is funny because I haven't thought about this stuff in a little while, and it's reminding me a few things on my my own. So my my dad's actually a software engineer. Like, years ago, he mentioned Erlang to me, and he was like, you should go look at Erlang.
Dax:At the time, I didn't I didn't understand why, like, I didn't get it. But I remember him saying specifically that the functional programming model, and specifically the actor model that's in Erlang, the model that you apply to make sure that it works across multiple cores in a single machine is that same exact model works whether the two cores are on the same machine or on different machines. And at that time, like, again, they distributed systems weren't as common, and he was like, we've moved from one core to multiple cores, the inevitable next step is distributing work across multiple servers. And stuff like Erlang and functional programming are gonna be very useful in that era. At the time, I was like, okay, sounds good.
Dax:And I didn't I didn't understand any of it really. It didn't click for me, but I came back years later. It was like when I found Elixir, and I was like, remember Erlang, but I remember my dad talking about it in like a very positive way, and Elixir seems like a nice modern way to address it. And then for me, the concurrency thing you talked about is what drew me as well. I originally had been doing stuff in c sharp.
Dax:I needed like like a better concurrency model. At the time Go was kind of sitting right there, like you said Go came out with like very intentional concurrency baked in. And we built this we're working on it on like a distributed data system at that time, and there was an update to Go that made it throw it it would panic when you did concurrent map access, which it wasn't doing before. And we updated it and we realized, wow, we have a ton of bugs in our system that we didn't know. So at that point, we were like, we can keep fighting with this or we can rethink, like our like you said, we can remove the problem.
Dax:That's when I found Elixir, Erlang, and I got all clicked together. It's okay, this whole class of problems of two things accessing the same piece of data, it's entirely gone in this model and you can still do everything and actually that's just one of the many things that it simplifies. So yeah, I think it's you made a good point that functional programming means a lot of things to a lot of people and there's like a spectrum of of what that means. But for me, it's that it's you think in this way where everything's actually just simpler. It maybe takes a little while to like unlearn the way you used to think, but once your brains which is over, it's actually a very simple model.
Dax:Everything's very predictable, but behaves in a way that you expect. And what I loved about Elixir was unlike other functional programming languages, I think back then people typically thought of like Haskell or like, you know, these more esoteric things. It was such a nice introduction to thinking this way because you need to go through the initial phase of rewiring your brain. But the step from going from whether you're a Go programmers, it looks like a programmer, very easy transition and you use it for a couple months and your brain is rewired at that point. You start to think functionally by default and that means like a lot of things and like you know, like you said you think about immutability, but also things like like recursion becomes a little like a lot more natural thing in your head.
Dax:It like came out at such a good important time, at least for myself. So the other thing I was gonna ask Do remember is
José:when it was?
Dax:So I think 2015 for me? 02/2014, fifteen that
José:is early. It was just about one o's. So
Dax:Yeah.
José:You were you were early. Yeah. Yeah.
Dax:I think I got into it right when Elixir in action got published or like
José:Yeah. Super early.
Dax:Yeah. Okay. I guess that is relatively early.
José:That's nice.
Dax:Yeah. So and and again, I remember when I brought it to my team, they were like, oh, again, you're bringing us like another new thing that we got to learn, and then there were no one was that excited at first. But and I remember a specific person who wasn't very excited, and, you know, he gave it a shot and he ended up falling in love with it. And to this day, he's now started a new company, tons of engineers work there, 100% on Elixir, and it's been Nice. Almost ten years now and has really stuck with it.
José:You you know what what is cool from what you said, like like your father said, like, you know, if we went from concurrency, we went to distribution. What is really interesting is that Airlang went the opposite way because when Airlang was created, multicore wasn't really a thing.
Dax:Right.
José:So they did a distribution. Right? And then when they had to think about multicore, they're like, oh, we got it. Yeah. This is fine.
José:Right? Like, it it just it just works. And and I think it's, like, one of the when you read and you learn more about how Airlang was created in that, like, everything they were doing was really pragmatic in the sense that they were trying to solve a problem, which is to build, like, telephone switches and these kind of applications. They did not say, like, we want functional programming. We want distribution.
José:We want, like like, actors. They were not thinking about this. They were thinking, like, what is what are going to be the best abstractions for us to solve those problems that we have at hand and and all the decisions derived from that. And for me, like, this is it's so interesting and so exciting. Right?
José:Like, to take such an a problem driven approach, like, to design a programming language.
Dax:Yeah. It's a whole bottom up versus top down thing. Like, when when you start at the bottom and, like, solutions emerge, you end up with something a lot more resilient. And, yeah, that point you made is great. It's like, you know you really know when it's it's a fantastic abstraction when a entirely new problem comes up and your model just works.
Dax:You know you've you've discovered something that that makes a lot of sense. So when you actually ended up you said it was like a project to learn. At some point, you must have realized like you said you this could be a lot bigger, it's gonna be very useful. At what point do you feel like, okay, like I can actually make a language that can be somewhat like adopted. That feel impossible like because that seems like such like an impossibility today.
Dax:So imagine starting there, it must have felt like, does it even make sense that I'm doing this?
José:So what what happened is that I built this prototype, and it was and it was really bad in in many ways, but it was a really learning opportunity. So, like, the first Elixir version, and if you go back in the Git history, it was actually object oriented because that was the background I had.
Dax:Right.
José:And one of the things that I learned with this prototype was pretty much, like, Erlang did many of the decisions correct. So I should just do what Erlang is doing and try to extend on the parts where I feel it's lacking. That was, like, the the big lesson from this prototype. And then I I worked on this prototype for, like, three or four four months, and then it it the I was not happy with the result, and I I stayed, like, the rest of the year. It started in January, then the rest of the year, I was like, I did not touch it.
José:But the idea was there in my head. I was thinking, like, about the problems, how I would solve those problems, looking at other ecosystems. And then at some point, I thought, hey. No. I think I think we can create something.
José:I think and the value was always there in the early learning virtual machine, to me. It it it was always clear. Like, the value is the early learning virtual machine. And I thought, wait. I think we have an opportunity to create a language that is going to provide certain characteristics, and and it's a gap that is not filled in today.
José:So what I remember I told, like, I I had a consultancy with some partners at the time. Mhmm. Well, I remember what I told them is that if the future is really concurrent, and we're using mostly Ruby at the time, and then we we say that and I think the future today is concurrent. And if we if we, for some reason, we we think that Ruby is not enough, I told them what is the what which language would we migrate to? And I told them it would be Clojure.
José:I think it was the language that, at the time, most resonated with our practices of, like, being productive, being expressive, and Clojure brought the concurrency story. But I said, like, you know, like, for better or for worse, it's like, I don't want to rely only on a single option. So for example, me as a Ruby developer way back in 02/2006, 02/2007, I always thought, like like, I have I have a symbolic. It's Python. Like, if, you know, if I wanna try something out, something goes wrong with this, I have a bolt to jump into and felt fight.
José:And then and then I felt like it feels like if we have only a single option, I don't think that's good. I would like to have more options just in the general sense. And I would like to have an option that is not the the JVM for whatever reason. And and and I think we can try creating this option. And and that's when they said, okay.
José:You can work on Elixir part time. So at that point, I was working on Elixir part time. But I knew and I told them, like, look. I can work on this for, like, two years and nothing may come out of it. It will it will be like, we have to close down the project.
José:There is no, like, adoption, and it's basically two years of my time and opportunity cost that is going to go to go down the drain. And and this led to a couple things. So one of them is, again, like, if you look at the languages at the time, Microsoft with f sharp
Dax:Mhmm.
José:Apple came with Swift later, and then Google Go. Like, if you look at the competition, it's like, well, you know, we were a third a 30 person company in Brazil. Right? And those are literally, like, the richest companies in the planet.
Dax:Right.
José:So one of the things that we thought is, like, whatever we do, it needs to be decentralized. Mhmm. Like, we are not going to try to say, we organize the events. We are the Elixir company. We always said, like, we want to for us, the value was always in growing the cake instead of having, like, we would rather have a small slice of a big cake than the the like Right.
José:Have a whole small cake. Right. So everything is going to be decentralized. You're always going to let the and that reflected in the language. It's an extensible language that people can get and extend it to new domains.
José:This stuff we can talk about later. But we had this decentralized approach, and I was always prepared psychologically. Like, this may fail. Mhmm. And this was good because it kind of, like, isolated me.
José:I didn't want to see, like I didn't want to do design decisions based on adoption or popularity. I wanted if I thought that something is right or wrong, you know, that decision needs to be isolated from from necessarily, like, the feedback that we this sort of feedback. Feedback is important, but you want to decouple from if you are failing or succeeding. People say, like, have you ever expected this? And then my answer is, I haven't, but I also, by design, I didn't put too much thought into this.
José:I never thought about where it would go into. Like, I'm extremely happy that I was able to my work was to work on an open source project and create a programming language, and that for me was enough. You know? And I would do my best to to foster the community, to to foster the ecosystem, but I would not set goals or expectations.
Dax:Right.
José:But there were moments where I where it was like, you know, you get, like, the the team was like, oh, like, you know, like, things are happening. So I think when Dave Thomas, who, you know, who's really known programmer when he said that he's going to write a book on Elixir, within a week, sent Lauren from O'Reilly, who was an editor at O'Reilly. He said he was going to write a book as well. So there were, like, two books announced within a week. So that was like, that's nice.
José:Yeah. And then we have, like, the first Elixir Conf. So those were, like, really nice, like, I think, achievements that said, like, hey. You know, like, I guess other people are is going to use it. And maybe we'll still die, but we're not going to die, like, next year, maybe in three years.
Dax:Right? Willingness to say this can turn to nothing is pretty important because especially early early on in the project, even if you get adoption, it might be like localized adoption, where if you just end up stuck focusing on that group, it might prevent you from actually becoming, like you said, making the cake bigger. So willing to like kind of kill your own ideas to like make sure that it gets to where where it needs to get to. That's something that we think about a lot and it is tough because oftentimes your early adopters are your most loyal. The version of what they want you to build is not necessarily the thing that makes it much bigger.
Dax:Right? So it's always a tricky thing to juggle.
José:Yeah. And and it's even thinking about that, just to give some ideas of the numbers, like, I started working part time on it in 02/2012.
Dax:Mhmm.
José:At some point, especially when the first books were announced, I think I became full time, and we would have our first client. The first time the company made money from Elixir was, like, 02/2016, like, four years later. Wow. And for us to kind of say the project can support itself, so, you know, it's not like we are paying back. We're just, like, not we are not, like, a a cost center.
José:Yeah. It was six years later.
Dax:Okay.
José:So, you know, it's it's really a lot like a long period of time and, like, a long term investment.
Dax:Yeah. Almost every project I've seen because I think when most people discover a framework or a language, it looks pretty, like, complete by the time they discover it. So it feels like, oh, this is a new thing I found, it must not be that old. But almost every time you like look into how long people have been working on it, it is always much much longer than you expect. Because getting the stuff right takes a lot of time.
Dax:And I'm sure there was like different flavors of what you built that you abandoned and it kind of like went a different direction. The other thing that I was curious about is, so you said at one point is object oriented, you know, that's you came from the Ruby background. Elixir today just has modules and functions, and this is something I talk a lot about in the JavaScript world and that you feel like you might need a lot more than that, but functions organized into groups of modules, that's really all you need. You can do everything you possibly need to with that. Was there a, like, a moment where that clicked for you, or, like, do you have any thoughts on that pattern?
José:Yeah. So so going back to history. Right? So when I thought that Elixir needed objects, and the reason why it wasn't object oriented, what I really wanted and I did not know at the time is that I wanted dynamic dispatching. So I wanted to stay so for example, hey.
José:I want you have some sort of contract, like, oh, I want to convert this data structure to product buffs, for example. You you need to be able to specify contracts, and you want things to to obey those contracts. In object oriented languages, especially the dynamic ones, that's a matter of calling a method. You say, look. If this thing has a true product buff method or something like that, or I think maybe, like, a generator thing in JavaScript, you know that that thing abides that contract.
José:And we did not have this in our link. And because we did not have that, I thought, hey. The solution for these are objects. Right? So I added objects.
Dax:Right.
José:And and then a lot of things, they became clunky as a result of that. And then later, I figured out, do I just want the polymorphic part of objects? And I don't need objects to have that. And that's when I learned how Clojure does things, how Gold does things, how Haskell does things. I'm like, okay.
José:I'm going to bring that idea in. I mentioned Clojure, and I think Rich Hickey, he was he helped me a lot to understand those ideas because one of the the way I think what is the talk? I'm not not sure I'm going to remember the title of the talk. Maybe people in the chat will know and they can comment. But it was really about how how you can see things as coupling and how the coupling, breaking those things apart brings you benefits.
José:So for example, if you take an object, is by definition coupling behavior and state.
Dax:States. Yeah.
José:Right? And there are a lot of a lot of things that come as consequence of that design. Like, suppose of, oh, why you need inheritance? It's because sometimes you want to extend because you have to couple the behavior of state, and it comes with its downsides. And then say, well, if you break those things apart, not only you solve problems that were created by a consequence of you putting things together, but you can now express things more naturally.
José:Right? There are a bunch of, like, design patterns that happen in object orientation exactly because of this coupling. Like, oh, what if you just want the behavior? What if you just you just want the state?
Dax:Right.
José:Right? And then it starts going, like you have, like, now those beta classes and those kind of things because you have to undo the coupling that you did. So his talks, it was have always been, like, very informative in in trying to understand things and and, you know, why certain things are advantages, where are the disadvantages. And, yeah, and objects for me, yes, I see them. Not only it's actually three things.
José:It's not only the coupling of behavior and state, but objects, they are also mutable.
Dax:Right.
José:Which means that they change over time. So you have, like, three dimensions now in this one thing. And that's why sometimes, like, trying to understand how an object has evolved over time, it gets so complex because all those things, they are all related to each other. So I remember, for example, like, sometime when you when you would be writing like, if you're working, like, on applications, sometimes you have to write a test. And in order for you to test the scenario that you need, you have to call these and these
Dax:and these and these and the object.
José:Yeah. You have to build the whole history. Right? Because all those things, they are coupled. And then you can do the assertion.
José:So yeah. So, you know, for me, it's like breaking those things apart and being able to see each of them individually brings a huge amount of clarity to the code. To the point today that you were saying, like, earlier that you you work with Elixir or a functional programming language, you rewire your your brain. And it you know, in some ways, it's kind of annoying because when you go back to use a language that, for example, has mutability everywhere, it kind of makes you paranoid. You're like, wait.
José:You call a method, and you're like, wait. Will it mutate my things or not? What else can this thing do that I actually don't know about?
Dax:Yeah.
José:Right? So you're kind of, like, living on the edge. You're like, what what can this thing be doing? And then if you break those things apart, you can reason about them. And I think for the understanding of the software at the end of the day, it is much clearer.
Dax:It's interesting because I think people ascribe I see this all the time where people ascribe certain properties or capabilities to some feature. They're like, oh, if you don't have classes, you don't have object orientation, you can't do x y and z. And I actually did a whole stream where I kept asking the chat saying, okay, name something I can't do without classes and they kept saying stuff, but you can do them because they're not properties of like that exact solution like you said, they're actually three separate properties that you can they exist on their own. And then I was able to show there's all sorts of other stuff you can do that are kind of awkward with like the opted oriented setup. At least in JavaScript, you know, you can't do like multi inheritance and how like flipping all of this.
Dax:You can like compose different things together, so that this behavior actually inherits like three other things that you find useful. When you use certain abstractions, sometimes you don't really understand what they're providing you or how. You just can't understand the end the end feature sets. But yeah, that that's that's that was another key thing that when I moved to elixir it made me writing code in other because I always had to write JavaScript for the front end. So it made all of my JavaScript code much better because Elixir kinda taught me that, like, you can do everything with just these these simple things.
José:Just one thing on that because there is a quote that I I I like a lot from a book. It's it's it's a it's a more, like, academic technical book. They they they started with a programming language. It's principles, concepts, and techniques of programming languages or something like that. And it it's talking about programming language that, like, can't do much, and it slowly add features to it, like objects, concurrency.
José:And one of the quotes in that book that I like a lot is that you should always use the simplest abstraction available
Dax:Mhmm.
José:To solve your problem. And I think it's very powerful because if you think about object, it's a very powerful abstraction exactly because it has those three things. Yeah. Right? But a lot of the times, you could be using just simpler things.
José:Right? As you're saying, just modules and functions to solve the problem and have the object to be not the the full solution that you reach into. Right? It's like you should reach into that when you need
Dax:it. Again, a lot of stuff is arbitrary. You start working on a language or you start working on a code base or certain patterns that you think are the way to do things, but it turns out you can kind of reduce it a lot. Yeah. So anyway, the the other thing I was gonna say, we start very low level, just talking about the language.
Dax:But another great thing that I love about Elixir, which I miss so much is the amazing standard library. So in the JavaScript world, the standard library is like it's not very good, I think, especially compared to other languages. It's one kind of inconsistent internally just because of the years of decisions that have changed, two just very incomplete. You have to like reach out for external libraries for basic things. And I think I always point people to is go look at the enum module in elixir, go look at the stream module in elixir.
Dax:It is such a complete thing like almost any like operation you need to do, there's probably something that's helpful there. My friend made a joke that there's probably like a enum dot sort by birthdays in there like it just like has everything you possibly could ever need. So when designing the center library, how how did you approach that? Like, do you take influence from other places? Like, how did you decide what belongs in standard library, what what to outsource into external external things?
José:That's a great question. That's a great question. Because we don't have objects. Right? So in Elixir, we have a predefined amount of data types.
José:So we have lists, we have two posts, we have maps which are like dictionaries or like JavaScript objects, but without the behavior parts, just the key value association. We have integers, we have floats, but it's predefined. It's probably about 10 or below 10. So in a way, if you just look at that, the standard library doesn't have to be large because you need to cover that surface. Let's say that's the minimal acceptable, the minimum that I have to do.
José:But we also of course, we can create our custom structs that, for example, if you want to represent a URL, you can create a custom struct that belongs to the language. So in order to I I think, like, growing the standard library was somewhat focused on solving certain problems Mhmm. And expressing certain capabilities. So for example, I mentioned lists and I mentioned maps. Both of those things, they are they are collections.
José:They can hold different things in them. And then, again, Rich Hickey, he had this whole talk about if you have collections in your programming language, instead of having, like, three or four different APIs. Right? And then streams, they are collections, but they are lazy. And we also have ranges, which is like an interval.
José:It's from one to 10. So I could define, like, four different APIs for working with each of those things in, like, in separate modules, namespaces, but which you'd say, no. You wanna have one namespace where you put everything. Right? And that namespace can work with all different kinds of collections.
José:They can be lazy. They can be eager. So I was like, okay. That's an important concept, and I want to solve that problem in the language. So we have in the terms, we have streams
Dax:Mhmm.
José:And we have some data types that are very useful to think about that problem. So it was very, like, specific to the problem. And then the other one is related to concurrency. So concurrency in the Arlene, I I know you know this, but Yeah.
Dax:For the audience.
José:Yeah. Yeah. Yeah. Concurrency in the in the Arling virtual machine and in Elixir, it's done by starting processes. And those are not the operating system process.
José:Those are cheap lightweight threads of execution that exists within their own virtual machine. You can, like, literally create million of those. So they're all running at the same time. They're all running concurrently. And those process, they define, like, some very basic primitives.
José:So for example, they can send messages between each other, it doesn't matter if those processes are in the same node or in different machines in the cluster. That's how we get distribution. But they can only send asynchronous messages to each other. One of the thing that is very common is that you wanna send a message to a process and get a reply back. So you need some sort of synchronicity abstraction.
José:So and Airline has they call them behaviors. It has behaviors for something that works like a server that is client response. Mhmm. So one of the things I was used to say at the time, right, when I was starting with Alexion, I was thinking about this that, like, we have, like, this great environment for working with concurrency, right, or even distribution. But if I want to get, like, a simple problem, like so for example, even JavaScript.
José:Like, imagine that you want to access, like, four URLs at the same time, and you don't want one to block on the other. Right? You can compose that using async await something and then right? Like, you do async async async and then you await. It's very easy.
José:But in AirLane, to do that at the time, it would require, like, a good amount of, like, boilerplate to do that. Like, may you would you would eventually encapsulate that boilerplate and put in your application and reuse it, but then it's like, everybody's, like, reinventing their own boilerplate.
Dax:Right.
José:You know? It's so in Elixir, was like, okay. I want to have, like, more specific tools for concurrence as well. So we have, like, the task module in Elixir, which people usually really enjoy using, which is about encapsulating those patterns. So it was, like, very it was really focused, I think, like, on solving certain problems and, like, closing the gaps in some areas that the other language machine was not solving at the time.
José:So for example, coming from another example, coming from Ericsson, Arlang is, let's say, Swedish technology. And, at the beginning, it would handle, like, Latin one characters. The encoding would not handle UTF eight. So it was very important. Mhmm.
José:I I like to joke, like, a Brazilian person living in Poland and communicating in English, it was really important for me to handle UTF eight encoding really, really well. And so the standard library has the string module that also handles all of those corner case for you. I'm up casing a string, lower casing, and handle those things that they may be broken in in other ecosystems. Right? Like, when you have a case a string with my with a in my name with the accent, it would not do it correct correctly, for example.
José:So it was that, and then I it was like a a good core. But Elixir is going to ask you to to learn things. Right? Elixir is a functional programming language. It's a concurrent distributed language.
José:It's a distributed concurrent programming language, distributed programming language. You can create things that are fault tolerant. So and those things, those for me, they are, like, no negotiable properties. Like, that's why the language the the language exists because those things, they come from AirLane. And because this is a learning process, even even for we love it, we may still lose people who are cannot go through this learning curve.
José:Right? They cannot for some reason, they they cannot go for the process. So I wanted to make sure that the tooling was great. Like, initial experience, like, when you create your project, everything compiles, you can run the test, everything is built in. Go was a huge influence on that.
José:I think Go was one of the programming languages that said, like, you know, how important tooling is. Tooling was not part of the programming language. It was something that you would install separately.
Dax:Right.
José:And I think Go was really important in changing that. So we're like, with the tooling needs to be great because in our learning experience, we cannot afford to introduce any, like, accidental friction. Right? Like, the frictions that we can solve, we need to solve.
Dax:Right.
José:So that was the focus on like, I was thinking about all those things and constantly send library. Today, what I like to say is that so if you say, I want to add this abstraction to Elixir, I say, there are there are two reasons for adding an abstraction to Elixir. It's because, one, is that abstraction can have it will be enhanced, and it will be able to do things if it is part of the language that it would not be possible to do if it was not part of the language. So it has to be justified to be part of the language. And it's really hard to do this because Elixir was designed to be extensible.
José:Anybody can go and can extend the language. Like, the most recent example is that you can, today, compile Elixir code to run on the GPU. Mhmm. And that's a project that does not like, it doesn't change the language in any way. It's built on top of the existing abstractions.
José:So that's one of the thing. Or it's something or it's an abstraction that we need to build the language itself. So so today, like, that's kind of the the guidelines. So we have a very clear process for somebody to say, hey. I want to have, like, this obstruction in the language.
José:Can I? If you you you have to prefer at least one of of those those roles, and then the the core is focused today. We're not trying to. So for example, if you want, like, convert encoder decode something to JSON, it's not part of this on the library. But we have really great tooling that makes it easy to add that to your project in one line of code.
José:Right? But it's not a concern that we want to have within the library. We want to let the ecosystem solve those problems.
Dax:That's a good tricky line. Right? So this is a this is a frustration that I have in the JavaScript world where I mean, the thing you said about tooling shouldn't be something you add on top because that's like unnecessary friction. I think it's a great way to summarize that. Some of the justification I see for that though is people argue that letting the ecosystem solve those things potentially results in better solutions.
Dax:I think that's good in theory. I think in practice, there are certain base things like package management, like things like that that when you don't have a really great solution that's built in, it ends up being a fragmented annoying experience. I think Go kind of went through that as well, right, because they didn't have like an opinion in a way to do package management. Then they eventually did add that in and, you know, that there was like five or six that kind of emerged and they kind took the best ideas and added it in. And additionally, like, when some things aren't in the standard library, and there's like multiple options, so let's say the thing you talked about like all the operations you do on collections, right?
Dax:The equivalent of that exists in JavaScript as well, but it's spread among like a bunch of different libraries. Some libraries will be, you know, like more functional, some libraries will be more like class oriented, so that the fragmentation is is a little painful. But people do argue for the diversity that happens, you know, the ecosystem solve it. I feel like we're not at a good whatever balance there is you need to find out that, don't think we're at a good balance. Like, how do you think about this, like, diversity argument when it comes to things like the package management and and stuff like that?
José:Yeah. So I no. I think I think you nailed it. It's a it it is a trade off, but I think I think I think I know I'm not sure if I can come up with, like, a rule, a precise rule. Yeah.
José:But I think but for me, it was, like, when it comes to package management, I think, like, it it is clear to me that it is part of the onboarding experience. Yeah. JavaScript, for example, it it does have other luxuries. For example, like, yeah, like, you you know, if you wanna do something for the browser, you have to learn JavaScript, which means that it some of the frictions that we have because we have to attract people, and then we have to convert to to to to convince them
Dax:Right. Right.
José:To make the onboard experience good. It's not necessarily a friction that JavaScript has because you can tell anybody, like everybody has a browser. You can just tell them, like, open the console and try it out. So I think, in a way, it's understandable for them to say, look. I we we we won like, we don't have some frictions here.
José:So it's fine for us to have friction in the package management space, and and and that's fine for us. Maybe it's a fine trade off to do it. But I think, like so for example, I do write web applications, and so we have the Phoenix web framework. And we went for a lot of churn, like, through the JavaScript tool. We went for branch, I think, gold for a while, and then webpack.
José:And and now, like, the first time that I'm, like, really, really happy is when we migrated to ES to build, which just seems to be, like, stable, very specific. It's not really focused on, like, changing things all the time. And I'm now really, really happy with it, but, you know, it it does generate traction. And, like, for me, I think, yes, package management is important enough to to to be worth unification. Maybe you you don't wanna unify within the first x years to let people try out and explore different approaches.
Dax:Right.
José:But I think the unification is going to is important. And I think, like, some of the new run times, I think they started incorporating more, like, Deno and Bun. Yep. They're like, we are all in this problem. And I think and I think that's a very good sign that, you know, like, well, this is probably we ex we explored enough.
José:We are comfortable in the solution. Yeah. And we are we are picking we are picking an approach. And I if it's really important at some point, it needs to happen. We went through this with, like, releases.
José:Mhmm. So for a while
Dax:Oh, right. Yeah. I do remember that.
José:Yeah. So, like, in the way that you would deploy Alethree applications would be how you would deploy, like, JavaScript applications. So you would, like, say, look. I'm going to run on this thing. It needs to have a node or something.
José:And I'm saying like, back end justifications. You need to have node running on that machine, on that environment, restart it separately. You install the dependencies, and you run the thing. But I think people coming from languages like Go, which allow it to kinda like, I'll just pack everything into this one thing and ship it. They wanted more streamlined solution for a a big chunk of time, like, between for, like, two years, if you ask an Elixir developer what is, like, the biggest pain for Elixir applications, they would say it would be deployment.
José:Mhmm. We also have, like we have a configuration system that we also took time to figure out how that should work. So for a long period of time, it was seen as, like, the the biggest flaw in Elixir. And then we went to the ecosystem and how they solve those problems to it was clear for us that from feedback from the community that it wasn't a problem worth unifying and provide, like, a unified experience, out of the box that we decided to bring that into the language.
Dax:Yeah. There's phases of, I mean, is this happens in business all the time. There's, the unbundling phase where, like, there's, a million little solutions for everything. And there's, like at some point, it's followed by a bundling phase where that gets, like, consolidated into something unified. So things you pointed out about the new runtimes, think, yeah, we might be entering a consolidation phase, hopefully.
Dax:But yeah, it's like there's just been a lot even on the package management side in JavaScript, right, you can use NPM, you can use PNPM, you can use Yarn and it on some hand, these each projects did improve on the previous ones, but the state we're in is in this weird fragmentation. And if someone is new, they're just like there's always a conversation about, okay, like bring the project, wait, do you have NPM install oh, sorry, do you have PNP install and they're like, what is that? And then you have to teach them about that, so those little friction frictions are hard even though, and I think you made a good point about everyone's forced to do JavaScript, like you you have to pretty much because you do web stuff, so the bar I guess didn't have to be as high to get adoption and that may that might that might reflect why some of the cultural things are there where we expect very little from the ecosystem of tooling. So I think I wanted to ask about so now that we're talking about more ecosystem stuff, so we started with like the language, kind of moving on to some of the other stuff.
Dax:You know, Elixir is not just a language, the fantastic set of ecosystem projects like Ecto and Phoenix. So Ecto is for the audience, it's like how you talk to your database. The JavaScript world you have things like Prisma or Drizzle, things like that. And then Phoenix is a web framework. So when did those start to emerge?
Dax:And I know I know like that was more I know Chris McCord was very involved in in in some of those things, but yeah, when did that start to emerge as, like, things you wanted to make sure that you guys had?
José:You know, I was always thinking about the language being extensible. Mhmm. And and then and and I knew it would be used for web because that was my background. That's where I had reached. I knew people, you know, they were coming from a little bit.
José:A lot of them were coming from, like, Ruby or from if I was invited to a conference, it was usually a web related conference even if it was not about Ruby. So I knew, like, eventually, something would appear. But even early on, when I was trying to validate if the language was extensible, I built, like, a a mini web framework, like Express or Synatra in Ruby just to to see, like, hey. Is this thing actually extensible? And to try to, like, try to dog food the what I was building.
José:But people are like, are we going to maintain this? And they're like, hell no. No way. I've I've I I did I did my share of, like, a web framework, and and Right. And I I I I I don't plan to to to to to to start, like, doing this.
José:I want to explore the ideas. So Acto came Acto came first, which was a database driver, and this was from Eric Miraeus Johnson. He's the creator of Hacks PM, which is the package manager. So he did, like, an amazing work because the community was at the very beginning. I'm almost sure he was still at university, and he was like, oh, we need a package manager.
José:So he built the whole package manager and the whole package manager infrastructure.
Dax:Wow.
José:But because we didn't have something to talk to the database, he built the libraries to talk to the database and the ORMs. So he was like, okay. Like, I'm I'm building everything, and everything that I'm building is this open source thing. Right? Yeah.
José:And I'm going to build on top of that. And the thing about the web framework, it was really interesting because because of the background, there are a lot of projects at the time that were trying to be, like, rails for Elixir. Or it was trying to be, like, the Sinatra for Elixir. And they were exploring those different approaches, but it was really, like, you know, rails for Elixir. And Phoenix was actually so when Chris McCord, he started with Phoenix, he did not start by having rails for Elixir.
José:What he actually wanted to do is that he wanted to create, like, rich interactive applications where you could exchange information with the server and receive it back at any time. Mhmm. And he was trying to do that in Rails, and then he it was not working out. Things could not work. It was too complex, could not scale.
José:And then he heard about Elixir, and he decided to solve that problem. And for me and and and for me, that's why it worked. He was not trying to bring something to the platform. He was like, oh, I want to leverage this platform in the best way possible. The platform was the solution to that problem, and he built Phoenix around that.
José:That's why today we have things like Live View because everything was designed from, like, the rich interactive user experience
Dax:Mhmm.
José:Angle. And so I remember seeing a presentation. I don't remember if it was on the first or the second Alexir call, and I was, like, immediately sold. I was, like, alright. Like, this is it.
José:And then I got involved in the project. I also see, like, my job at, like, making sure so and then so web so for example, web was expected. I knew what was going to happen. But for example, other things came out later. For example, NURBS, which is
Dax:Yeah. I was gonna bring that up as well.
José:For building, yeah, for building high end embedded systems. I am not it's not a domain that I'm familiar with, but I see it my responsibility to make sure that they are not blocked. Right? Like, if they have an issue with the language, they have an issue with the tooling, it's my job to make sure that they can address that and evolve further and not be blocked. So, you know, always trying to get people excited about building new ideas, new domains on top of the language.
Dax:Yeah. I mean, it's it's amazing how far it's spread. Right? So we thought about that web framing, you mentioned nerves. Like, I remember that so when I was getting really into Elixir, I was like trying to use it in places that was not appropriate and I was working at a company that did have an IoT device and I was like, let's let's put Elixir on it and we're like doing something Elixir and it ended up not sticking and like, know, they they held their stuff in Go, so they moved to Go.
Dax:They initially started in C, so going from C to Go, like, much much big improvements for productivity of the team. But then like six months later, nerves came out and I was like, oh, so I'm not I wasn't a complete idiot. I did feel like this makes a lot because like there's this all the supervisor stuff, like there's a bunch of different tasks that needs to run on these IoT devices and you see a schedule and coordinate them, so it felt like the elixir primitives made a lot of sense there. And I saw nerves come out, was like, okay, I wasn't dumb. This, like, actually doesn't make sense.
Dax:And that project has grown so much as well. It's really don't know if you probably never imagined that that would be a thing. Right?
José:Yeah. No. I would never if you told me. Like, there are a lot of things that we do today. Like, if you get in a time machine and say, hey.
José:Alex, we would have those things. I'll be like, sure. Sure. Right? And and and the story from Nerds, it's also so cool because they they had experience from building embedded systems.
José:Right. Right? And they were like, well, we have this Wi Fi driver thing, and it crashes all the time because, you know, the Wi Fi disconnects
Dax:Yep.
José:Or because it's, like, driver, you know, c operating system code. And then when they heard about supervisors, which is something that when there is a crash, it restarts itself. They're like, that's what we need. And then and that's what drove them. And then what and I think at the beginning, they were actually Erlang, and then what drove them to Elixir was the tooling.
José:They were like, we had the abstractions, but we also want the tooling. They were they they the tooling and making it fast to making it easy for you to get started and build something was something that really resonated with them. But, like, today, like, yeah, you can you can get you can write the elixir, and it's going to compile to the GPU. And you can get, like, large machine learning models and run on the GPU, and you can deploy in a cluster and route requests to them. Those are things that, like, you know, starting like, the the machine learning thing, even if it went, like, three years ago, I would not believe that it it would be something that would be happening.
Dax:Yeah. Yeah. It it is really crazy. It's it's in every space now. No matter what you're doing, there is probably a way to do it with Elixir.
Dax:And it actually makes sense. You're not forcing it at all. We So talked about language, we talked about the tooling, we talked about frameworks, but there's also like patterns, right, like how do you build applications that are maintainable. And in the Phoenix docs, they teach you about bounded context and like domain driven design and all of that. And that's another thing that's sorely missing from the JavaScript ecosystem.
Dax:It's kind of like, you're on your own to figure that out. There's no, like, guidance on, like, building this reliable, maintainable system. So, like, where where did that come from? How come that was an important piece of the story?
José:Yeah. The It's it's so it's so curious to hear that because the this was and maybe it still is, in some way, it's a little bit of a I don't know what it's I don't wanna use the word polemic because it's not really polemic, but it's something that it leads to questioning if that was the good call.
Dax:Mhmm.
José:But but, for example, like, what I what I've always what I've always tried to do so going back to the failed prototype, what I learned is that it's not because some other language does it, is that it's going to be a good fit. Right? We have to think about, like, what what what that thing is doing and why, and what are the benefits, and if that thing needs to make sense. But if you think and so you always have to reason about the process. And but even then, like, it's it's hard to be on guard all the time and not have something you slip through because that's how you always did things.
José:So you just continue doing that way because that's how you always did it. So, like, in the first version of Vacto, our database thing, we had, like we solved some problems that, I think, really elegantly compared to to other frameworks. So I love ECTO change sets, for example Yeah. Which is something that builds how you want to make a change to the database, while all other languages are kind of, like, doing it implicitly. You mutate the object and that other thing is reflected.
José:Acto allows you to build and build, like, this whole lineage of, like, the things that you wanna do, and I absolutely love it. But sometimes we'll just add callbacks, for example, and add things that are really just coupled to to your data. And and it took a while for for us, like, to learn those things. And again, especially because all we have is modules and functions. Right?
José:So how can you get a structure out of that?
Dax:Mhmm.
José:Right? And then Phoenix context was a way of saying, okay. We are going to provide, like, some guidance to you, but so you when you organize our code, you don't want to just, like, put everything into a flat namespace and just throw everything out there. That's clear to us. You also don't want, like you don't want to have, like, the the god objects.
José:Right. Because usually what happens in those systems that you have tables with 200 columns because you're just putting things. So what is the guidance that we can give people to say, hey. You know, you may wanna think how those things relate to each other and start organizing them into, like, distinct contexts. So there was a lot of discussion.
José:And at the beginning, we we had we had a concern that this would lead, like, to a paralysis fear or something like that. You know, like, people because now you're asking people to to, like, think. Right? It's like, hey. You have to think how you want to organize your code.
José:But maybe you are at a point out of your project where, like, look. I don't care, honestly. I just want to ship this and have this this proof of concept ready to show to somebody. Right? So there was some friction on that because, like, asking people to think is great, but if they don't wanna think for whatever reason, that's friction.
José:And it's it's fine to acknowledge that, yeah, it's fine if you don't wanna think about it now, but, you know, at some point, you may want to. So later, we were like, well, if you want to define one context for database table, that's not how you should do. But if you wanna do that, it's fine. But you have to think about the consequences and telling about design. And that was something that I don't think there was, like, a unique moment for that to happen.
José:It was really, like, learning from the mistakes that we've copied, learning from the new things, looking at the programs that we wrote, and trying to learn from that and how we can lead to better code. It it was a long process.
Dax:Yeah. No. And I think you guys landed a very good spot because even to this day and we struggle with that too. Right? It's the same thing.
Dax:Like, you can make something we wanna we really wanna focus on maintainability like code bases is last for years, like, do we help people achieve that? But that doesn't look great as a first look at what if you're trying to learn something, it kind of feels overwhelming or like too much, so that balance is tricky. But to this day, and I'm a big proponent of like, some of domain driven design stuff, but if you go and look like Google that phrase, you end up with like a thick book like this that has all of these concepts and it's like very complicated. So I I to this day, still tell people, here's a link to the Phoenix docs. I know we're not doing elixir, but just go read the ideas of how they think about these things.
Dax:It's not too extreme, it's gonna be helpful and you you should organize your code in that way, but it still provides like that long term maintainability thing. I was like, yeah, don't focus on Elixir part and I the docs aren't even too focused on Elixir at least the parts where they're explain explaining ideas about a context. But that's how I tell people to kinda, like, first, like, understand these ideas and and why they make sense.
José:Yeah. And and we didn't want you said exactly. Like, we didn't want to say it was a bounded context, and we didn't want to mention DDD.
Dax:Mhmm.
José:Exactly because then somebody's going to search for DDD, and it's, like, a really fake book.
Dax:Yeah.
José:Right? And and and that would add to, like, to they were like, well, if somebody knows DDD, they'll immediately recognize it.
Dax:Right.
José:So we don't have to tell them. Right? But we also don't think that grasping all of the, like, big parts of the DDD should be a day one requirement for you to jump into Phoenix. So we we we try to distill that.
Dax:I'm in the TypeScript world now, so a big thing is the type safety stuff. And I saw that, I think you guys just put out, like, a recent paper on some of the type safety research on Elixir. So Elixir has gotten massive without having, like, an incredible type safety solution. You know, that's still an area that I think is interesting and I think you guys have been exploring. So I think just just for the audience, what's a good summary of, like, the current state of things and and where do you see potential?
José:Yeah. So yeah. It's funny because the Stack Overflow survey results came out, like, two days ago. And I what is the term that they use? Ad admire.
José:Like, the two most admired languages is Rust, which is, like, really statically typed, like, to the point that error allocation is guided by a tech system. And then the second one is Elixir, which is not a very dynamic language. Like so that's something else that in Elixir, because we have better matching and we have words, we actually usually know what what are the types and the things that we are working with. So it but it is a dynamic language. So I thought it was very interesting.
José:And and but for us, like, why we are doing this? I think we've like, the Elixir team did a good job of listening to the community, like, historically. So, like, back this is probably now I mean, seven, eight years ago, like, a lot of people, they were wanting for the for a matter because the the language the language was started to grow and getting adopted. So now people are working in larger teams, and then you started having, like, discussions about, hey. You know, what is how are you going to format this?
José:And somebody's like, oh, you should do in this style. And then we had some formatters in the community. And then and then that was one of the things that we're like, okay. It needs to be unified because if it's not unified, like, you're always going to have a discussion. Like, somebody in the project is going to want to use this tool or this other tool.
Dax:Right.
José:They're like, it's going to be unified, very few configuration options for for better or worse, and it is what it is. And it's something that, for example, I struggle at the beginning, like, with the formatter. Right? Like, there there's, like, the joke. Like, the best formatted code is the code that I write.
José:Right? Is and then and then I I was pretty much in that camp. And and as we're writing the formatter, we're we're discussing with the team. And then I was looking at my own code, and they would ask, what is the logic for formatting this? And there was actually no logic.
José:It was just, like, it was just, like, you know, whatever. It's like, I think this is better. So it was a hard process, but we're, like, we're going to be better because of it, and then deploying. So if you ask that later community today what is the biggest thing that is missing in the language, they are going to say static typing. Mhmm.
José:It's like it's like the huge majority. It's not like, you know, it's like more like two more than two thirds. So I think it's I say, like, we are not I'm not sure if Alexa is going to be statically typed at the end, but I think I we have to give a honest try. Like, we have to try to do it. I tried to do it, like, five years ago, and it absolutely sucked.
José:But I'm used with my prototype, Saki. And and now we are, like, partnering with a senior researcher, all type of systems. We have a PhD student. Because the thing is, usually when you're starting a programming language, right, and you say I want you to have types, you design the programming
Dax:language
José:with the types system. And when you are in that scenario, doesn't apply to typescript. But when you are in that scenario, usually, the typescript the the type system is restricting the language. There are things that, you know, you you think it would be like, if you look at the program, it's like, hey. Maybe this should be possible.
José:But the types of this is going to be to be like, no. You can't do that because I can't understand that.
Dax:Right.
José:And language is like gold. They they use the type system as a mechanism to restrict the language and make sure that the language is simple. Right? But if you have a dynamic language, like, there was nothing restricting it. So it has a a large expressive power.
José:There's a lot of things that you can do. Right? So the challenge is that, well, we need to find out a type system that is going to obey the rules of this existing language. Right? And most likely, you're not going to to get a 100% fit.
Dax:Right.
José:It's not possible. You're going to have to make trade offs. And when you don't have this a 100% fit, two things are going to happen. One, because the type system does not understand that code, it's going to say, hey. This code is not valid.
José:But if you remove the types, the code runs. But if you add the types, it's like, this code is not valid. I cannot understand it. Mhmm. Right?
José:So that's one of the things that can happen. But the other thing that can happen is that it can say, oh, I don't understand this code. So what I'm going to do is I'm going to introduce a dynamic or a TypeScript, a any. But every time we introduce the any, it means that you're getting fewer guarantees from the type system. Right?
José:So that's the trade off. So what what we are doing is that so we were in a research stage, and we were like, okay. How do we have a type system that can map and understand as much of the LXR language as possible? So we are working, you know, with the the research at APG student exactly to do that. And it was really valuable work with them because we were able to get a type system that is based on set theoretic types, and I'm going to talk about it very briefly, that is based on set theoretic types and apply to the language.
José:And in the parts that it did not apply, they actually went and developed new theory. They're like, we can't do that.
Dax:We are
José:going to, like, to develop theory that, like, the mathematical proof that I actually don't understand, and and we are going to prove that this is good so we can incorporate that into the language. So right now, we have the formalization of a type system that can express, like, a really large amount of Elixir idioms and Elixir features.
Dax:Yeah.
José:So that's important, and that's based on satiratic types. And what that means, it's a thing is that the type operations, you think about it everything as set operations, which for me is really important because set operations, they you know, we we learn them, like, early on. Right? High school or even before. We think about unions, intersections.
José:We it's easy for us to reason when things belong to a set or not. Right. So a lot of the things that you if you work with type language, like, oh, bounded polymorphism, subtyping, like, all those fancy things. For us, it's always going to be, oh, that's asking if something belongs to a set. Right.
José:That's the union of two sets. So that's really elegant. So when I learned about this approach, I really enjoyed it. And that's what we did. So we have something that, in theory, maps really well, trajectory theorems, and we are getting out of the research approach.
Dax:Mhmm.
José:We're sorry. We're getting out of the research stage. Mhmm. And now we are heading to so that's very big validation, but we would work in practice. We would be efficient.
José:So for example, imagine that we had this type system, and now it's your compiles, like, two times as low slower, five times as lower. Is that a trade off that we are comfortable with making? So what we'll do is that we'll start implementing the type system. And because in Elixir, again, you usually know the types that you have because of pattern matching and parts, our goal is to actually leverage this information and feed that into the type system. So we can start giving you type errors without you declaring any type whatsoever.
Dax:Yep.
José:And this is going to be great because it's going to be you won't have to change your code. You get feedback. We are going to use this as an opportunity to improve the error messages, make sure the experience is great. We are going to use it as a way to validate the performance. And then if for some reason we are not happy with this initial stage, we will be like, alright.
José:You never had to change the language anyway. So we're just going to get a type system. We are going to throw it out. We gave our honest effort. Yeah.
José:But if it really works, then we can go to the next stage and tell users, now you can start writing types.
Dax:Right.
José:And so that's what we are doing. It's really like a it's really I think it's, like, it's really a honest effort in trying to make this work in a way that it feels part of the language that it doesn't feel fragmented. It doesn't feel like the language becomes, like, those two different things where one is dynamic and one is static. That's what we wanna do, but it's still unclear. I always try to temper people's expectations.
José:Like, the answer may be no. The answer may be, like, it it it's still not doable, and we'll try again in ten years.
Dax:Yeah. Yeah. No. That makes a lot sense. Mean, thanks for that summary.
Dax:That's, it's pretty exciting because, yeah, it's the thing you said about the the all the pattern matching code you write, there's just such use useful information there. And I think a lot of people think about, especially when they compare it to Rust, right, they think about, oh, a type system isn't useful unless it's like complete or like entirely sound. But what most people are just asking for is like better hints when they're writing the code and just being able to infer more of that from the code they've already written, like that's already like a huge milestone to like, it's like first step. You know, it can only get better from there. So yeah, I think for me when I desired type safety in Elixir, I just meant like like like a little bit more than what we have today and that kind of brings me to a practically like a like a good place.
Dax:So I think for me like inferring from the code you write, that's kind of what I think of when I think about, you know, when I want type safety. I don't necessarily think about like, you know, writing my types first and like doing all the them writing the, like, my implementation from the code, which I do do sometimes, but it's not, like, the default way that I work on things. So yeah. I mean, I'm I'm really excited that you guys are doing that research and, yeah, excited to see where that where that ends up.
José:Yeah. And I think and I think what exactly what you described. I think it's a common sentiment. A lot of people, they're not like, I we want the type system because I wanted to catch a 100% of my bugs or something like that, but it's because they're like we recognize and we appreciate that a type system is going to lead to a better experience. And even though it's not perfect, the the the better experience is worth it.
José:But we'll we'll we'll try to we'll try to we'll try to do both. We'll try to do the correct and good experience, and we'll see where it's going to fall flat.
Dax:Yeah. Yeah. Yeah. And I I think TypeScript has proven that, like, that's like the common sentiment as well because again TypeScript is not like a 100% correct or sound type system, but the amount of adoption has gotten people like it despite that. Right?
Dax:So yeah, that that does make a lot of sense. Cool. So that's that's everything I had. This is like a summary to the audience. The stuff we talked about, there's like a one or so hour conversation.
Dax:We covered a good amount of stuff that exists in the elixir space. This is still just tiny there's still like so so much more that we hadn't really covered. I've just so strongly recommend that everyone it doesn't matter if you're gonna use it in production, doesn't matter if it makes sense at your company. Go explore it. There's just so many good ideas there that you can learn from that will make the rest of your code better.
Dax:Just be inspired from like just like like Jose has been talking about the overall ecosystem, the way it all comes together and the consistency. And just like there's so many places where you can use Elixir. I think that just will change our expectations out of whatever language you're using today. There's a lot of opting for inspiration.
José:Yeah. I yeah. I've thanks for having me. I really enjoyed it, and everybody have a great weekend. Yeah.
José:And see you around.
Dax:Alright. See you. Good day.
