Let's talk about the developer experience
At the October 2018 edition of the AWS User Group Belgium Meetup I presented a short lightning talk. It was a call to action for the fact that operations people should pay more attention to the developer experience.
Slides: here.
Tonight I want to briefly talk about a subject that is dear to heart to me: the developer experience. This might be a strange topic for an infrastructure meetup but bear with me.
Let’s have a look at how our profession as software developers and system administrators has changed in recent years. Things are evolving all the time and one of the things I’ve noticed was the change in complexity.
We’re tackling larger and more difficult problems, which has led to an increase of the complexity of our solutions.
For instance, who remembers the good old days of system administration? Just
SSH into a server, do some yum install, perhaps rsync some files in there.
And that’s modern for that age, you could’ve been using FTP.
So easy, so fun.
But oh the horror. Things really quickly became impossible to manage: how do you keep track of what runs where, what versions are in place etc.
We’ve tried to solve this in many ways, but here’s something that resembles a modern stack nowadays:
So to make things doable, we’ve added all these hugely complex things, like the cloud, CI/CD pipelines, Docker images and orchestration tools like Kubernetes.
Now don’t get me wrong, the complexity is there for a good reason: it solves a real problem. As such we have to embrace it and appreciate the misery it takes out of our hands.
But we also have to admit that things have gotten slightly more difficult to understand.
For developers the same thing as happened. In the past each application was usually a simple monolithical LAMP application. A pile of scripts that talks to a database. Nothing special, trivial to understand.
Off-course, not every problem can be solved with a simple web application, so we’ve built many new technologies to tackle different problems. Think of different types of databases, caching systems, worker queues and messaging systems. We’re also increasingly slicing our software into many different pieces, micro-services, which bring their own heap of problems along with them.
So much more complex, but again for all the right reasons: what we can do now, often with a very small team, was completely impossible not so long ago, even with a huge team (and the budget that goes along with it).
The DevOps movement emerged while all this was happening. It’s the interplay between development and operation. These used to be separate silos: now there’s philosophy that both go hand in hand, together solving larger problems more efficiently.
And I think we’ve been doing a pretty good job on the operational side. We have all this great technology nowadays, it has completely changed the way operations are done, with a much higher quality and productivity as a result.
But on the development side of things, I’m not sure we’ve done an equally great job. Sure, all the new operational technology is a strong enabler for development, but we’ve also made the life of a developer much harder.
Especially for the kind of developer who doesn’t really care about the operational side of things. Say for instance a frontend developer who occasionally wants tweak some endpoints in a backend service (and thus has to run it).
In the new world of micro-services and tons of highly specialized external services, how on earth do you start running a local development environment that somewhat resembles production?
Especially when there’s many moving parts, with different technologies (languages) at different versions?
You can start mocking things, but at some point you’ll need to talk to real software.
If the first response from operations is this:
Well uh… you’ve mastered
git, you might as well start learningkubectl.
You’re not going to get a cheerful response.
And it matters. We can introduce all kinds of shiny infrastructure (for all the right reasons). If development gets pain in return, you won’t get buy-in and thus adoption will be very hard.
The net result should always be positive for all stakeholders.
So that’s why we should think more about the developer experience. Most operational technologies sadly put this very late on the roadmap, so there’s definitely room for improvement here.
As an example of what I’m talking about, here’s an example of work I’ve done
for Ticketmatic. Their platform consists of
around a dozen services running on Kubernetes. Getting this in the hands of
developers is tricky, so we’ve built a tool called devmatic.
devmatic is just a single command, which starts a
minikube-based virtual machine
that runs a full Kubernetes cluster, automatically provisioned with a trimmed
down development variant of the platform.
So no need for developers to go through complex steps, just run it and you’re ready to go in no time.
No need for complex configurations, no need to install different language stacks and no need to adjust things whenever somebody adds new dependencies. All of that happens automatically.
It also comes with a small and focused web dashboard:
It only does the crucial tasks: the different workloads are shown along with their status.
There’s some nice productivity helpers in there as well: for instance, you can easily view log output of all services, even the ones you aren’t working on.
But we don’t hide the fact that there’s Kubernetes underneath. For instance, if
you want a shell in one of the services, we give you a copy-pasteable command
to invoke kubectl.
In a way, this is the gateway drug to start using kubectl 🙂.
One other cool feature is the ability to switch to source mode. This reconfigures a service: instead of using the pre-built Docker image that comes from the CI system, it’ll mount the source code of your local machine into the cluster. It then runs it in a sort of livereload fashion. So whatever you edit instantly runs in the cluster, with all of the actual components and services available.
devmatic is just one possible solution, which I’ve talked about in much
greater depth last year at the CoreOS conference. There’s a full write-up of
this talk available on this website.
Things move quickly in the world of cloud-native software, so there are definitely things I would consider doing differently now.
I think the main thing we ought to be doing is share experiences. If you’ve been thinking about these kinds of problems, I’d love to have a chat. If you haven’t been thinking about this at all in your organization, it’s probably even more important that we talk.
Only by sharing ideas and figuring out a good way to make life easier for everybody will we be able to make that next big jump in productivity.
So let’s talk!
