Ruben Vermeersch (rubenv)

Home Assistant & Volvo

Tue, 13 Aug 2024 09:11:24 +0200

Here’s a neat little trick for those of you using Home Assistant while also driving a Volvo.

To get your Volvo driving data (fuel level, battery state, …) into Home Assistant, there’s the excellent volvo2mqtt addon.

One little annoyance is that every time it starts up, you will receive an e-mail from Volvo with a two-factor authentication code, which you then have to enter in Home Assistant.

Fortunately, there’s a solution for that, you can automate this using the built-in imap support of Home Assistant, with an automation such as this one:

alias: Volvo OTP
description: ""
trigger:
  - platform: event
    event_type: imap_content
    event_data:
      initial: true
      sender: no-reply@volvocars.com
      subject: Your Volvo ID Verification code
condition: []
action:
  - service: mqtt.publish
    metadata: {}
    data:
      topic: volvoAAOS2mqtt/otp_code
      payload: >-
        {{ trigger.event.data['text'] | regex_findall_index(find='Your Volvo ID verification code is:\s+(\d+)', index=0) }}
  - service: imap.delete
    data:
      entry: "{{ trigger.event.data['entry_id'] }}"
      uid: "{{ trigger.event.data['uid'] }}"
mode: single

This will post the OTP code to the right location and then delete the message from your inbox (if you’re using Google Mail, that means archiving it).

Comments | More on rocketeer.be | @rubenv on Twitter

Recent tech reading

Tue, 09 Mar 2021 22:46:29 +0100

So much things going on these days, it’s already shaping up to be a pretty crazy year, in the good sense. Pretty much as I predicted at the start of the year, though it must be said that 2020 didn’t exactly raise the bar much. Pretty easy to clear that hurdle.

But that’s for another day. For now, here’s some interesting things I’ve been reading recently, in no particular order / theme:

Modules, monoliths, and microservices

Pretty common sense way of looking at this whole discussion. I’ve seen both ends of the spectrum and as always the right answer is: it depends. Inform yourself and choose wisely.

There certainly isn’t a solution that works for everyone, in every situation.

You need to be able to run your system

So much truth in this one. It requires a bit of investment, but it’s one of those things that act as a force multiplier: it speeds up developers, giving you faster development, more head-space to build a solid product and more time to focus on what actually matters.

Just consider the inverse: if you make their day jobs as cumbersome and frustrating as possible, how do you expect your development team to perform?

Any project I’ve helped roll this way of working out has benefited massively, so I recommend it each and every time. Talk to me if you need help with this.

Breaking down and fixing Kubernetes

As an ops person, I’m a big fan of these kind of fire drills, where you deliberatly damage a system and then try to fix it. Doing this as an exercise, when things aren’t on fire, gives you so much more confidence when things do break down for real.

Comments | More on rocketeer.be | @rubenv on Twitter

2021

Sun, 03 Jan 2021 13:29:09 +0100

And suddenly, before you notice it, the year has passed. And what a year it has been…

It’s easy to brush 2020 off as a year to quickly forget, given the pandemic we suddenly find ourselves in. But I’d rather not. Looking back, despite everthing we took for granted but currently can no longer do, it’s been a year full of great experiences, new friends, new business, launching things and lots of joy with the family.

I for one am very optimistic and excited what 2021 brings in terms of plot twists. You can’t always predict what will come, but flexibility goes a long way. Onwards and upwards!

Comments | More on rocketeer.be | @rubenv on Twitter

Angular: ng not watching for changes

Tue, 28 Jan 2020 19:15:44 +0100

Running Linux and noticed that ng serve or ng test don’t observe changes and thus won’t reload/test automatically?

This might be caused by simply having too many files in your tree, causing your system to hit an internal limit. This limit can be raised using the following command:

echo 524288 | sudo tee /proc/sys/fs/inotify/max_user_watches

You’ll have to do this after every boot. Want it persistently? Use the following:

echo fs.inotify.max_user_watches=524288 | sudo tee /etc/sysctl.d/40-max-user-watches.conf
sudo sysctl --system

Comments | More on rocketeer.be | @rubenv on Twitter

Rust: First Impressions

Tue, 14 Jan 2020 21:03:15 +0100

I’ve been studying the Rust Programming Language over the holidays, here are some of my first impressions. My main interest in Rust is compiling performance-critical code to WebAssembly for usage in the browser.

Rust is an ambitious language: it tries to eliminate broad categories of browser errors by detecting them during compilation. This requires more help from the programmer: reasoning about what a program does exactly is famously impossible (the halting problem), but that doesn’t mean we can’t think about some aspects, provided that we give the compiler the right inputs. Memory management is the big thing in Rust where this applies. By indicating where a value is owned and where it is only temporarily borrowed, the compiler is able to infer the life-cycle of values. Similar things apply for type safety, handling errors, multi-threading and preventing null references.

All very cool off-course, but nothing in life is for free: it requires a much higher level of precise input with regards to what exactly you’re trying to achieve. So programming in Rust is less careless than other languages, but the end result is guaranteed correctness. I’d say that’s worth it.

This very strict mode of compilation also means that the compiler is very picky about what it accepts. You can expect many error messages and much fighting (initially) to even get your program to compile. The error messages are very good though, so usually (but not always) they give a pretty good indication of what to fix. And once it compiles you’re rather certain that the result is good.

Another consequence is that Rust is by no means a small language. Compared to the rather succinct Go, there’s an enormous amount of concepts and syntax. All needed, but it certainly doesn’t make things easier to read.

Other random thoughts:

It’s a mistake to see a reference as a pointer. They’re not the same thing, but it’s very easy to confuse them while learning Rust. Thinking about moving ownership takes some adaptation.
Lifetimes are hard and confusing at first. This is one of the points where I feel you spend more attention to getting the language right than the actual functionality of your code.
Rust has the same composable IO abstractions (Read/Write) as in the Go io package. These are excellent and a joy to work with.
My main worry is the complexity of the language: each new corner-case of correctness will lead to the addition of more language complexity. Have we reached the end or will things keep getting harder? One example of where the model already feels like it’s reaching the limits is RefCell.

In all, I’d say Rust is a good addition to the toolbox, for places where it makes sense. But I don’t foresee it replacing Go yet as my go-to language on the backend. It all comes down to the situation, finding the right balance between the need for performance/correctness and productivity: the right tool for the job. To be continued.

Comments | More on rocketeer.be | @rubenv on Twitter

Go: io.Reader gotchas

Mon, 25 Nov 2019 19:47:20 +0200

I’ve really come to appreciate the elegance in the io abstractions in Go. The seemingly simple patterns of io.Reader and io.Writer open up a world of easily composable data pipelines.

Need to add compression? Just wrap the Writer with a gzip.Writer, etc.

But there are some subtleties to be aware off, that might bite you.

Let’s have a look at the description of io.Reader.Read():

Read(p []byte) (n int, err error)

Read reads up to len(p) bytes into p. It returns the number of bytes read (0 <= n <= len(p)) and any error encountered. Even if Read returns n < len(p), it may use all of p as scratch space during the call. If some data is available but not len(p) bytes, Read conventionally returns what is available instead of waiting for more.

This is fairly straightforward. You call Read() with a byte slice, which it may fill up. The key point here being may. Most IO sources (e.g. a file) will generally read the full buffer, until you reach the end of the file.

But not all of them. For instance, a gzip.Writer tends to do incomplete reads, requiring multiple Read() calls.

Recommendation: If you need to read a buffer in full, use io.ReadFull() instead of Read().

When Read encounters an error or end-of-file condition after successfully reading n > 0 bytes, it returns the number of bytes read. It may return the (non-nil) error from the same call or return the error (and n == 0) from a subsequent call. An instance of this general case is that a Reader returning a non-zero number of bytes at the end of the input stream may return either err == EOF or err == nil. The next Read should return 0, EOF.

Callers should always process the n > 0 bytes returned before considering the error err. Doing so correctly handles I/O errors that happen after reading some bytes and also both of the allowed EOF behaviors.

This means it’s perfectly legal to return both n (and thus read a number of bytes) and an error at the same time.

It also means that the standard pattern of immediately checking for an error is wrong:

// Don't do this
n, err := in.Read(buf)
if err != nil {
    // Handle err
}
// Do something with n and buf

Always process n / buf first, then check for the presence of an error.

Implementations of Read are discouraged from returning a zero byte count with a nil error, except when len(p) == 0. Callers should treat a return of 0 and nil as indicating that nothing happened; in particular it does not indicate EOF.

The important take-away here: always check for err == io.EOF, some implementations might give you an empty read even if there is still data to come.

Running into either of these corner cases is generally rare, since most IO sources are quite well-behaved. But being aware of the corner cases will save you a massive amount of debugging once you do run into them.

Comments | More on rocketeer.be | @rubenv on Twitter

Go: JSON and broken APIs

Tue, 22 Oct 2019 20:19:47 +0200

If you’ve ever used Go to decode the JSON response returned by a PHP API, you’ll probably have ran into this error:

json: cannot unmarshal array into Go struct field Obj.field of type map[string]string

The problem here being that PHP, rather than returning the empty object you expected ({}), returns an empty array ([]). Not completely unexpected: in PHP there’s no difference between maps/objects and arrays.

Sometimes you can fix the server:

return (object)$mything;

This ensures that an empty $mything becomes {}.

But that’s not always possible, you might have to work around it on the client. With Go, it’s not all that hard.

First, define a custom type for your object:

type MyObj struct {
    ...
    Field map[string]string `json:"field"`
    ...
}

Becomes:

type MyField map[string]string

type MyObj struct {
    ...
    Field MyField `json:"field"`
    ...
}

Then implement the Unmarshaler interface:

func (t *MyField) UnmarshalJSON(in []byte) error {  
    if bytes.Equal(in, []byte("[]")) {
        return nil
    }

    m := (*map[string]string)(t)
    return json.Unmarshal(in, m)
}

And that’s it! JSON deserialization will now gracefully ignore empty arrays returned by PHP.

Some things of note:

The method is defined on a pointer receiver (*MyField). This is needed to correctly update the underlying map.
I’m casting the t object to map[string]string. This avoids infinite recursion when we later call json.Unmarshal().

Comments | More on rocketeer.be | @rubenv on Twitter

Retro Operations

Tue, 21 May 2019 16:40:03 +0200

In his post Iconic consoles of the IBM System/360 mainframes, 55 years old, Ken Shirrif gives a beautiful overview of how IBM mainframes were operated.

I particularly liked this bit:

The second console function was “operator intervention”: program debugging tasks such as examining and modifying memory or registers and setting breakpoints. The Model 30 console controls below were used for operator intervention. To display memory contents, the operator selected an address with the four hexadecimal dials on the left and pushed the Display button, displaying data on the lights above the dials. To modify memory, the operator entered a byte using the two hex dials on the far right and pushed the Store button. (Although the Model 30 had a 32-bit architecture, it operated on one byte at a time, trading off speed for lower cost.) The Address Compare knob in the upper right set a breakpoint.

Debugging a program was built right into the hardware, to be performed at the console of the machine. Considering the fact that these machines were usually placed in rooms optimized for the machine rather than the human, that must have been a difficult job. Think about that the next time you’re poking at a Kubernetes cluster using your laptop, in the comfort of your home.

Also recommended is the book Core Memory: A Visual Survey of Vintage Computers. It really shows the intricate beauty of some of the earlier computers. It also shows how incredibly cumbersome these machines must have been to handle.

Even when you’re in IT operations, it’s getting more and more rare to see actual hardware and that’s probably a good thing. It never hurts to look at history to get a taste of how far we’ve come. Life in operations has never been more comfortable: let’s enjoy it by celebrating the past!

Comments | More on rocketeer.be | @rubenv on Twitter

New beginnings

Tue, 07 May 2019 17:03:21 +0200

A couple of weeks ago our first-born daughter appeared into my life. All the clichés of what this miracle does with a man are very well true. Not only is this (quite literally) the start of a new life, it also gives you a pause to reflect on your own life.

Around the same time I’ve finished working on the project that has occupied most of my time over the past years: helping a software-as-a-service company completely modernize and rearchitect their software stack, to help it grow further in the coming decade.

Going forward, I’ll be applying the valuable lessons learned while doing this, combined with all my previous experiences, as a consultant. More specifically I’ll be focusing on DevOps and related concerns. More information on that can be found on this page.

I also have a new business venture in the works, but that’s the subject of a future post.

Comments | More on rocketeer.be | @rubenv on Twitter

Let's talk about the developer experience

Fri, 05 Oct 2018 21:58:35 +0000

Yesterday, at the AWS User Group Belgium Meetup I presented a short lightning talk. It was a call to action for the fact that operations people should pay more attention to the developer experience.

Annotated slides of the talk can be found here.

This is an important subject to me: how can we make sure developers stay productive in the ever more complex environment of the cloud.

Photo: Nils De Moor

Comments | More on rocketeer.be | @rubenv on Twitter

Jupyter lab with an Octave kernel

Wed, 07 Mar 2018 18:44:11 +0100

Octave is a good choice for getting some serious computing done (it’s largely an open-source Matlab). But for interactive exploration, it feels a bit awkward. If you’ve done any data science work lately, you’ll undoubtedly have used the fantastic Jupyter.

There’s a way to combine both and have the great UI of Jupyter with the processing core of Octave:

I’ve built a variant of the standard Jupyter Docker images that uses Octave as a kernel, to make it trivial to run this combination. You can find it here.

Comments | More on rocketeer.be | @rubenv on Twitter

Go: debugging multiple response.WriteHeader calls

Fri, 26 Jan 2018 16:11:43 +0100

Say you’re building a HTTP service in Go and suddenly it starts giving you these:

http: multiple response.WriteHeader calls

Horrible when that happens, right?

It’s not always very easy to figure out why you get them and where they come from. Here’s a hack to help you trace them back to their origin:

type debugLogger struct{}

func (d debugLogger) Write(p []byte) (n int, err error) {
	s := string(p)
	if strings.Contains(s, "multiple response.WriteHeader") {
		debug.PrintStack()
	}
	return os.Stderr.Write(p)
}

// Now use the logger with your http.Server:
logger := log.New(debugLogger{}, "", 0)

server := &http.Server{
    Addr:     ":3001",
    Handler:  s,
    ErrorLog: logger,
}
log.Fatal(server.ListenAndServe())

This will output a nice stack trace whenever it happens. Happy hacking!

Comments | More on rocketeer.be | @rubenv on Twitter

Distrinet R&D Bites

Mon, 04 Dec 2017 18:49:02 +0000

The Distrinet Research Group at KULeuven (where I studied!), recently asked me to speak about “Cloud Native” at one of their R&D Bites sessions. My talk covered Kubernetes, cloud automation and all the cool new things we can do in this brave new cloud native world.

Annotated slides of the talk can be found here.

Comments | More on rocketeer.be | @rubenv on Twitter

CoreOS Fest 2017

Mon, 12 Jun 2017 09:51:52 +0000

CoreOS Fest 2017 happened earlier this month in San Francisco. I had the joy of attending this conference. With a vendor-organized conference there’s always the risk of it being mostly a thinly-veiled marketing excercise, but this didn’t prove to be the case: there was a good community and open-source vibe to it, probably because CoreOS itself is for the most part an open-source company.

Also fun was encountering a few old-time GNOME developers such as Matthew Garrett (now at Google) and Chris Kühl (who now runs kinvolk). It’s remarkable how good of a talent incubator the GNOME project is. Look at any reasonably successful project and chances are high you’ll find some (ex-)GNOME people.

I also had the pleasure of presenting the experiences and lessons learned related to introducing Kubernetes at Ticketmatic. Annotated slides and a video of the talk can be found here.

Comments | More on rocketeer.be | @rubenv on Twitter

Commercial open-source: Sentry

Thu, 29 Dec 2016 17:18:32 +0000

Commercial open-source software is usually based around some kind of asymmetry: the owner possesses something that you as a user do not, allowing them to make money off of it.

This asymmetry can take on a number of forms. One popular option is to have dual licensing: the product is open-source (usually GPL), but if you want to deviate from that, there’s the option to buy a commercial license. These projects are recognizable by the fact that they generally require you to sign a Contributor License Agreement (CLA) in which you transfer all your rights to the code over to the project owners. A very bad deal for you as a contributor (you work but get nothing in return) so I recommend against participating in those projects. But that’s a subject for a different day.

Another option for making asymmetry is open core: make a limited version open-source and sell a full-featured version. Typically named “the enterprise version”. Where you draw the line between both versions determines how useful the project is in its open-source form versus how much potential there is to sell it. Most of the time this tends towards a completely useless open-source version, but there are exceptions (e.g. Gitlab).

These models are so prevalent that I was pleasantly surprised to see who Sentry does things: as little asymmetry as possible. The entire product is open-source and under a very liberal license. The hosted version (the SaaS product that they sell) is claimed to be running using exactly the same source code. The created value, for which you’ll want to pay, is in the belief that a) you don’t want to spend time running it yourself and b) they’ll do a better job at it than you do.

This model certainly won’t work in all contexts and it probably won’t lead to a billion dollar exit, but that doesn’t always have to be the goal.

So kudos to Sentry, they’re certainly trying to make money in the nicest way possible, without giving contributors and hobbyists a bad deal. I hope they do well.

More info on their open-source model can be read on their blog: Building an Open Source Service.

Comments | More on rocketeer.be | @rubenv on Twitter