Posted in:

This December I completed the Advent of Code puzzles once again (it's the 5th year I've solved in full). It's a challenge I highly recommend to programmers of all abilities, and it was especially fun for me this year, as one of my sons was also attempting the challenges in competition with all his classmates on a computer science course. (Impressively he got up at 5am each day to attempt the challenge as soon as it went live - I was somewhat lazier!)

Advent of Code 2022

I thought it would be fun to write a bit about what I think the benefit of attempting challenges like this is, and some of the key lessons that I learned by doing it. If you're interested in seeing my code, it's available here on GitHub, with solutions in C#.

1. Stretch yourself

One of the main reasons I think its worthwhile to invest in attempting challenges like this is that it will stretch you as a programmer. If any of you are musicians, you'll know that it's all to easy to "plateau" in your skills in an instrument. This is where you stop improving because you always rely on the same limited toolset of techniques and skills you've already learned. If you want to improve at programming, just like with music, you need to attempt things that are outside your comfort zone. Try things that are difficult, and be willing to go slowly and get things wrong.

2. Don't worry about "realism"!

Probably one of the main reasons many programmers will ignore challenges like Advent of Code, is that they don't view it as being "realistic". And to a certain extent, I can understand. In Advent of Code, I'll be regularly reaching for Regex to parse the problem input, and using breadth or depth first searches to find shortest paths, or choosing data structures like linked lists. None of these regularly feature in the "real world" applications I develop.

The puzzles also often require performance optimization, but they are CPU-bound, with no network requests involved in solving them. Again, in the "real world", performance issues more commonly revolve around network requests.

Having said that, there are many aspects of these challenges that do map quite accurately into the real world. For starters, there's the time pressure. Although one is forcing you to complete each puzzle on the day it comes out, I knew full well that if I wanted to complete all 25 days I could not afford to fall behind. So it often forces you to make the classic "technical debt" trade-off of doing it the "quick way" versus the "right" way that takes longer (and may not actually provide benefit in the long-run since its rare that you need to return to a previous day's code).

3. Learn to cope with changing requirements

One of the great things about the Advent of Code challenges is that they come in two parts. You get the requirements for part one, and only once you've solved that do you find out what part two involves. If you've written your code in a flexible and extensible way, it may be that part two is easy to solve. But just as often the rug is pulled from under your feet and you realise that the part one strategy will be useless for part two (often because it would be too slow).

What this means is that you constantly find yourself in the tension between YAGNI ("you ain't gonna need it") and YAGNI ("you are gonna need it"). As you're solving part 1, you're already thinking ahead to what the part 2 challenge might be. If you guess correctly, you'll save yourself lots of time for part 2. And if you guess incorrectly, then you just wasted a bunch of time.

This is constantly the case in the real world, where you know that requirements are going to change under your feet, and the ability to foresee the future direction is invaluable when making difficult to change decisions like database schema or defining service boundaries.

The reason that "YAGNI" is a thing is that we're sadly not as good at predicting the future as we think we are. Ultimately I think deciding whether or not to design for an anticipated future feature comes down to a trade-off involving the level of certainty that we have about the likelihood of us needing the behaviour versus the cost (and complexity) of implementing it before it's actually needed.

4. Improve your language skills

I often recommend Advent of Code as a great way to learn a new programming language (I've solved it in F# and JavaScript in previous years). But there's also real benefit in sharpening your skills in your main language (for me that's C#) - even if that's just making a conscious effort to use more of the latest language features.

One of the things that becomes apparent is that every language has its own peculiar strengths, and may be better suited to solving a certain type of problem. On of the challenges this year involved parsing a deeply nested array of arrays of integers. It took me a while to implement this in C#, but had I been using JavaScript or Python, the input was already valid code! And this year we quite often needed memoization, which in Python is as simple as adding the @functools.cache decorator.

5. Learn to aggressively optimize performance

Many of the challenges in Advent of Code require you to performance tune your code. The "brute force" or naive solution might work for part 1 of the problem, but once you get onto part 2, you need to look for ways to speed things up. And although I said that in "real world" apps you're more often looking at network calls and database queries as performance bottlenecks, many of the same optimizing principles apply.

In particular, you're looking to aggressively avoid doing more work than you need to. One of the key strategies here is avoiding making the same expensive calculation more than once, usually by caching (or "memoizing") the previous results.

One of my favourite puzzles this year was Day 14, where you simulated falling sand. You had to track huge numbers of grains of sands as they fell down and eventually came to rest. This was painfully slow if you simulated each grain from the top. However, I eventually realized that because each subsequent grain mostly follows the same path as the one before it, you can cut out most of the work if you only simulate it from the point its path diverges from the previous grain, resulting in a much faster solution.

In my day job I've actually been focusing a lot on performance recently, and I am convinced that in most typical enterprise applications there is significant potential for substantial speed increases simply by identifying and eliminating simple inefficiencies. Repeatedly making network calls to fetch the same data, or doing things individually that could be batched are two of the most common culprits.

6. Avoid silly mistakes by writing readable code

One of the most humbling aspects of attempting Advent of Code puzzles is realizing just how frequently I make "silly" mistakes. It's not so much that I don't know how to solve the problem, but that in my rush to implement something that seemed "easy", I often introduced bugs that took me a while to track down.

We'll talk about unit tests in a moment, but there are other techniques that help us avoid silly mistakes. Probably the most important is focusing on readability, rather than trying to squeeze the whole solution into a single line of code (which is usually only possible for the first few days of Advent of Code anyway).

For example, instead of packing more and more into tuples e.g. a method returning (int,int,string,int,int), define a custom Record to give meaningful names to your variables. Another of the really fun challenges this year was a Tetris-like puzzle, which I found relatively easy to implement, but messed up my final calculation by muddling rocks with rows. Its something that F#'s Units of Measure feature could have prevented, but giving all the variables meaningful names made it much easier to spot where my mistake was.

7. Write tests first

One of the really nice things about the Advent of Code challenges is that they each come with some example input and expected output. This makes it really simple to write some unit tests to validate that your solver works as expected before putting it to use on the real problem input. So it's a great opportunity to practice some "Test-Driven Development" where you write the tests first, before you write the code that causes them to pass.

Of course, this does slow you down a bit - writing tests takes longer than just diving in, especially when you already have a fairly strong idea of what you are going to write. And I think that's one reason why TDD hasn't taken over the world, despite all its benefits - we're often too lazy to discipline ourselves to do it.

However, the extra time of writing tests is often paid back when you find yourself needing to debug your code. Having a comprehensive set of unit tests helps avoid making regressions where fixing one bug results in introducing two more. The final day's puzzle this year was a great example of where having a set of parameterized tests meant I had a safety net while I reorganized my number base conversions.

Of course, even with good unit test coverage, bugs can sneak in. On one of the puzzles my unit tests passed when I ran them individually, but failed when running them together, because I'd forgotten to clear out the cache used in part 1 before starting part 2.

By the way, don't forget to use Git to regularly checkpoint your work. With TDD, you follow a "red, green, refactor" cycle, but sometimes the refactor bit can go wrong resulting in broken tests. So I always like to commit to Git after getting a test to pass. You can always squash commit later to avoid an excessively long commit history.

8a. Get to know the libraries...

Tackling the Advent of Code puzzles has the benefit of encouraging you to familiarize yourself with the libraries available in your language. In .NET this includes getting to know Regex, the various LINQ operators, and familiarizing yourself with all the various types of collections.

One of the crucial keys to being successful at puzzles like Advent of Code is developing your pattern recognition skills. Often the puzzles are simple variations on common problems (such as the "travelling salesman problem", or Conway's Game of Life). Once you start to spot these repeating patterns, you'll immediately know that you need a depth-first search, or that a sparse grid approach like using a HashSet is great when you have an infinite size grid.

And of course there's third party libraries that have already implemented many of these commonly used algorithms and data structures for you. I've often recommended MoreLINQ, but this year I tried out SuperLINQ, which is very similar but doesn't have the problem of method signatures that clash with built-in LINQ operators. Some of the methods I found helpful this year were GroupAdjacent, Window, TakeUntil and Cartesian.

8b. ...but also create your own utils

Having said that, one of the main lessons for me from this year was seeing how much time I saved by bringing just a couple of utility helpers I'd made myself from the previous year. I'd built basic grid and coordinate helper classes, and these both saved me huge amounts of time.

They're still missing some features I'd like to add, but they picked up a few useful additions along they way, and I'm sure if I do this again next year they will prove invaluable again. It's a reminder that sometimes the best utilities are the ones you built yourself because they work in a way that makes sense to you.

9a. Battle through even when its hard...

The final topic I want to touch on in this (already too long and rambling) post is what to do when you are "beaten" by one of these challenges. This thankfully didn't happen to me this year, although in previous years I have on a few occasions admitted defeat, not being able to solve the puzzle by the end of the day.

Of course in the "real world", if a particular task is too difficult for you to implement, there's usually the option to make the problem easier by negotiating the requirements. Arguably that's what I did for my Day 22 part 2 solution. I made a solver that was specific to the format of my personal input data, but wouldn't work with all possible inputs. I think I know how I could update my solution to be fully general purpose, but haven't got round to it yet.

Or you might write code that technically "works" but is just horribly slow. That was the case for my Day 16 part 2 solution which thankfully spat out the correct answer after 30 minutes, but was still running before I abandoned it a couple of hours later. I eventually used memoization to speed it up considerably, but it still took 90 seconds to run.

Generally speaking I think you get the best value out of Advent of Code if you battle through to solve it yourself, rather than just copying someone else's answer. So don't give up too quickly. But, having said that...

9b. ... but be willing to learn from others

I always learn a lot from seeing how other people tackle the same problem. Often I discover approaches I'd not considered at all. Sometimes really "obvious" ones, and other times, incredibly clever ones I'd never have thought of. And you also get to see the strengths of other languages (I find that Python solutions are often especially simple and elegant).

So when I'm beaten by a puzzle (or if I've solved it but want to learn how I should have done it), I sometimes allow myself to read the Solution Megathread on Reddit, and pick out a solution that makes sense to me. Usually this is in a different language, and so I'll spend a while porting it over to C#, which itself is an educational process.

The only puzzle I did it on this year was to find a faster way to solve Day 16 part 2, and this Python solution was the fastest one that I could understand, so my repo also includes a port of it.

Summary

In summary, no matter how long you've been programming for, tackling challenges like Advent of Code will undoubtedly improve your programming skills. You don't even have to wait for December to come around - the puzzles can be started at any time. Let me know in the comments of any similar challenges you know. Another that I really enjoyed was Wes Bos' free 30 days of JavaScript course.