Elliot Morris

I admit it! I maintain a shadow backlog. That is to say, I habitually spend time working on tasks that have not been prioritized by my team, but I have unilaterally decided are important enough to devote some fraction of my working week towards. I know, I should be ashamed. It’s a miracle I’ve not been chased out of the profession by now.

Luckily, I am in at a level of seniority where this is somewhat expected of me, even if that goes unstated for the most part. However, I have been doing this for the bulk of my career, and I’m here to tell you that you should too.

What I am not here to tell you is that you should prioritize your own gratification over the success of your team. Exploitative corporate structures aside, your team consists of your colleagues, and you owe it to them not to be a burden. (I know, this thinking can in its own way be another mechanism of control by the owning class, but not everything can be a class war my guy.)

I spend 10-20% of my time on my shadow backlog, and often drop it for months at a time when mainline work is more pressing. Sometimes companies formalize this as 10/20% time or Innovation Days. Whilst I appreciate the intent of these, I’m not a fan as it’s the flexibility that illegibility provides that makes shadow backlogs work for me, and frankly I don’t think employees need permission to be doing this.

That being said, take this all with a grain of salt. Shadow backlogs are not allowed. If you can’t do this without it adversely affecting the elements of your work that are actually important to your organization, you should not do it. However, if you find yourself just sitting there for extended periods of time, unable to muster any motivation, as I know a lot of us do, then keep reading.

Off the top of my head, here are some examples of things currently in my shadow backlog:

• Add mock injector functions such that particular service errors can be tested. There is some work coming up that will require this, and the mocking framework in our tests is haphazard and incomplete, I’d rather have the groundwork laid to avoid having to bundle the test complexity into the upcoming ticket, especially if more junior colleagues end up taking it.
• Finish up some gnarly work in our interop code generator around value types. The work this relates to got paused over the holidays, and having this done before we reboot will make the overall story simpler.
• Continue work on a side-bet of mine, involving an integration of our core framework into a DCC tool.
• Write a bunch of conceptual documentation for a client repository that I am the sole knowledge-holder for.

You know, before writing these out I expected that I would find I am disagreeing with many of the organizations prioritization decisions, but it’s not really that is it? It’s more that the cost/value calculation of just doing these, rather than communicating and planning and prioritizing, leans towards them being appropriate for shadow backlogging. I’m sure the specific texture of everyone’s shadow backlog will be different, but what I am sure will remain constant is that you, like me, will have more context than anyone else in these particular domains.

In particular, what I mean by that is :

• I have more context in technical minutia around areas of which I am familiar.
• I have more context over my own motivations and capabilities at any given moment.

The second one is the really important one.

Perhaps you’re a machine and can take any task given to you and churn through it. Fantastic, I’m proud of you, and you don’t need a shadow backlog for the same reasons I do. For the rest of us however, we are not machines, and are never going to be machines no matter what delivery framework we are placed inside.

A shadow backlog can serve as an escape hatch. There are times, rather frequent times for me, where I physically cannot motivate my hands and fingers to move such to progress a specific task. This sounds pathetic, perhaps I am broken. I don’t think so however, as I observe this behavior commonly in others, although people are ashamed to admit it. The arrogance of complaining that it’s hard to move your fingers to type code because it doesn’t currently speak to you? Bah! Don’t you know there are people in jobs who don’t have that luxury, who actually have to do labour? You pathetic, useless worm.

Maybe that’s true, but you know what, I don’t care. Shadow backlogs are a way of coping that works for me, and if you’re an employer who understands that humans are human, they work for you too. I don’t have to deal with the existential terror of finding myself physically unable to move forward on the one thing that is currently important, and the company gets a worker who is doing something rather than nothing.

The bonus win for everyone is that the things in shadow backlogs tend to be more valuable than the things that actually get prioritized for the reasons everyone knows but no one talks about.

As a closing thought, I’ll ask, are shadow backlogs an antipattern? I think probably yes. I have worked on teams where I didn’t need one. Communication was tight, values were aligned, the things we wanted to do and the things we needed to do were the same. However, we all know how rare and difficult this is, so I don’t begrudge this pattern as an effective coping mechanism.

Most software engineer job descriptions will have a requirement like this :

Has the ability to deliver ticketed tasks promptly and to a high quality standard.

This is well and good, it’s the primary gameplay loop of software engineering afterall. Receive ticket, make changes to match the behavior described in the ticket, make sure your code is reasonably readable and documented, deploy code to main. Via this mechanism, you deliver value.

After a time, you gain confidence in this, your peers and managers will praise you for your ability to do these tasks mostly unaided. The product becomes malleable to you, you start to think there isn’t any task you can’t accomplish given enough time. Heck, maybe you could even rewrite the entire product.

Congratulations on beating the tutorial.

Most organizations would have already promoted you to senior engineer by this point. This is an industry-wide mistake. Whilst I won’t go so far to encourage folk to turn down promotions, I will encourage them to avoid conceptualizing themselves as experts before they are ready. The real journey has only just begun.

Being able to deliver any given feature somehow is table stakes. Up until this point, you have not been contributing very much to your organization, not really, in fact you’ve probably spent a significant part of your career being a net-negative contributor in terms of absolute product value. This may seem shocking, clearly you’ve been delivering features, probably some customers even find them useful, but this is missing the point.

All change has cost, and although the organization will assert that the value of ticket delivery is always worth the cost of change, (otherwise they wouldn’t have asked for the feature right?) the truth is more complicated.

Creating any one single behavior in a computer system is almost always trivial for the experienced engineer. When the experienced engineer on your team says that something can’t be done easily, they almost always mean is that the thing can’t be done easily in a way that is acceptable to the health of the product. Junior engineers tend not to have to consider this constraint. Especially on teams that are more feature-mills, junior engineers will frequently add features in ways that are at best value-neutral, and at worst value-negative over the lifetime of the product.

This is intentional! All things exist in contrast, good and bad are paired, you must be allowed to fail in ways both varied and numerous in order to figure out what success even looks like. The technical growth that comes from this sort of work is the point.

It therefore worries me when I see newer engineers talking about their careers as though feature delivery is the final goal of technical growth.

This is a systemic failure to lead, but the pushing of this attitude is also arguably an intentional and malicious attempt to commoditize the craft of engineering to the benefit of a privileged few at the detriment of all software users. LLMs are a recent extreme accelerant to this trend, but they are not the cause, it’s been happening for a while.

For any given business need, I normally consider dozens of approaches to achieve the desired outcomes. Some match the expectations of the ticket author, some don’t, but nonetheless fulfill the actual requirements. Some approaches are high risk, some low. Some manifest their value in that they require no collaboration with other teams, some only work if there is an expert available for integration. All are immediately viable, all have trade-offs, many are secret dead-ends that will make your product less competitive in ways that are utterly illegible to the rest of the organization.

Beyond even that though, there are sometimes viable options that leave the systems we steward in a better place than they were before, and this is important. You want to get exponential? Here’s where it happens. What I mean when I say better here is undefinable, it’s a highly connotatively connected property that encapsulates business necessities, predicting the future, interpersonal relations, technical realities, ethics & culture, etc. These options don’t always exist, but they will tend to stop presenting themselves if an organization habitually avoids/is unable to identify them, and will present themselves more readily in the inverse case.

Exploring the shape of this make better quality across different contexts is the actual game of software engineering, and doing so will take you much, much longer than merely figuring out how to deliver features faster.

Perhaps this is also still just the tutorial. I’ll let you know if I beat it.

* I’d rather stop using the word feature, it belies a false perspective on what good technical work actually is and how it comes to be, but given these blogs posts are supposed to train me how to write without over-qualifying everything I say, I’ll just make do with this footnote.

Here we go. First “real” blog, 30 minutes, white page. How do I even do this? Should I write sections headings and fill them in recursively? That’s how I do PRD’s and such, although it feels a bit too planned for something like this, I think I’m just going to let my fingers run.

As I said before, the primary blocker I’ve been having in my writing, and my projects in general, is trying to say everything I have to say all at once. I need to pick a small, tiny, minuscule topic, and somehow try to ignore how everything is connected to everything else.

Ach, screw that, let’s talk about one of the big problems. Agreeing on a definition of value.

How on earth are we meant to say one thing is better than another? I’m advanced enough in my career that I don’t bring this up so much at work anymore, as it’s not really a helpful conversation to have when trying to decide on a technical strategy. However, if folks aren’t reasonably aligned on this, your organizations effectiveness is going to be hard capped in a particularly illegible way.

I suspect you’ve probably heard of the concept of “Gel” when it comes to technical teams, I read about this first in 1987’s Peopleware: Productive Projects and Teams, I think that book coined the term. Gel is, to use a personal definition, the tendency for two independent agents in a system to come to similar conclusions without having to explicitly communicate. It’s telepathy by the backdoor, and it can exist when folks values align such that their intuitions and deductions tend to lead them to the same place.

Whilst there are some downsides to gel (monoculture reduces rigor), the upsides are great enough that organizations generally try to encourage it. Some organizations even manage to realize that this isn’t possible without some sort of shared consensus principles to derive value judgments from, which is where the fun begins.

To solve this problem, organizations tend to produce and publish lists of corporate values. In the most common case, these are assembled by consensus, are vapid, self-similar, self-congratulatory documents, and are completely ignored by the majority of people. Nonetheless, I don’t discount these entirely, the are a potential definition of value, and they get credit just for acknowledging that this is something that needs to be considered. You have to watch out though, as is the norm with initiatives concerning formal definition of dynamic concepts such as this, any org that manages to be successful probably didn’t need the artifact in the first place, and conversely any org that really does need something, will almost certainly be incapable of producing anything useful.

More commonly, where shared values actually come from is by humans doing what humans do naturally, manifesting values and culture in an undefined, unguided sort of way. This is best, but is also unpredictable. If you’re smart, you can try to build an environment that encourages this sort of thing, but you can never really guarantee it.

I’m not optimistic on the future of shared values. Work from home, despite all its many upsides, has clearly eroded the natural value-alignment that tends to occur when two people share the same physical space. At least in my experience in the UK, the best of this tended to happen at the pub, in that sort of tribal ritual/group therapy session I’m sure most of us who worked in tech pre-pandemic are familiar with. This was especially advantageous as discussions often didn’t have to be couched and distorted such to avoid damaging fragile executive egos. Keep in mind that any substitute mechanism of alignment that occurs during work hours will be subject to this constraint, so will again be hard capped on its effectiveness.

Some folks might be reading this and be thinking that sort of control over value alignment sounds appealing, we can make sure everyone’s on the same page from base principles right? I’m going to assert that it’s incredibly difficult to actually change a persons values and you shouldn’t even try. In fact I believe it’s impossible by definition, perhaps I’ll explain why in a later post.

Many may think alignment is about changing hearts and minds, but it isn’t, it’s more akin to intimacy. There is a gestalt understanding of what everyone in your org thinks and feels, it’s ludicrously multi-dimensional, impossible to pin down, inherently inaccurate, but it is there. If you can tap into it, you can massively reduce your communication overhead, and be able to work through concepts that would be actually impossible to deliberate on otherwise. Yes, there are going to be elements of it that make you uncomfortable, that you might find icky, counter-productive or even offensive. Nonetheless, acceptance is a prerequisite to intimacy, which is a prerequisite to alignment, which is a prerequisite to high performing organizations.

If you’re looking for a prescription here at the end, I don’t really have one beyond the solution that solves every organizational problem. Hire only the right people, surrender control almost entirely, and hope.

I suspect this is rather a familiar story. You're a regular person in tech, you consume a lot of tech media, and you've had a long and storied enough career that you've got things to say.

However, outside of a dry technical writing context, you've never written before. You try nonetheless, and discover that it’s rather difficult to get the words out.

Christmas has just come and gone, and I have dozens of half-finished essays sitting on my hard-drive. They are all similar in that they try to address enormous, philosophically dense topics, and they all fizzle out as I find I don't have good enough writing muscles to communicate what I mean in a way that satisfies me

This does not come as a surprise to me. I'm a bit of a psychedelically inclined person, and something that becomes evident in that space is that it's a simple enough thing to experience “the grand truth.” What isn't easy is bringing those revelations out of that dynamic space into a form that won't dissolve when exposed to the winds of consensus reality. Revelation isn't work, what is work is pulling those concepts upwards, negotiating with them, and mocking their conceptual dependencies well enough that they can be communicated in a way that doesn’t require absolute connotative alignment between individuals.

Boy, that sounds like absolute nonsense, am I going to explain what I mean by the above? No I’m not, that's the point. I am not yet capable of doing so, I haven’t developed the muscles for it.

Therefore, I'm going to write a blog a day for the next two weeks. Call it exposure therapy. They won't be significant or groundbreaking or particularly well written, but they are going to exist.