I've been fortunate in my career to have worked with some amazing folks who have each taught me a different aspect about technical leadership and management. Like many before me I try to emulate these people in my own day-to-day and "wear the right hat".
By no means is my learning on technical leadership over (on the contrary, I feel like every month/quarter is a new beginning) but I've hit a point where I want to start documenting these slivers of advice and have something to refer back to. I'd urge anyone to do the same thing.
This will be a "living" blog post.
You may have arrived at this juncture in your career as:
<ul>
<li>The most senior engineer on your team</li>
<li>The only engineer in your team/organisation</li>
<li>The person bestowed the title of Tech Lead or Staff Engineer, etc.</li>
</ul>
Or another situation entirely 🤷‍♂️
It is likely to differ per organisation and industry, but technical leadership (for me) is being adept in the following spaces:
<ul>
<li>Programming</li>
<li>Architecture</li>
<li>Project management</li>
<li>Engineering management</li>
</ul>
I'll try to break down what I would expect to see from a technical leader in these spaces. It's important to note that these are all positions which would occupy a single person (or sometimes multiple people) so it's definitely a testing time but it gives you a flavour of what is to come if you would like to specialise in one of these areas. For example someone who specialises in Programming and has no particular interest in the other areas can progress to becoming a distinguished Individual Contributor. Technical leaders who enjoy engineering management and looking after people can go all-in on an engineering management track, etc.
<h2>Programming</h2>
It was quite difficult to pick a heading here. It's changed from Development, to Programming, then to Coding, and back to Programming. What I mean here is the nuts and bolts, writing and shipping code.
The difference as a technical leader is you need to, and I'll use the same terminology verbatim as it was first said to me as I've not found a better way of saying it, consistently nail everything you lay your hands upon. Codebases should be left in a <a href="https://www.britannica.com/topic/positive-sum-game">positive sum</a> state. Your work should be used as examples for the rest of the team. Leave <code>TODOs</code> but then make some time to go through and action these. Write tests that give your team confidence in being able to delete vast chunks of dead code. Improve the tooling on your project so it's quicker to ship code and makes feedback cycles shorter. Write detailed documentation and document debugging setups, and share all this knowledge with your peers and continually strive to make this better. Know "where to look". Leave lots of constructive comments in code reviews and prioritise code review over everything else as it is often a blocker to release. Expedite work which unblocks your colleagues. "Pull from the right", as in concentrate on getting things on the right of your Kanban board over the line, rather than starting new things and having lots of in-flight contexts to switch between.
Hopefully you have noticed that the amount of programming you are taking on now is decreasing, as you begin to explore other facets of Engineering, and well away from the critical path to delivery.
<h2>Architecture</h2>
You will need to be an expert in every domain node your team owns and have deep understanding of any adjacent nodes and be able to sketch these systems out on napkins/whiteboards when needed. You'll be able to visualise chunks of this architecture and, especially when planning something new, where your gap/hole exists and any constraints involved.
With your years of experience you will have developed a deep "back pocket" of best-practice, abstracts solutions which you can delve into when minds go blank. Does a solution already exist to a similar problem? Keep things which follow the <a href="https://en.wikipedia.org/wiki/Pareto_principle">Pareto principle</a> to hand.
Think in events and packets of data, and how these flow across your business. Security should not be an after-thought in these discussions, so try to keep a threat-modelling channel of your mind open or, even better, call upon a specialist who knows what they are talking about.
<h2>Project Management</h2>
This was the main thing that blindsided me about Technical Leadership. It should be Project Management with an asterisk though as it's more of a "hat" - please still let the product manager do their job. There's a <a href="https://tgvashworth.com/2016/07/20/technical-lead.html">great article by Tom Ashworth</a> which explains this role better than I ever could.
As a technical project manager you're going to be using the architecture skills mentioned above and apply them to a business need or requirement. When you do this hopefully a "unit of work" should appear, an example would be a task or a ticket (or maybe it's a collection of these things and it's something larger?). Can you measure the impact? Are you able to show this to your team in a simple, visual and digestable way to know if you're on the right path?
It's also your job as a technical leader to "front-load" your team with the learning you've done. Before the work starts, you'll have run ahead and scouted the area which you're hoping to have the impact in and hopefully noted edge cases and limitations before your team encounters these issues, and prepared or documented work arounds to unblock them. It's very hard to change things at the end of a project, even with the noblest of intentions, so I cannot stress enough how important this phase of a project is and that time should be set aside for it. If not you'll need to set time aside to deal with issues that pop up at the end, so adding the <a href="https://martinfowler.com/bliki/TechnicalDebt.html">cruft</a> multiplier to this means it's much more economical and efficient to do this at the start, rather than at the end.
Project managers are often the busiest people on a team, in terms of the amount of meetings and knowledge they have to consistently have at the forefront of their minds, so ask if you can help them out. My previous Engineering Manager (thanks DJMC) encouraged and gave me the freedom to pluck responsibilities from the levels above me even though I hadn't been granted an official job title, rather than waiting for cycles of performance reviews to roll around. It was in doing this I started to take on some of the project management tasks such as leading Agile ceremonies that would help take the burden away from our project manager but also allow me to make that step up.
<h2>Engineering Management</h2>
This is possibly the most alien part of technical leadership to Engineers. As <a href="https://twitter.com/skamille">Camille Fournier</a> writes in The Managers Path, a lot of the time you'll find a Senior Engineer moving into this role and it's important to acknowledge that you're starting an entirely new career as a manager; technical mastery will not really help you in this field as you will need to learn empathy and galvanising people for your cause.
I have very limited experience in this area but I'm on my path. Something I've found useful, which may sound slightly creepy at first, is creating a living document of notes for each person on my team and filling it with feedback which I collect whenever I notice something which deserves praise, shows growth or demonstrates company values - and also for things which could be improved. I find this always gives something to talk about in 1-1s and is great for performance reviews, and is better than digging through scribbled notes or trying to pluck thoughts from the top of my head.
I also don't exclude myself from this process. In the same folder I have the same "brag" document for me - <a href="https://jvns.ca/blog/brag-documents">Julia Evans wrote a great article about it</a>. If there's anything which I feel needs shouting about at my next performance review I'll put it in there. If I feel like I didn't really handle something particularly well, it goes in there too as something I can share with my manager and we can hopefully figure out how to use it as a stepping stone.
In turbulent times lead from the front. When everything is going smoothly lead from the back. Don't forget that in your own quest for career growth that others will share the same goals, so make space for them to become their own experts. Give Senior(ish) Engineers their own projects, ask them what they aren't sure about and support them. Check in with them regulary. Encourage them to read up on best practices and produce work to high standards. Pair with them on programming, but also if they want to present work to the wider company or community especially if they may need a hand with a presentation or public speaking.

photo-1507652955-f3dcef5a3be5

Technical Leadership

When I think back to my humble roots in web development, there was only production. Editing files directly using an FTP client, or SSHing into a server to fix a bug - they were some cavalier moments.
Things have moved on a lot since then. Tools have been created, processes have been developed and we've now got this amazing thing called <a href="https://en.wikipedia.org/wiki/Continuous_delivery">continuous delivery</a>.
Some things haven't changed though. So many times I've found myself either in public or private projects where an API will only work in production. The strangest edge case you've ever heard of will only pop up in production. Your product manager will insist upon seeing a feature in production, sometimes right before canning it entirely.
And when we find ourselves in similar situations, we cover our eyes with our hand, peek through the gaps and press the big red "Ship it!" button, even though we are taught (usually through experience or by others who have had the misfortune) to never do this. Especially on a Friday.
You'll get it in the neck. You'll be up until 3am fixing it. You'll have to bug your tech-lead who is on holiday for a script to fix it. Yadda yadda yadda.
I have been on both ends of this (the fixer and the fixee) and what I can confidentally say, more confidentally than I know the sun will rise in the morning, that I will be on both ends again. 
<blockquote>
Happiness can only exist in acceptance - George Orwell.
</blockquote>
I think the reason I'm so accepting of this is either that I have ripped the band aid off so many times the pain has made me numb, or I've tried to make testing in production relaxing experience - dare I say - a joy.
<h2>Feature flags</h2>
Feature flags (or toggles) are truly amazing. If I'm being honest I wanted an excuse to write an article about feature flags and I've not-so covertly managed to sneak it into an article about testing in production.
Why are they so good? They allow you to "wrap" different sized parts of your application in conditions which can be dynamically turned on and off. There's a few platforms which handle the "server" part of this for you, such as <a href="https://unleash.github.io">Unleash</a> and <a href="https://launchdarkly.com">Launch Darkly</a> - you store the toggles and logic on these platforms, and they afford you different "strategies" in which these can be activated and an API (or SDK) to access them.
The simplest strategies are a binary on-off, but you can also have user-based strategies which can check a whitelists, ranging to gradual roll-outs (for example, a percentage of your visitors) right through to custom strategies.
This really shines when you've got a nice single page app which is driven by an API. So long as the feature you're wrapping doesn't inadvertantly break something else on the page (GraphQL - I'm looking at you) and you have set a sufficient strategy that means random visitors are not going to see the feature, you can pretty much test whatever you like in production.
This does not come without it's drawbacks though. There's a fair bit of investment to make - in terms of time for building these wrappers into your UI (and maybe building a proxy endpoint for grabbing the feature flags in your API), and also financially in that you will need somewhere to host the feature flag service (either yourself or paying for a platform). You can achieve a very basic and crude implementation of feature flags with environment variables so that's always an option if you want to give this a try.
Also there's the opportunity for a less-experienced developer to open massive security holes with this too. Oh, I checked the Network tab and saw that <code>newSuperSecretButUnsecureLoginPage</code> is <code>false</code>? What if I set that to <code>true</code>?
With feature flags though you get, which I think is their most criminally underrated feature, is the ability for other people to deploy your features for you. There's been a handful of times were I've had to stay late (or rise early) to push something out, usually sat around awaiting the "go-ahead". It wouldn't be so bad, but you've still got to actually get through the deployment ritual and it's usually when something in CD decides to change or break. Oops, didn't add that environment variable. Oh no, now a test broke. And now I've fixed it coverage has decreased. It's a spiral of doom, why would you not want to instead just flick a (virtual) switch?
<h2>Logging</h2>
Feature flags are great on the front-end (and there's nothing to stop you using them elsewhere) but for back-ends it's hard to be as "visual" with your testing. What do you actually need to surface for testing?
Logs are your friend. I will be honest and say that it wasn't until I transitioned from web development into software development that I really used logs. Yeah there's the nginx logs or whatever - great, looks like I'm getting hacked. But actual application logging is super powerful and when used correctly allows you to build a very visual narrative to how your application is behaving.
Invest time in learning about your logger. Winston, the popular Node.js logger, is what we use day-to-day and a colleague of mine pointed out that you can create a parent logger and give it some attributes. This means you can search logs on an attribute and get all the subsequent logs back.
For example your application might look like this:
<pre><code class="language-javascript">import logger from 'winston';

function updateUser(userId) {
 logger.info(`Trying to update user ID: ${userId}`);

 try {
 // ...
 logger.info(`Successfully updated user ID: ${userId}`); 
 } catch (error) {
 // ...
 logger.info(`Failed to update user ID: ${userId} error: ${error.message}`); 
 }
}
</code></pre>
and your log lines will look not dissimilar to:
<pre><code>Trying to update user ID: 123
Successfully updated user ID: 123
Trying to update user ID: 456
Successfully updated user ID: 456
Trying to update user ID: 789
Failed to update user ID: 789 error: Network request failed...
</code></pre>
Not bad, but not great either. If we want to find out everything that happened to user <code>789</code> we have to do a fuzzy search which could throw up other things containing those numbers. However if we refactor to:
<pre><code class="language-javascript">import loggerFactory from 'winston';

function updateUser(userId) {
 const logger = loggerFactory.child({ userId });
 logger.info(`Trying to update user`);

 try {
 // ...
 logger.info(`Successfully updated user`); 
 } catch (error) {
 // ...
 logger.info(`Failed to update user`, {
 errorMessage: error.message,
 }); 
 }
}
</code></pre>
Our log lines now look like:
<pre><code>Trying to update user {"jsonPayload.userId": 123}
Successfully updated user {"jsonPayload.userId": 123}
Trying to update user {"jsonPayload.userId": 456}
Successfully updated user {"jsonPayload.userId": 456}
Trying to update user {"jsonPayload.userId": 789}
Failed to update user {"jsonPayload.userId": 789, "errorMessage": "Network request failed..."}
</code></pre>
Much better! We can now do a search like <code>jsonPayload.userId=789</code> and get the full story.
Hopefully you didn't need to read this article to realise logging is ace. It doesn't come without it's pitfalls though. If you trawl the logs of most tech companies you're bound to find some kind of Personally Identifiable Information (PII) floating around in there. So be careful.
<h2>Dry mode</h2>
If you're using logs to their potential then you can begin to use them to tell a story. Front-end work is, by it's visual nature, straightforward to demo to stakeholders, QAs or other engineers, but what do you do if you're a back-end engineer?
One thing I've tried to push when doing back-end work is being able to run things in some kind of "dry" mode (also called "shadow testing" or "toothless mode", etc.). When running in this mode a task would be carried out as it normally is, with lots of logging, however any calls to mutate any data are wrapped in a "are we running in dry mode?" check. These then just log what they would do (rather than actually doing it) and continue on.
Sounds simple I know but I love dry mode. I've had to deal with some extremely fiddly issues with a event-based distributed system and guess what... the data fetching only worked in production. We'd invested a lot of time creating an exact replica of the environment and thoroughly tested our code with several QAs, only to find several transient issues when ran in dry mode in production.
Again we have limitations as it's difficult to test any data which depends on any data that would have been mutated. You can use things like database factories or dependency injection to use fakes rather than the real rows but then you've got to justify this extra time investment for code you're probably going to delete at some point in the near future.
<h2>Conclusion</h2>
It's about trade-offs, and common sense too. I may have painted a rosy picture about testing in production but would you do this if it were avoidable? Hell no. Invest your time in things like building proper CI/CD pipelines, blue-green deploys which allow you to hit that deploy button at 5pm on a Friday, beer in hand.

Testing in Production

Although I don't get paid to write PHP anymore (my full-time job is full-stack JavaScript/Node), I still like to dabble with <a href="https://laravel.com">Laravel</a> as it's "batteries included" approach makes it really easy to knock together a concept in a matter of hours.
Something we've been rolling out at work at the moment is the integration of <a href="https://unleash.github.io">Unleash</a> into some of our projects. If you are unfamiliar with Unleash, it's an open-source feature toggle package which aims to be trivial to deploy, yet fully featured with a plethora of strategies to choose from - you can even make your own.
If you are unfamiliar with feature toggles/flags (spoiler: they are awesome), I'd suggest reading up on them otherwise this package will not really make that much sense! A TL;DR on feature toggles/flags:
Hiding features in your codebase behind toggles that are controlled externally.
Some of the benefits of this are:
<ul>
<li>You can experiment with rolling these features out, great for measuring the impact of a particular feature.</li>
<li>Non-technical people can choose when to roll out new features, for instance to coincide with a marketing push, taking the burden away from the developer.</li>
<li>You can test things in production without having to worry about degrading the experience for actual users.</li>
</ul>
Doing a bit of searching on <a href="https://packagist.org">Packagist</a>, there didn't seem to be anything that was compatible with Laravel which I thought was a bit of a shame as Laravel's <a href="https://laravel.com/docs/blade">Blade</a> syntax would lend itself perfectly to working with feature toggles.
So here's my attempt at an Unleash client:
<a href="https://github.com/mikefrancis/laravel-unleash">https://github.com/mikefrancis/laravel-unleash</a>
It's still very much a work in progress as I am still building strategies to handle all the default Unleash strategies.
If anyone would like to give it a whirl, I'm looking for testers at the moment to see how stable it and if any performance improvements can be made.
Happy feature toggling!

Software Engineer & UI Designer based in London, UK. Currently building and scaling a modern, renewable energy provider at Bulb.

Latest Posts

Technical Leadership

Testing in Production

Laravel Unleash client