All posts filed under “management

Employee Churn and Software Rewrites

When knowledge walks out the door, so does the reasoning for many decisions that led to the current architecture along with it.

We all know that rewriting something from scratch is rarely a good idea. The most common time for people to fall into the “big rewrite” trap is when the original authors are no longer around to explain and defend it. Writing code itself is far easier than reading it, so given the chance, often developers will opt to start from scratch. As systems grow, evolve and become increasingly complex, the ability to understand it as a whole goes down. So we take what we know of the system and believe that we can create a more straightforward representation of it by starting again. But in the same vein as Gall’s Law, any complex system has evolved from one simpler that worked, and over time, any rewrite is destined for the same fate if taken over by a new set of developers. It’s too much work to understand the existing system because it’s complex. We don’t understand the system, so we take nothing but assumptions into our rewrite.

So how can you protect yourself from this endless cycle of rewrites? The most straightforward but most challenging solution is to ensure that you don’t have any employee churn. All the authors of the system stay on the team and never leave. Simple in theory, difficult in practice. Another simple but complex solution is to try and architect your software into discrete modules, which are individually easy to understand and refactor, but that can then be composed together to form a more complex system; Think microservices. The trouble is that there is a lot we don’t know about a system when we start, so unless you’re staying on top of your tech debt and refactoring your code to match your evolving understanding of the problem, then you will end up with a “big ball of mud”. Clear and up-to-date documentation can help. Especially documentation that explicitly calls out designs (Design Docs) and individual decisions (ADRs). But who reads documentation anyways?

This isn’t to say that there aren’t times when a rewrite is a good idea. Not all systems built for the challenges of their time will necessarily address the challenges of today, so starting from scratch may be the optimal solution. And when the original authors of the system are present to acknowledge the shortcomings of a system and how a new architecture could address them, that can be a strong signal that it may not be a rewrite for rewrite’s sake. 

The Only Constant Is Change

We can never step into the same river twice.

During my five and a half years at Which? I’ve worn a few different hats. I initially joined as a Ruby contractor to build a new publishing service. The project lasted a year and was a fantastic mix of successes, failures and learnings for the whole team. As the project came to an end, I pitched to my manager that I should come on board full time to lead the Backend team and help set the architecture’s future direction. That pitch led me to come on as a Tech Lead.

Previous to working at Which? I had spent a few years running a couple of startups with two friends. Even though we ultimately closed shop, we accomplished many great things, and I learnt a lot along the way. It was refreshing stepping back into a developer role where I could enjoy the feeling of satisfaction from writing code. Fast forward back to becoming Tech Lead, I found myself gradually coding less and being more involved in long term planning and decision making. That change started to reawaken the passion and excitement I previously had during my time running a startup for building a vision and using technology to get there.

I had grown a genuine love and respect for the craft of leadership and the possibilities that it could bring. So when the position of Head of Engineering became available, I jumped at the chance to apply. After pouring my heart and soul into my presentation and interview, I was offered the position. 

Throughout my career, the one thing I have come to expect more than anything is change. We are creatures of habit. Those habits form a significant part of what we do every day, from our morning routine, to what we eat for lunch, to how we do our work. So we avoid anything that threatens that and actively resist change. But what if on the other side of change is something better? What if, instead of running away from change, we embraced it? What if we became aware that what got us to where we are now won’t be what gets us to where we want to be in the future. And as a first step, if we recognise that change is happening around us all the time, we start to be comfortable with it and begin to use it to our advantage.

Changes in my role are not the only thing that has changed over the years.

Technology

Working in technology, the most significant change that we’ll experience is that of technology itself. In the space of a few decades, we have gone from using punch-cards to run Fortran and Cobol on large mainframes to now using ephemeral “Function as a Service” platforms such as AWS Lambda to run anything from Javascript to Go. 

Through the lens of Javascript frameworks alone, for a long time, the running joke was that it wasn’t worth learning any as a new one would be out soon to replace it. When the jQuery library came out in 2006, it transformed frontend development. Yet, only a few short years later, a whole raft of new frameworks came out, such as Knockout, Backbone, Ember, Angular, React and Vue all coming out within a couple of years of each other.

While changes in technology can often be dizzying and challenging to keep up with, the boosts in productivity they create can often be significant. Work that was once highly complex and required weeks or even months of development time is now available as drop-in modules that can be configured and running within minutes. The months of time that would have taken to build and maintain your own on-prem infrastructure can now be spun up all around the globe with just a few scripts. 

Making continual choices about technology is a key part of ensuring we are building products and platforms that will see us into the future, as well as sunsetting ones that are holding us back.

Engineering

The work of an engineer has changed significantly over the years as well. For most, gone are the days of Waterfall, and it’s rare now to find an organisation that doesn’t approach software development with an agile mindset. Yet the Agile Manifesto, which popularised many of the values and principles used by organisations today, is only 20 years old, having been published in 2001.

For a long time, engineers were seen purely as a delivery vehicle for what the business wanted. There was no say in what got built. Requests would make their way to a Project Manager, scheduled, built, and released as part of a yearly, or maybe even twice yearly, release cycle. 

When I first joined Which? in 2015, engineers worked in delivery teams. Features and tickets would be decided outside of the team but passed down to them to be implemented. Today, engineers work as part of cross-functional teams in collaboration with Product, UX, and other parts of the business to figure out what would be valuable for users and to decide what to build. Engineers are expected to understand their product’s users and to have a voice when it comes to the features that are delivered.

Not only has there been a change in how engineers interact with the organisation and people around them, but there have also been changes on a day-to-day engineering basis. Pair Programming, Trunk Based Development, Peer Reviews, Continuous Integration, and Deploy on Demand are some of the practices that have come into their own, as engineers have honed and sharpened the software delivery process. 

Organisation

In an organisation that has been around since the 50s, change is not only important, it is necessary. But ways of working can be difficult to change when they have been the modus operandi for many years, and yet long term success is based on the ability to adapt to new and different ways of doing things. Squads and OKRs (Objectives & Key Results) are two examples of how Which? has started to make that change.

The introduction of OKRs means that there is a continual focus on ensuring that what is being done aligns with the organisation’s objectives. If it doesn’t help the organisation meet its objectives, it doesn’t get done. Simple. This, combined with the introduction of Squads, means we can make better decisions on what we build to ensure we are constantly delivering value.

At the technology level too, with new competitors not encumbered by the challenges of maintaining systems that have been around for many years, we have to ensure that we strike the right balance between delivering new features, stability, and addressing tech debt. This is like trying to change the engine and the tyres on an already moving car—no mean feat.

Productivity and Success

How we measure productivity and success has also changed. Crude measurements such as lines of code written, the number of tickets completed, or team velocity, have been replaced with much more meaningful ones. Cycle time measures how quickly value is being delivered to users, driving you to look for ways to reduce bottlenecks. Deployment frequency measures how often you are getting new code in front of users, driving you to break work into small batches. Change Fail Rate measures how often your changes result in a system failure, driving you to improve and automate your quality checks before each release.

And on a personal level, at each step of my journey at Which? I have had to re-evaluate how I measure my own productivity and success. As an individual contributor, the feeling of accomplishment is felt with every line of elegant code, every successful commit, every successful build, and every successful deployment to production. Moving into a management role has meant the loss of that feeling of continual success. There are fewer feedback loops, and sometimes it can take months, if not years, to see the fruits of your labour. This is often a difficult transition that managers go through, so finding a way to measure your success is essential to staying motivated, learning and growing as a manager.

Summing-up

Change is as much about what you start doing as it is about what you stop doing. To move forward, we have to be prepared to give up something in return. Whether that’s technologies, ways of working, roles, responsibilities, or simply the metrics by which we measure performance.

Heraclitus, the Greek philosopher, said, “No man ever steps in the same river twice, for it is not the same river and he is not the same man”. The world that we work in is forever changing. More importantly, we as individuals are constantly changing as we evolve and learn to adapt to that world. Change can either take us by surprise, but if we take the time to look around and prepare for what is on the horizon, we can make the most of each opportunity as they arrive.


This post was originally published on LinkedIn.

Listening for the Real Challenge

A situation that often comes up in meetings is that someone will raise an issue, using a recent situation as an example of what’s going on, but the discussion switches into being about the recent situation rather than the original issue itself. For example:

Person A: We really need to think about re-architecting Project X. Just last week, the team’s changes had a significant performance impact on the production site. What could we do differently to make the system more scalable?

Person B: Marketing were the ones to ask for the changes {insert long back story about the feature and Marketing} …

Person C: Marketing are always putting pressure on us to deliver features that make little sense. Can’t we push back on them?

Person A: Well, the problem with Marketing is…

The discussion went from trying to address issues with the system to be about Marketing. It’s easy to get sucked into carrying on the thread because it feels productive, but the original point is forgotten, then the meeting runs out of time.

When a discussion veers off course, I try to remind people what the original challenge was to get things back on track. But often, precious time has already been lost. It can be difficult to rein a discussion back in if strong characters are involved or from not wanting to be rude and interrupt the flow of the conversation.

For everyone taking part, listening to understand, rather than listening to respond, is an essential skill to ensuring discussions stay on point. One question I learnt during my coach training is, “What is the real challenge here?”. I often say that to myself while listening to ensure that I’m paying attention to what’s being discussed. When a problem is being described, resisting the urge to latch onto something related but isn’t the real challenge.

Team Health > Team Output

The majority of my time so far as Head of Engineering has been spent looking at the team’s health. Team health is a broad term, as it encompasses more than “is the team doing okay?”. At a minimum, it means, are people on the team engaged and happy, right through to, are there processes and documentation in place to support them in the day-to-day process of delivering software.

Managers tend to spend a lot of their time looking at just the output of their team. Are we delivering? How can we deliver faster? How can we write better code? etc. But unless you have your underlying team health in order, this is the wrong thing to be thinking about.

If you’re the coach of a football team, the aim of the team is to score goals and win matches. So you could spend all your time focusing on drills, improving core skills, and practising shots. But what if all the players on the team are sick or injured? All the will in the world won’t win you the championship if no one is in good shape. You need to let your players rest and recover before you focus on their skills and scoring. The health of the team underpins everything about the output of the team.

Unseen Work

We often mistake the job we see managers doing with the job they are really doing. We’ve all asked, at one time or another, “What does my boss do?”. We see them go to countless meetings, ask questions, or send around documents. We then equate those activities with the actual job of being a manager. Until my promotion into the role of my departing boss, the perception I had of what they did was totally different to what I found myself doing. All this unseen work going on behind the scenes led to the countless meetings that led to the questions, and that led to all the shared documents. On top of that, the un-seen time and thought went into ensuring the right questions were being asked, and the right documents were being created and shared. This is no different to equating Engineers to writing code. There is so much unseen work involved in writing the right code. It is knowing what technologies to use, what techniques, data structures and algorithms to use, and how to put it all together into building a functioning product.

There is No Debugger For Leadership

There are many mechanisms for understanding issues and creating fixes when there are bugs in your code. Your computer will tell you when you’ve made a mistake, give you feedback and await your next command. So as a developer, you learn to live in this constant cycle of feedback. Constantly making mistakes and constantly making changes. It becomes second nature. But it’s through these constant mistakes that we learn what does and doesn’t work. We learn to see patterns and spot errors sooner. These mistakes and what they teach us are what make us grow.

Moving into a leadership role can often feel like the total opposite. There is no fast feedback for your decisions. Sometimes it may take months to know you’ve failed, and sometimes you may never know at all. So you can end up doing everything you can to avoid making mistakes. But just like the developer, you need to make mistakes to grow. It’s uncomfortable and painful, but you have to make calls when afraid or in doubt. You have to learn to live in the cycle of mostly absent feedback. Sometimes you’ll pick up on the effects of the choices you made, sometimes that feedback will be faint, but it’s only in that faint feedback that you’ll find growth as a leader.

The Other Side of the Fence

As I have gotten older and my career has grown, many of my thoughts and opinions on software development, management and leadership have changed. Reading back through some of my posts from ten years ago, I see many of the same thoughts and ideas in discussions with more junior team members. And it’s not that they are wrong, but how the effects of experience and having a different vantage point can drastically change your view of things.

Whereas I used to be for building a custom solution, now I would favour buying off the shelf more. I hated meetings, but now I value the high bandwidth communication and alignment they can give when run correctly.

A big factor is the difference that being accountable can have on your opinions. Before my promotion, I would often rant about how “we have to immediately stop doing X” or that “we need to start doing Y before it’s too late”. But now that I’m in the role, all the second-order effects I had not previously considered start to raise their ugly heads. I had only seen things through the lens of my IC role. Now I see them through the lens of someone responsible for the team and how they can meet wider objectives.

As my career grows, a lot of the thoughts and opinions I have now on software delivery, management and leadership will change. And all the things I’m critical of will look very different when I’m on the other side of the fence.