Often times when working with developing software you get this urge to think “if it can do X, what about Y”. When thinking this you get this nice feeling inside and a sparkle in your eye. It’s the feeling of being A Proper Engineer. As the stereotype of one is that they are capable of future sight and gifted with the ability of crafting the most amazing designs up front by just Thinking Right.
We have this in us, most of us, I think. This longing to live up to this ideal. Even when I know it’s all a blatant lie and that Proper Engineers™ Stack Overflow just as much as the next person. Fighting this urge is very important, as it means your setting yourself up for iterating and leaving the door open for the shocking possibility that you might be working on The Wrong Thing 😱
Working in a team, you have some tasks in there here and now which are quite tangible. Then there are people taking about possible future enhancements and next steps. As a senior engineer you have the responsibility to ensure you stay on the here and now as often as possible. Pretend there is a Fog Of War around you as of in a computer game. Instead of constantly thinking about what could be in the fog. You should ensure the team is just gradually nudging forward uncovering small pieces at a time. Ensure the team has their focus on present and near future topics and then just keep an eye out on what’s lurking in the outskirts of the fog. This is what good engineers does and what seniors should help newcomers to embrace.
What I just talked about contradicts the urge mentioned in the beginning. Our longing for being Proper Engineers™. Embracing the fog of war and being comfortable with uncertainty is a far more important skill for a developer than The Future Sight of the Accomplished Engineer. Our surroundings change frequently and thinking too far ahead is both distracting and it often ends up being wasted time and money. As by the time you think you are about to embark on The Next Thing™, priorities change and you’ll never see the next thing.
This is a common challenge once a company has built enough “stuff”. The burden of maintenance and the difficulties in refactoring makes one cry out for “Simplicity, now!’. A common perception of reducing complexity is to “dumb things down”, this is a reaction to the tension of it being difficult to untangle things. It’s often a good idea not to act on this tension, but look a bit further at the issue at hand.
Before even starting to consider doing anything, it is worth while to take a step back and analyse what exactly is complexity. People working together sometimes create established truths and create a shared understanding of how things are. Not based on research or analysis. There are multiple forces at play when a team starts talking about “too much complexity”. It can be a failure to deliver on their promises leading to a sense of failure and a need to place blame outside themselves. In companies with a hierarchy teams often create a lore of complexity as an explanation of why things are not going as planned. In the post The hidden meaning of complexity in the context of software development I talk about how the word complexity can mean something different entirely.
How to go about changing things?
One way to reduce complexity is to have to deal with less. Are there parts of the application which hardly ever changes? Would it make sense to split out isolated behavior from the main build? When doing this you can often get the counter argument of “we don’t want micro services!”. True, splitting out into separate might reduce some complexity and focused services that hardly gets deployed are great! However you can also isolate code which hardly change into libraries you compile in, reducing the number of things to build.
The misconception of thinking that reducing complexity always means removing or reducing things. You might actually reduce complexity by embracing new paradigms and undertake larger changes in your architecture. Doing this might require you to acquire new capabilities and learn new things, however this is not the same as adding complexity. Learning and evolving is always a key part of software development and should be just as natural to do when trying to reduce complexity.
Let’s say you have monolithic API which has grown out of control with long build times and a lack of efficient horizontal scaling, one way of reducing complexity could be to make drastic changes. Perhaps you need to insert a message queue and embrace an event based architecture? This might seem counter intuitive to reducing complexity , as setting this up is not that simple. However I would argue that often is this planned complexity preferable to the accidental complexity of a module which has grown out of control.
The means of production
When rescuing complexity an essential task is to look at the means of production, which is software is the delivery- and runtime infrastructure. Whatever path you choose in order to reduce complexity, it has to start here. If you continuous delivery system is fragile, or non existing, that is a place to start. If your infrastructure isn’t possible to replicate without manual procedures, that’s a thing to fix.
The reason for this is simple, no matter what you’re looking at doing it’ll be a nightmare if these things aren’t running smoothly. It will greatly reduce the types of things you can choose to do and it will also increase the time it will take to dig you out of the hole you are in.
Disclaimer: All my “On..” posts are things I write from beginning to end without any editing or thinking about structure (so it’s like all other posts?). They are just dumps of thoughts I’ve had which I deem that maybe they’re useful for something or someone, so I’ll just dump it here where nobody actually sees them.
A friend of mine told me his friends reaction when he said he was starting to learn how to play the flute. They said: “why do you want to learn that now?”. What they meant was why are you, a grown man, starting to learn to play an instrument? As if that’s an outrageous idea and that it’s not something grown up people do!
It’s a really sad outlook if we’re not supposed to learn anything new in what in most cases will be half our life time! We learn a ton of stuff and then suddenly because we’ve gained responsibilities and have obligations we are to stop learning? This is a preposterous idea. Learning and changing is the most natural thing we do, so why stop? I know some people think it’s too late to change and things like that. What I think is the case is that we gradually neglect and pay attention to our own willingness and openness to change. That’s why we stop, because our minds are closed.
You stop growing and evolving. One example is often see is music. You can choose to stop exploring and being open to new impulses, which means you enjoy the same music as when you where in your twenties. There is another option, which is to continue being curious and open to the idea that just maybe there is the odd chance of someone being capable of creating music you might enjoy even after you’ve surpassed the age of 35 🤷🏼♂️ It requires a mindset of being open to new impulses and challenge what you belive is good music. Engaging and making an effort to understand something new.
Many new parents find themselves in the same position, having suddenly to deal with the fact that their lives are forever changed with the arrival of the infant. You can choose to constantly look at what you are missing out and the life you used to have, constantly looking for opportunities to get a tast for “the old life”. I believe this will only make you miserable as that life will never return. Instead one could choose to be open to the new things that your life now offers you. This huge change is a great opportunity to grow as a person. Learn new things about yourself and also be open to all the learnings your child will give you. If you pay attention, you will notice that the child is learning you just as often as you are learning it things.
What’s the key difference between someone “stuck” and one that evolves? It is, I think, a willingness to seek out new impulses and to be open to the fact that you might be wrong in your current assumptions. In order to grow and learn you have to be open to receive new impulses. It means you must reflect on your view points, you must and should dare to change your position on things.When someone comes to you and say that you perhaps could have solved something in a different way. It’s natural to go in defensive mode and try to explain why, that you didn’t intend it that what and you explain all the reasons why you did what you did. I’ve learned that this is not how you receive feedback. First step towards learning from feedback and input is to listen, like actually listening. Take in and focus on understanding exactly what the person said, without judgement and without trying to defend yourself. In order to achieve this, I think it’s vita to be open to the fact that you might have to change or adapt how you do things.
Starting out in the IT industry you work on the ground floor and all you have to do is to show you know the craft. Gradually you’ll be expected to take into account things outside the realm of just the one thing you know. The progression from junior to senior is not about years, it’s about widening your perspective and to evolve you understanding of what it is that you do. Often a good senior will get offered the opportunity to lead. This is often done without any real formal training or coaching mechanisms in place. A good crafts person is somehow automatically a leader.
This fallacy leads to many dysfunctional teams and some times destroyed careers.When accepting the challenge to lead you must be open to change. Everything you do is different when you are a leader and you have to be able to adjust your thinking and behavior. You have to put in the work to understand the power dynamics between leader and worker. In order to help your workers grow you have to learn how to activist listen to them and to turn that input into actions. It is an entirely new job, it’s like going from a car mechanic to becoming a nurse. The requirements of you as an individual have completely changed and you have to change. You have to be open to the fact that this will change you as a person and you’ll be a different one on the other side of your new position as a leader.
This isn’t only about the classic worker-to-leader scenario where openness to change is essential. During the Covid pandemic a lot of people have had to suddenly work from home. This is a great opportunity to learn new things about yourself, if you are open to change. One option is to dig your heals in and try to mimic “the good old office vibe” in a remote setting. We’ve all seen that it does not work and people just become really tired of the endless video calls. A different approach is to look at working remote as an opportunity to learn something. All trends point towards the new workplace being much more duos and flexible, so instead of fighting against it you should embrace it as a learning opportunity.
I have been so fortunate as to be part of several young companies and starting new teams in existing organizations. This post is is a summary of my subjective observations, you should treat it as such.
1️⃣ Introduce departments too early
All young companies wants to grow up and become proper organizations. In order to get a head start, many young companies start splitting up their organizaiton into departments way too early in an attempt to look good for stakeholders and potential hires. “See, we know how to set up a business!”
The issues is that doing this when a company is just 10-12 people (I have experience this myself on more than one occasion) is setting your self up for problems you could avoid. With the introduction of constructs such as product, sales and operations you are creating boundaries and a need for synchronization. When you have a department, of course there needs to be someone running it. Hence you need to introduce a leader. Once you have three or more leaders, then there needs to leadership meetings, etc. What the company has been doing is creating the constructs which prevent larger companies from being efficient. Communication lines multiply, hand-overs and synchronization is introduced. You also formalize the tension which is between say sales and product.
I have been part of companies lulling themselves into the corporate mentality way before it is required (yes, it might be required at some point. However not when you are so few people). As we all know, the c-title people in young companies are usually the ones who where there first. Not necessarily the ones who’s best equipped to handle the role (I have first hand experience of this, I was CTO because I was the first hire!). You might need to have all kinds of titles and thing in your pitch deck to investors, but you should probably not actually have any of it in reality until there is a dire need of them. There is no going back once you introduce it.
2️⃣ Prematurely introducing processes & practices to scale
Software organizations are like crazed teenage fans when it comes to processes and tools. They blindly follow their idols without really taking the time to contemplate why. You will see small companies mimicing companies the aspire to become, as they think that is what has made the great. That is a grave mistake, as most unicorn or gazillion euro evaluated company probably did not start out with those processes. They needed to install certain processes because of any number of things. Maybe they grew into a thousand people company in two months. Perhaps they acquired a company in a different timezone. Someone gave them a truck load of money and they suddenly need to ramp up quickly. Or they have a total lack of trust in their organization and therefor install rigid processes.
Whatever the case may be, one of the key factors for rapidly growing small companies is their lack of these processes. It is a huge competitive advantage to not have to do the Spotify model (which is the go to model of choice the past years in my country Norway). Instead of trying to do what the big ones do, young companies should cherish and embrace the time when they don’t need any of it. It is the most fun you’ll probably have and the time you’ll be having the most creativity and being the most productive. Young companies should postpone installing processes as long as possible and most likely invent their own thing once they do.
The same thing goes for tools. There are certain tools which inevitably will make your organization less efficient (yes, I am talking about that product starting with Jir). Why does a 4 person development team need a bug tracker? If you only have the capacity to do two things in a cycle, do you really need a roadmap tool? I would say no. In my first startup we had a bug tracker, it was a sheet of paper which could only fill 10 bugs. If there where more, we’d either need to solve one or take one out. The sheet was visible for everyone (this was in ancient times when we used to go to things like The Office) and when it changed there was always a discussion about priorities. We did the same thing with items to do in a cycle. To date, this has been the best tools I have used to get work done in a small company.
The problem with task managers etc is that they put focus on the items, not on what value you want to create. Everyone gets focused on the individual work items and forgets about what it is that we bring of user value this cycle. The outcomes of each cycle becomes less important than completing the things we thought was a good idea when the cycle started.
3️⃣ Letting the engineers decide on their own
From https://twitter.com/thepracticaldev
You know what is going to happen when you let some engineers have free reign and build The Greatest Technology Platform Ever™ to propel your startup into the stratosphere. They will prematurely create something they think resemble what their idols do. It will be Planet Scale from day 4 for all of the companies 4 customers and 1 concurrent user. I would not say that this is a big exaggeration, I have seen it happen on many occasions. The enthusiasm to finally be able to architect the system they’ve envisioned all this time. To finally put into practice all the blog posts that they have read, all without really paying much attention to what the company actually needs in the beginning.
You should never let the engineers decide all things on their own, they need to articulate how it relates to where the business is right now and the next three months. There is no need for planet scale if you’re shut down in two months because you’re failing to get users. Opting for a serious cloud provider because “it will enable us to scale easily later” might be a decision which prevents you from shipping what you need right now and eventually kill the company.
Engineers should be allowed to choose many things, but they should not be doing it in a vacuum without close collaboration with everyone else. Prematurely architecting something complex has prevented companies from getting the chance to succeed more than on one occasion.
Take a deep breath, hold it for ten seconds and then exhale. Relax, put your shoulders down and instead try to enjoy what is the best time of a companies life. Rather than rushing into becoming a proper company, you should embrace the competitive advantage you have of not being such a company. The small size, lack of processes and hacky technology is the secret sauce unicorns are made of. They are not made up of practices from Google, Facebook, Amazon and Microsoft. If investors want structure, make a power point. Don’t mess up what potentially could be the best time!
Complexity is on the one hand something which can be measured by tools to give an objective value of how complex something is. However, this is not how I’ve come to see the word used in my work as a software developers. In this article I will talk about some of the aspects tucked away in the concept labeled complexity. It tends to be used to cover up subjective opinions or believes getting portrayed as objective observations. Being able to detect when this happens is an essential skill to ensure you are able to no loose your confidence in your own abilities.
In my early days as a developer I worked as a contractor on a project for a client where we where hired to get a project over the finish line. They where facing a deadline and needed some people in to product. This was a Microsoft based architecture and it was in the years just after ASP.NET was released. Upon entering the project we learned about the guiding principles by the clients own architect. What we hear left us a bit puzzled as the way in which we where expected to go about coding components and pages went against everything we thought was right. Essentially we had to write our web application as if we where stilling concatenating strings in Perl, but in C# on the .Net platform. We where only two fairly young programmers from our company at the time, but we both thought this looked funny.
We tried to advocate for ASP.NET actually being a productivity enhancement and something which could help us reach the milestones with less effort. Especially since what we where making was a booking calendar solution, which would be ideal for the built in calendar components in ASP.NET. We tried to advocate this and show how it would make things easer, more robust and reduce our time to market. The architect was having none of it. Presenting reasons such as performance and complexity for reasons to keep the current pattern of development.
In reality it turned out to be fear of loosing control and fear of the unknown which lead us to having to implement a solution which was far from ideal and which ended up consisting of way more code than was needed. Writing the calendar booking component in a “Perl-style” was in an objective analysis far more complex and added may more code paths to the solution than the one we presented using the ASP.NET controls. What was diguised as something too complex, was in reality something new and unknown for one individual.
More advanced does not always mean it is more complex
Knowing everything about all different software libraries and frameworks is really hard. Therefor it quite often happens that there are different levels of experience in a team. Developers who’s spent more time with a certain piece of software will know different patterns of how to solve certain things than the novice just learning it. In cases like this the seasoned practitioner could be faced with their solutions being “too complex”. What the person uttering this often should have said is this: “I was not aware this was a way go about solving this problem. Could you please tell me more”. Figuring out how to best leverage a framework or library takes practice. Especially senior developers are quick to jump to the conclusion of something being “too complex” because they don’t understand something or lack the practice in this particular framework or library. When encountering this, it’s important to outline how the statement of something being “too complex” is false, it’s just a matter of a more advanced usage of the software we use.
I remember being thought this lesson as a young developer with the attitude and failures of many other young developers: I knew it all. Luckily I had a patient teacher in Per Otto who had the patience and skills to learn me new things. Where I previously would just “hack together” some code, he introduced me to the Gang of Four patterns as a way to more elegantly solve some problems. What looked liked something needlessly complex for me without knowledge of them, turned out to be elegant ways to solve common problems with fewer chance of errors.
I have on a few occasions opted to use a framework called Hapi for writing HTTP based APIs in NodeJS. This is not the de-facto option which most developers use (which at the time of writing is Express), but it has some traits I personally really like. With few exceptions the choice of Hapi is always scrutinized by developers accustomed to Express, as they are developed with two totally different philosohpies. Where one is about freedom of choice and providing few opinions in how to do things (Express) and the other is all about making decisions and laying out a preferred path as to how to use the framework to get the most out of it (Hapi).
Being introduced to Hapi is always a bit of a hurdle as there are conventions and ways to do things you just have to accept. If you don’t you will only spend a lot of time fighting a fight you will loose in the end. Most developers opt to follow the path outlined by the author, while others choose to fight the premises on which the framework was authored on every single occasion.
The concept of plugins is central to Hapi and seasoned users of the framework tend to rely heavy on building their solutions as a series of plugins. This concept does not exist in Express and the introduction of writing a piece of domain specific code as a plugin is often met with utterances such as “it is too complex” or “this adds needless complexity to our app”. What I think they actually mean to say is that “this is an unfamiliar approach to solve this issue” and instead opt to solve the problem at hand in their usual manner instead of using the mechanisms built into the framework to solve these things.
What is deemed complex by someone not familiar with the framework, is viewed as an elegant and preferred way by those with more experience in the framework. Writing a Hapi plugin does require some more knowledge, but it follows the architectural patterns outlined by its author.
🎬 In closing
Be on the lookout for the word complexity being thrown around in technical discussions. Much like other powerful words such as performance and security, it is often used to disguise something else.
There are of course some solutions, frameworks and technologies which are more complex (Hello 👋🏼 container managed persistence with EJBs in the Java stack of early 2000s), so there are situations where the argument of something being too complex is appropriate.
