development

A flock of sheep in a green grass field. The sheep are a bit dirty, but all looking at you.

Do engineers dream of endless complexity and problems in their sleep?

Yes, it is a “funny” word play on the legendary sci-fi novel Do Androids Dream of Electric Sheep? The title of this article is something I have had in the back of my mind since very early on in my career and it has stuck with me.

Note! If you are offended by swearing, bad language and grammar you will have a blast reading this article as it contains a lot of it 😂

🌟Green field projects

The bare mention of the word makes people have a certain glow in their eyes and can almost see their pulses rise. It’s that mythical thing that you have heard about while making your way through an endless maze of legacy code written by what has to be some programming illiterate that came before you.

It is you chance to Make Things The Right Way. You have dreamt of this moment for so long and now finally you are to be part of such a project.

The architecture diagram is a blank canvas. You have heard the phrase “we will use the best tool for the job on this one” uttered by managers. Deep inside you know this will be great.

Let’s gooooo! 🏎

There is a project kick-off and everyone is super excited. Everyone has been reading up on what is The Latest thing to be using. The room (virtual or physical, it doesn’t matter) is buzzing and spirits run high.

Designers, product manager and programmers alike have all arrived at the promised land together and it just feels amazing. This time, we know We Will Get It Right.

Fast forward, the project might be in an ok or it is in deep 💩. The outcome is not what I will be talking about here, it is rather the completely irrational way in which we go about setting up Something New.

Three steps for f**ng things up when starting something new

Pick technology based on what looks good on your resume and your personal preferences
Make sure you are ready for scale from the first moment
Focus on ensuring the programmers are enjoying themselves

CV Driven Development vs “WE ARE SO F**KED” Driven Development

CV Drive Development (CVDD) is a thing, heck I’ve practiced it for years early on in my career. Why wouldn’t you? If you work on dead-end technology, that means you career is heading in the same direction. Naturally we have a tendency to gravitate towards what makes us look good.

In a green field project, you absolutely have to have a grinch onboard. Someone to ensure that the team are not going completely ape-shit in the candy store just because they can.

A green field project should be treated as a startup, meaning you have zero money. It should’ve been done yesterday and investors are on your case to shut you down. That makes for a _great_ climate to make technology decisions. Why you say? Because it means you have to be honest and focused. No more CDD or reading blog posts about The Latest Thing. It’s about shipping value so you’re not shut down tomorrow. Most people would then choose Boring Technology. You choose something that you know, something that will give you velocity from day one and that is simple to work with.

Google Scale Baby 🚀

A project manager or some stakeholder early on in the projects would usually layout the possible future. How this project will be a game change and that it will be critical for the business ability to grow in the future. It is intended to inspire and rally the troops.

Again, this is where you need a Project Grinch on hand. We all pretend we know about premature optimization, but let’s be honest we mostly use that as an excuse to skip things like testing, performance and security considerations 🤷🏼‍♂️ In reality, since this is a Green Field Project, we secretly are already at Google Scale.

We choose tools, frameworks and technology platforms which are often times months if not years in the future. Let’s face it you don’t need to setup a Kubernetes Cluster and create your own Development Platform from day one. In fact what you could do is to host the damn app on an iPhone 8. Because: YOU HAVE NO USERS!

Our Inner Engineer just can’t be silenced. We so desperatley want to do “what the big boys are doing” that we just can’t wait to get there. Which is why, especially on green field projects, we just go all out over doing everything. Be it the git repo structure & the pull request regime or the runtime platform you choose.

Green Field projects very quickly become the new legacy because we are not thinking about the context we are in. We are not considering what are the things required for the things right in front of us.

Developer Experience

The phrase has been abused by so many people with such a variety of agendas that I feel the meaning of the word is gone years ago. It is also a term hailed by Propper Engineers as the ultimate goal of any software development organization. If this is not your #1 priority, you are doomed. DOOOMED! As the art of typing instructions for the computer to run is The Most Important Thing. In reality it is never the most important thing, as it is always the business which is important. If that is not on track an amazing developer experience don’t get you anywhere but a new job for the engineers.

The Project Grinch would say: “Oh really?”. In order to ship value there has to be an effective way to get things in front of customers / users. I am not debating this fact. What I am sure of is that this should not be the ultimate goal to have ready from month one. I would say it doesn’t need to be there for years 1 or 2 either. Again, in a Green Field project, we have a tendency to just loose it on this point. Because FINALLY we can set it up just right. We can have the quick builds, with the automated tests. End-to-end tests and integration tests. Feature environments for all branches. Database migrations with rollback. EVERY DAMN THING!

The task of writing the code is so precious and whatever small obstacle preventing us from doing this has to be eliminated. Preferably with something shiny and nice. Whatever the actual cost and whatever the increase in complexity, it needs to be done. Let’s do a mono-repo and then when that starts to hurt we will just add more technology on top of it. Something like NX (in the JS eco system) which sounds brilliant. We throw all the shiny tools in to remove ANY hints of friction to the sacred art of writing code (completely ignoring the added complexity that goes with removing said friction both short and long term).

Don’t get me wrong, having all this in place already _could_ be useful. However all these things also come at a cost which you might not want in an early stage project. Let’s say you have a feature environment setup ready, so why not Just Use It? Well, it might be that this setup requires you to do feature branches. Which effectively makes everything related to continuously delivering value to users harder (yes, I know you _can_ do CI/CD with feature branches too. What I am saying is that you probably should not).

A different approach would be to say: “scrap that, let’s do trunk based development“. You reduce the complexity in getting things shipped and you a healthy mindset of getting things out.

How to not fuck up

Ensure you always have someone who has no stake in having shiny technology on their resume
Ensure you have someone who knows that any projects success hinges on the business side of things first and foremost
Ensure you have someone on the project to tell the Engineers to “stop f**ing about”

You are welcome 🙇

Fog Of War

Often times when working with developing software you get this urge to think “if it can do X, what about Y”. When thinking this you get this nice feeling inside and a sparkle in your eye. It’s the feeling of being A Proper Engineer. As the stereotype of one is that they are capable of future sight and gifted with the ability of crafting the most amazing designs up front by just Thinking Right.

We have this in us, most of us, I think. This longing to live up to this ideal. Even when I know it’s all a blatant lie and that Proper Engineers™ Stack Overflow just as much as the next person. Fighting this urge is very important, as it means your setting yourself up for iterating and leaving the door open for the shocking possibility that you might be working on The Wrong Thing 😱

Working in a team, you have some tasks in there here and now which are quite tangible. Then there are people taking about possible future enhancements and next steps. As a senior engineer you have the responsibility to ensure you stay on the here and now as often as possible. Pretend there is a Fog Of War around you as of in a computer game. Instead of constantly thinking about what could be in the fog. You should ensure the team is just gradually nudging forward uncovering small pieces at a time. Ensure the team has their focus on present and near future topics and then just keep an eye out on what’s lurking in the outskirts of the fog. This is what good engineers does and what seniors should help newcomers to embrace.

What I just talked about contradicts the urge mentioned in the beginning. Our longing for being Proper Engineers™. Embracing the fog of war and being comfortable with uncertainty is a far more important skill for a developer than The Future Sight of the Accomplished Engineer. Our surroundings change frequently and thinking too far ahead is both distracting and it often ends up being wasted time and money. As by the time you think you are about to embark on The Next Thing™, priorities change and you’ll never see the next thing.

An approach to problem solving with computer programs

When you start learning to code you develop an idea of what people who does it for a living knows. I remember when I started, I was convinced that everyone around me knew everything and was never in doubt about what the correct approach would be. There was little to read about how to approach coding except for academic articles and books.

In this day and age the situation is very different. There are countless resources where you can see how people approach problem solving using computer programs. You can watch live streams of people working on popular Open Source projects or watch tutorial videos. They all give good insight into the process of coding and shows how you can reason about issues you encounter. One thing you don’t get to see in these rehearsed videos is how they come up with the solutions and how they learned to solve it. In short, you don’t get to see their frantic googling, pulling of hair and general frustration which is usually involved. You don’t see the chat messages discussing topics and expressions of doubts about the solutions.

What I’m getting at, is that there are some things related to problem solving with computer programs which we don’t discuss that often and which is not shown in most coding videos. I will try to outline my preferred way of approaching a problem in an attempt to show that even though I have done this for a living in almost two decades, my approach is not one linear process where I know the steps nor the place I want to end up.

Make it visual

Regardless of wether it is coding up an UI component or creating an API endpoint to solve a data access issue, I have to make it visual. What I mean by that is that I got to draw something somewhere. It could be doodling something on a piece of paper, drawing on a white-board or putting some boxes’n things in a Google slide. This visual I try to update as learn more about the problem I’m trying to solve. More often than not I have to start over. Usually my first assumptions where wrong. That is not a problem! It means I’ve learned something and it’s better to “scrap” a drawing or slide than to revert code changes deployed in production.

I have met programmers who are able to process things like this in their head. They are able to find design flaws etc by visualising what happens it in their head. This is something I struggle with. I can perhaps “run” a couple of steps of a process in my head, but then I loose it. My short term memory is terrible and I’m also not that great at keeping focus when thinking. Therefor I commit my thoughts to paper or some digital tool. One nice side effect of this is that documentation comes pretty easy, as I’ve been doing that throughout the problem solving process.

The format in which I draw or the tool I use does not matter, it’s proximity and flexibility that matters. I find modelling tools constraining rather than helpful and enabling. Therefor I tend to use generic tools for visualising / drawing / doodling. There are formal approaches on how to modell software processes and create designs. Personally I find just “drawing whatever” is more useful then ensuring it is 100% correct UML syntax. I am not saying you should not seek learning techniques for modelling software. Everything you can learn which can be added to your toolbox is always worth checking out.

Large problems only?

My process is the same for wether I’m trying to create some distributed architecture or if it is creating a small UI component. Once the issue requires me to think more than a couple of steps ahead, this is my go to approach and it is something I feel comfortable with. I find that there is no problem too small where this does not help. Sometimes my head just isn’t in it and doing some simple drawings works wonders for me.

Solving the problem

Once the design and preparation is done it is time to solve it. I use similar approach when it comes to writing the code for solving the problem. You can visualise the problem to solve with a drawing, but you can also visualise the flow of a program using text. I learned this approach reading the book Code Complete by Steve McConnel. He said that before writing actual programming statements, you can flesh out the details just using code comments.

someFunction () { 
  // if there is an item
    // render delete button
    // if not render the add button
       // write the label of the item
  // then add a parenthesis and a number 
}

Once you have got some thing written down, read it back and see if it still makes sense. You should refactor the code block until it makes sense and it does what you meant it to. Perhaps you should extract a method or maybe you need to call out to a service and retrieve some more data. Refactoring when all you have is code comments is easy. As opposed fixing written and deployed code. I’m not saying this is something you must do, it is just one approach which I find helpful at times.

There are other techniques, such as test driven development, which serves the same purpose as writing code comments before coding (I am aware there are more advantages, but this is not the topic of this post). Others use pseudocode to achieve the same thing (check out Thomas’ post on the topic “I love pseudocode”). Learning multiple ways to solve problems is useful and something that will be beneficial regardless of the technological platform you happen to be working with.

“Kids, stay in school”

This article contains methods which happen to work well for me. Others will have different approaches and techniques. What is important is staying open minded and stay curious. To keep learning is essential to having a long career as a programmer.

Software development as a cooperative game

http://alistair.cockburn.us/Cooperative+game+manifesto+for+software+development

This quote got me thinking today, because if it really is a cooperative game and teams are how we structure people working to solve problems, shouldn’t we look towards other team activities for inspiration?

When you work as a team, it is important to have diversity and a mixed set of skills. Especially in software development cross-functional teams is something that can be a good thing. Now, I have worked on many such teams and I’ve interacted with even more. One thing that comes to mind is that many teams miss some of the key success criteria for teams in sports: trust and the focus on making others better.

Making others better

The last point is something which was a corner stone of Norwegian soccer coach legend Nils Arne Eggen. His philosophy on how to make a team of mediocer players perform better than teams with way better players was to have a focus on making sure everyone wanted to make their team mates better. Playing a pass that puts your team mate into trouble is not good. It prevents you from making a mistake, but putting the other guy in trouble is bad for the team.

In geek-culture we always strive for the next thing. Wether it is a new tool, metholdogy or technology. Regardless of what has been don previously, we always want to opt for something else the next time we do something. This of course is fun and everyone wants to be working with “the next thing”. It’ll allow you for a small period of time, until someone else has “the next thing”, to keep your head up high and look at all the grunts who work with “what we used to do”. Our continued strive for the next thing is of course a good thing, however it can also be harmfull. Especially in a collaborative game setting, where you want to make others better. Pushing ahead with “the next thing” leaving everyone else behind is not making your team mates better. What would be better is to push for “the next thing” while making sure everyone else can cath up with you. Naturally, this approach doesn’t come with as much personal glory or will make you look better then everyone else. However it will make everyone you collaborate with better. I believe that we can strive towards focusing more on making those arround us better. I also believe that this way of thinking is good for business. What good is one amazing team, when you have six others who are just lagging behind?

Of course, I realize that my opinions are a result of my upbringing in socialist Norway and that this way of thinking is perhaps very weird for some of you.

Trust

Trust is another thing that so many teams, especially in Norway (having worked mostly in Norway I can’t really tell if it is better or worse elsewhere), lack. You can see this lack of trust manifest itself in numerous ways. Micro-management is the legendary sign of a lack of trust in team members. A lesser known one is a team where every decission must be made when everyone agreees. This “democracy” is really just a front for a total lack of trust in each other. When a team consists of memeber who don’t trust each others abilities the democratic way of making decissions ensures that there is a sense of control (which of course is true, because in such team nothing really happens).

Cross-functional teams paired with a “flat hierarchy” is of course the worst kind of team where trust is just not in play at all. Using the “flat hierarchy” as an exscuse for micro-management is a classic. The director of what-ever-the-fuck needs to make decisssions on color. Or manager of God-known-what needs to be in place for user testing. All discuised as managers “contributing” and being “hands-on”, when it really is just good old fashioned micro-management applied because of a total lack of trust in the teams abilities to make good good work.

In closing..

What was the executive summary of this rant-post? It is that we are terrible at team work and most software development teams lack what great teams of other disicplines have. We need to broaden our horizon and look outside our fucked up industry to see how we should collaborate on achieving greatness

A collaborative game?

Photo by Michael Heiss

A popular mantra the past few years is that “software is a collaborative game”. However in a large organization or project, is this really possible?

When things get big and the number of communications lines exceed what is possible to handle for an individual, is it really possible to collaborate?

A behavior of teams when things get clouded is to construct a world which is tangible and possible to manage. They try to isolate themselves and put up mechanisms to detect breaches of their perimeter.

Architecture in such projects have a characteristic that there are numerous anti corruption layers all preventing a fail fast type approach. Each one added by the team which needs to have control over their domain. The result is that you get an architecture riddled with safety mechanisms you have a really hard time figuring out what went wrong when it does. Failing gracefully is good, but adding these mechanisms all over tour stack spells trouble.

There are of course other issues with teams moving into isolation. Collaboration with teams who’s main focus it to stay independent is challenging. How can you achieve a common understanding with protectionism as the strategy? Should you just accept the fact that teams need to isolate when things get complicated?

Optimizing the whole When everyone tries to create their own walled mansion with safe guards and security systems, how can you achieve a common way of doing things? Can you optimize the whole when everyone that would effect is only focused on their isolation?