The Ways of the Force - Mind Tricks
During my time as a Computer Technician turned Junior Developer 2 years ago, I learnt a great deal about what it means to be both a developer and an engineer. Though I can't list them all (we'd be here a while), I can certainly share those I've felt are most valuable. For some, I discovered them for myself. Others were bestowed upon me by a mentor as I was fortunate enough to be shown the ways of the force by a Master. With the start of a new year, I felt it important to reflect upon the start of my career to evaluate the present and future. Without further ado, let's get into it.
Even the lessons I do have time to share are numerous. So I may end up dividing this into two posts covering some of the technical and non-technical lessons learned under my mentor. I aim for both to contain a swarm of wisdom, tips, tricks, and approaches and thought patterns that lead me from student to professional developer.
With that in mind, let's dig into the software-oriented lessons. Specifically, this post will detail the non-technical group as should I write on the technical lessons later, we'll see how the advice below applies in practice. From here on, we'll be exploring how to improve one's professional frame of mind, adjust our focus, and improve our critical thinking and decision-making.
For the non-technical lessons, what better way to make these relatable and abstract them into simple mantras than through the medium of Star Wars (well, mostly). So grab a coffee and read on.
"Uuuuur Ahhhrrggg, Uurrrrr Ahhrrrrg."
- Chewbacca
While applicable to most walks of life, the age-old mantras of "take nothing at face value" and "read between the lines" remain key lessons of caution in software. Software and technology move as fast as our user expectations and needs do, and we have to keep up with delivering our work in a timely and quality manner. Often, this means applying established tools and frameworks to our circumstances to speed up development and provide assurance of quality.
I found this particularly prevalent during the business analysis or sprint requirements gathering stages. When working on existing systems, it may be that stakeholders understand what they don't like but are unsure or constrained in thoughts regarding their needs and desires as they may be unaware of the possibilities or constraints around those requirements. Similarly, we often need to think around and discuss the requirements that stakeholders might communicate in order to consider the feasibility of their requests, alternative solutions, or event the accuracy of their requests with regards to the underlying problem or aims it should address.
"Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix obvious to someone."
- Linus' Law
But, this can come at a cost. Open source tools and software thrive on the accessibility of reputable and supportive git-integrated online tools, such as Github, exploding in popularity. While Linus' Law stands true, it has a fatal dependency - having enough eyes looking in the same direction. Just because the source is publicly available, it doesn't mean it is highly supported. Bugs exist, and support can dwindle over time (implying it had any to begin with).
I've vivid memories from my internship of helping a fellow developer search for a front-end framework plugin for live search suggestions, some of which were abandoned, bug-ridden or lacked stability with habits of pulling the rug out from beneath you.
Now, I'm not pooh-poohing open-source - transparency is exactly what our industry needs and I don't wish to advocate avoidance of it. Only to say, be wary of the level of support they receive and evaluate the code and whether it is fit for purpose. After all, we have been warned about 'Evil Wizards' before. Github and Gitlab have great support metrics features - use them.
However, when applying Linus' law to our own teams, then I find it really depends on the organisation, team, and project sizes. In organisations with perhaps a single team of developers, particularly on larger or long-lived projects, it's important that members of the team are exposed to as many levels and areas of those systems and codebase as possible to maximise familiarity with its design and tooling in order to move quickly and accurately together. As the organisation or project grows, however, it's common to see responsibilities broken-up into smaller teams. For example, a front-end team, back-end team, SRE team, etc. Though the scope of change may be smaller for these teams, having an understanding of those other areas is still beneficial and impactful upon their decisions. But the law also applied internally to each team - each should be fully familiar with their area to maximise agreement on decisions, awareness of direction, and exposure of present or potential problems.
"Difficult to see. Always in motion, is the future."
- Yoda
Just like Newtons First Law of Motion states, our software will either remain at rest or continue at its current velocity unless acted upon by a force. Just as the distortion of space-time disturbs the straight-line trajectory of a planet orbiting around its star, changes to stakeholder requirements and end-goals should quickly influence the velocity and trajectory of our software.
Back in the 90s, our Agile forefathers recognised the art of making software was far too linear. They saw how these outside forces often had no effect on the trajectory of the software. And if they did, it only resulted in a great deal of disturbance and pain. Analysts had to regather requirements and edit specifications. Designers had to redesign. Programmers had to change the core functionality and behaviours in tightly coupled systems which expected no changes. They thought it about time we refocused our efforts and adjusted our workflows to react to these forces as they occur. And so Agile, often referred to as the imperfect replacement for the Waterfall methodology, was born.
During my time on my internship, I came to realise how the quote for this section embodies the simple core of the Agile movement. I witnessed how excessive planning and BDUF will inevitably concrete and complicate things for smaller projects, which reflects in the code and the end product. It became clear than in small to medium-sized projects, approaching each user story as a small cycle in the larger scope of things by incrementally planning, designing, testing and developing, was a powerful tool. With a little research and a little experience, I felt that to practice this, the following elements must be used:
- Effective communication with all stakeholders
- Transparency regarding the progress and state of the work
- Regular testing
- Regular refactoring
These 4 points all have an impact on one another. Our ability to be transparent regarding progress has an impact on how effectively we communicate. The more open and honest we are, we reduce the occurrences and costs of misunderstandings. With regards to the last two - having established testing stages delivers improved clarity and quality, immediate notice of issues during development rather than production, and greater flexibility and safety in refactoring.
With regards to my experience with these principles in practice, my mentor and I adopted and implemented the following:
- Centralised Source Control
- Git and Coding Standards
- Daily stand-ups
- Sprint planning
- Test-driven Development
- Pair Programming
- Code Review
- Continuous Integration and Automated Unit Testing
- Sprint review
- Sprint retrospective
Prior to the arrival of my mentor, the team rated about a 4.5 on The Joel Test. While it wasn't primarily a development team working on a single software product, a lot of small tools and large software was produced. We recognised a great deal of structure missing considering a high volume of code was being written and had very little regulation around it. So, we loosely adopted a hybrid Scrum/Lean/Extreme Programming style and begun mixing elements from each to have the best structure and organisation around the software and tools being produced we could achieve. By the end, I'd say we'd got that initial 4.5 score to around 8.5.
However, the main takeaways for this section were:
- Don't plan in a bubble
- Plan, but not excessively
- Iterate in short cycles with continuous feedback loops
- We should manage projects in a similar manner to Reactive Programming. Be reactive to, resilient of and loosely coupled in preparation for, inevitable change.
"Always remember: Your focus determines your reality."
- Qui-Gon Jinn
In relation to the previous point, this one stood out thanks to the process and thought reform which practising agile requires. Realistically, it can be applied to any of the principles of the manifesto, but I felt one of the most important was "Individuals and interactions over processes and tools".
If we put more effort into focusing on the individuals involved with the software and ensuring we're having regular, X interactions, our software will be very fit for purpose since it will have been crafted with regular review of requirements, iterative cycles of the full lifecycle process, and regular demos and feedback. A great example of this is the Three Amigos approach where each iteration sees a Business Analyst, Developer, and Tester (QA) collaborate and represent their stage in the cycle.
As true as this is, it doesn't mean forget the tools. Strong and appropriate tooling is essential to working or debugging fast, whether it be alone or with other developers. Tooling and infrastructure are vital to achieving critical processes such as code review, automated testing, and continuous integration and deployment.
Too often, it seems teams don't balance this - they go all-in on individuals and interactions, or all in on process and tools. It's a tight-rope walk. After all, you can't build a functional and quality table without a saw to cut the wood - no matter how many user-oriented meetings and feedback loops you have.
“You will find that many of the truths we cling to depend greatly on our own point of view.”
- Obi-Wan Kenobi
Further to the last point, I came to experience the value a user provides in the professional environment. Developing in a bubble only ends one way - it pops, and it's not pretty.
One of the first projects I undertook as a trainee computer technician was a simple one - relabel all printers to be more informative to users who wish to change toner cartridges themselves and to save the IT staff valuable time. Previously, the method was using a hand-held Dymo label maker. There were 150 printers and it seemed illogical and costly to hand-print at least 150 labels each time. So, I decided to automate it. Write a handy script to pull certain information about the printers such as their queue name and model numbers from the print server and determine where toner for that model is stored according to its location. This was an example where I decided the technical-minded vantage point was sufficient enough to determine the information required and the label design and production methods. It worked out in my favour. Ultimately, this is an example where the users welcomed the change but didn't feel they needed any input.
However, when undertaking one of my final projects as an intern (a centralised database initially with an internal lookup application atop), I knew the users needed to be involved at every stage. I needed to identify and understand my primary stakeholders and end-users, their use cases, and their expectations, among many other things. We needed information, so, I devised some high-level questions about a few specific projects, but mostly about overall service. I went out into the organisation to survey a random sample of our staff, students and visitors to gather as much information from varying viewpoints as possible. After this general feedback gave us more direction of which areas to focus our resources, I did the same again but specific to the project and its intended audience after initially asking how people currently solved the problem of locating or contacting colleagues. This became the requirements analysis and specification for the project.
In conclusion, involving your users is essential for developing a product to serve the businesses needs, and the users end-goals. Asking their opinions, asking for their pain points, asking how we can make an improvement to their everyday routine. We don't want to BDUF, but we want to take the time to understand our users and their problems - because they just became our problems. Our perspective as developers is not the perspective. Be open-minded and inclusive of those at the frontline of your product.
"Fear is the path to the dark side. Fear leads to anger. Anger leads to hate. Hate leads to suffering."
- Yoda
I thought this quote to be quite poetic when applied to my work. I will admit, as a beginner many moons before enrolling in University or taking my placement, this was the emotional rabbit hole I would fall down when I couldn't figure things out.
I would fear a challenge. If I tried to take it on and failed, I would get frustrated and angry with myself because I wanted to do it and I feared failure. Sometimes, this led to abandoning the project. After some thought, ultimately, it seemed the problem stemmed from a lack of belief in myself which resulted in a numbed ability to self-teach.
Thankfully, my years of experience and persistence with challenges have worked this out of me. I've adored new challenges for a while - I no longer fear the unknown. I've accepted failure is expected but can be prevented with good forethought and knowledge - mistakes are a learning experience which should be valued, not forgotten. I've first-hand knowledge of how I can contribute to a team and environment and I now know there is value in my skills - soft and technical. I've worked relentlessly to master the art of teaching myself driven by curiosity and the desire to learn rather than the need to. Overall, I'm more confident in myself which is the key to all else.
Though I learned this long before starting University, my placement did remind me of it. The environment, as supporting and nurturing as it was, did make me feel inadequate in comparison to my peers, at first. However, once I settled with my mentor and my team and began tackling the software-oriented projects and lessons which excited me, I quickly remembered how far I had come and how much further this experience would take me.
It reminded me not to fear a project or fixate on the "I've not done this before. Do I have the experience? Where do I get it?". It stopped me from doubting my abilities which would lead to fear. As ever, in the interest of balance, steer away from becoming a case study for the Dunning-Kruger Effect. Not overestimating yourself is as important as not underestimating yourself. We all progress one step at a time, just at different speeds What matters is that you keep taking steps - so don't fool yourself into thinking you take greater or lesser strides than everyone else.
"The master has failed more times than the beginner has even tried."
- Stephen McCranie
I admit this one breaks the Star Wars theme. But it's applicable nonetheless. Let me channel my inner Yoda for a moment:
"Want to know the difference between master and beginner, you do? Failed more times than the beginner has tried, the master has."
Close enough. It somewhat flies in the face of Yoda's "Do, or do not. There is no try". Confession time - I've never taken to this quote. I appreciate and agree with the "Jump in with both feet and be confident of yourself" sentiment, however, I prefer to take the view that to put one's heart and soul into something and still fail, is okay. More than that, it's expected. What you learn from failure is far more valuable than any reward gained from success.
No one expects your work to be perfect, especially not the first time. There is always constraints and trade-offs to be made based on the situation at hand and what's important is the communication and awareness of those. That's why we have prototyping, end-to-end spikes, and minimum viable products. It's another reason why we value short iteration periods of work, review, testing, and retrospective.
What is the value of success without mistakes and failure? A hollow victory. One must fail in order to learn. One must learn in order to succeed.
"Who’s more foolish; the fool? Or the fool who follows him?"
- Obi-Wan Kenobi
There is no end to the process, practices, and strategies for all stages of the software lifecycle. While many of these are founded on tried and tested methods, others are based on opinion, some are organisational-specific, and some are just slightly better alternatives to something else.
Be critical of them all. Remember that these processes are methods to better optimise the stages of development, but they're not the be-all and end-all solutions. More often than not, it's about finding a methodology which suits your situation or organisational structure.
For example, as discussed, Agile methodologies have seen a large increase in popularity for many software companies and even some non-software. However, consider whether short, rapid iterations of the full development lifecycle with less definition and gathering of requirements upfront is suitable for life-critical systems. With these systems, all stages of the lifecycle are critical and therefore deserving of the team's full attention as to minimise the occurrence and impact of errors.
Make judgments for yourself - tools, pre-rolled software, methodologies, best practices. Be critical. Don't be afraid to pick and choose to suit the needs of your team, project, or organisation. Be careful not to blindly adopt them expecting everything to go smoothly.
"Your eyes can deceive you - don't trust them."
- Obi-Wan Kenobi
An exception, error, or bug isn't always what it seems at first sight. Sometimes, "It's a trap!". How many times have you fallen down a rabbit hole, chasing what you believe to be a problem, but only to find hours (or even days) later that you were simply chasing one of many symptoms of a larger issue? Or even, a minute problem (cough - syntax error anyone?) I'd imagine there are few who could deny this has happened to them.
Remember that errors are telling you something - read them, and check the documentation. What variety of things could cause that to happen? What is related to it? Remember that solid problem-solving and debugging processes go a long way to ruling out red herrings and ultimately identify the underlying problem. Don't make assumptions and expect them to be true, and never eliminate anything as a possibility - jumping to conclusions will land you in hot water. Take your time, understand the symptoms, understand their causes, hypothesize, test. Make sure you are familiar with the tooling, its documentation, its help communities, and its caveats.
There is a wealth of information out there for how to approach such problems and dissect them, so if you're struggling with this, take the time to find and practice a methodology to give you structure in these situations.
Appendix: Bonus Advice I Came to Discover
"Great kid. Don't get cocky".
- Han Solo
Allow yourself to recognise and be proud of your achievements. But, just as we shouldn't underestimate ourselves, we should be wary to over-estimate ourselves and lose sight of that which is important - such as the project and its goals, relationships we form, and skills we develop.
In the past, I myself had firmly been in the 'underestimates oneself' camp. Now though, I realise I'm beginning to level out and see my value. I understand the unique perspective I bring to a team. But more importantly, I understand I can optimise how my perspective and skillset integrate with others. I think this is partly due to time and experience which have grown my confidence, but also thanks to a reduction in comparing myself to my peers. We all move at different speeds.
Test, test, test
What good is my saying "I've built a car that will do 150mph" if I can't then prove it? What if it can only reach 90mph? What if it can do 150mph, but to do so, you have to strip it of all its essentials and creature comforts - is making it go fast in its current design worth the sacrifice of it being a useful, family car? What if it can do 150mph, but in doing so, the breaks fail? Not good. This is what testing is for. It helps you identify the weak spots in the design; the edge-cases that you didn't think of; the unforeseen bugs and side effects potentially detrimental to suitable or even safe operation.
In failing to test, one could argue we are lying to ourselves and others. If we consciously make no effort to prove or back-up our claims because you believe it is working, you're enabling situations where the brakes fail at high speed - and they will exist. I'm not suggesting you test everything - in reality, there are often parts that simply don't need testing as we can rest assured they work - but we should be testing everything public and mission-critical. Let's not forget, we are often not (representative of) the users. Real end-user testing is invaluable to effective modelling and solutions, and when combined with constant collaboration, can reduce the cost of them identifying problems in design and construction near the end of the project. Test smart, collaborate throughout, test throughout, improve throughout.
Believe me when I say there is an endless supply of arguments, opinions, and advice on testing in books and online (and no doubt much of it is more conclusive and actionable than mine) but the message I'm conveying is that simply not testing because it is hard, or supposedly more time consuming, or a waste of resources - is simply mad. We, as the ethical, professional engineer, should be making the conscious choice to test, and choosing the methods which give us the most value and are most suitable for the current context. Be wary not to get consumed by coverage metrics, though - these are helpful, however, they detract from the focus on what should be tested and how, in favour of how much is being tested, in some cases.
Avoid lullaby language
Back in 2017, Marcus Blankenship wrote a post which both entertained and resonated with me about the use of some language which lulls us into a sense of false security. The term originates from Gerald Weinberg who saw that various words often carry an element of ambiguity or false-promise. As he exampled below, we're all familiar with this kind of language in daily life often having little malice intended, but we often don't understand the detrimental impacts it can have upon projects and people.
"should ("The customer should provide the sales data by August 1" -- does this mean "must" or "might"?)"
I can empathise with Marcus that one of the most disruptive in our industry is "just" or "simply". Whether it be from a stakeholder when specifying a feature or system or from someone involved in the project from a business analysis or technical perspective, it assumes an element of simplicity and ease which often don't reflect reality. Though it may be said in innocence, we must consider how those listening may interpret this differently and impact or reflect upon their wellbeing. We've all had a moment where someone has suggested something is as simple as "1, 2, 3..." but in reality, complications arise for a variety of reasons which mean it's not so simple or that it won't be delivered on time which can be damaging to a project and its running, but also to our self-confidence or mentalities. Underestimating complexity or unforeseen circumstances is never good, and such language can be a very blasé form of that.
Such language can also be a sign that someone familiar with a subject or thought isn't correctly communicating it with those who may not be and are assuming or implying a subtext. This is particularly damaging between language barriers and cultures when such implications of meaning or simplicity can be difficult for non-native speakers to understand, the blame for which lies solely at the feet of the lullaby language user. In these situations, Marcus advocates stopping the conversation and clarifying what is meant, or what assumptions are being made about the subject or people involved.
Then it's time to stop the conversation, and politely ask for the missing details. This used to be hard for me, because it made me feel like I was asking “stupid” questions. For many years I felt that if I asked people to explain what they meant, I'd look dumb. Or unprofessional. Or I'd be wasting their time.
I finally realized that professionals aren't content with generalities or vague requirements. They stop and ask for specifics, even at the risk of looking dumb. They have the confidence to know they aren't dumb, and to not pretend to understand something they don't.
You can use phrases like…
- “Let's pause so I can clarify what you mean. Are you suggesting that we…”
- “Wait, before we continue, can you explain that feature more?”
- “Going back to what you said, can you explain how you would implement that?”
- “I might be a bit slow here, but can you explain?”
Eat your own dog food
I came across this post back in June 2019. Safe to say, I was sceptical at best with a title such as that. But, I wrongly judged a blog post by its title because Lucas' story outlined something I'd experienced but wasn't sure how to phrase. And that is, that without belief and stakes in what we create, it's not fit for purpose. There are two phrases which I feel go hand-in-hand:
- Eat your own dog food
- Have skin in the game
The key in the previous statement is 'belief'. Enforcing your team only drink your sugar-water because you don't want them being seen with the competitors isn't what this is about. A show of force is a sign of weakness. If your product is the best, your team will want to use it. And if they don't - there's work to be done.
We should naturally have the belief and passion in the product you create or sell to use it. This drives up quality and development because we either have a real stake in its success, or we need it ourselves. It becomes far easier to prioritise what is needed versus what is not.
As Dave Thomas and Andy Hunt discuss in the Pragmatic Programmer Chapter 7, 36 'The Requirements Pit', when gathering requirements for a product, as developers, we should become the user, not impersonate how they think. That is, we should experience the product on the frontline. An example they use is sitting with the help desk staff for a week or two to experience the areas of issue users have. You can then take that a step further and engage with those users to truly experience their pains.
Well, this embodies that concept and goes a step further to ease that difficulty. Once we have skin in the game, everything changes. We feel greater responsibility, but also greater drive. And both of those things will drive up the pressure and ultimately the quality of the software and code.
Whether junior or senior, what are some key lessons you've learnt in your career that you would give as advice to new developers?