Problem Solving Framework & Principles for Software Engineers

As I am reaching the 10-years-of-experience milestone in my career, there’s still a bunch of stuff that I am hopeless at. But, what keeps my impostor syndrome at bay is a couple of things that I have understood well.

For example, testing eluded me a decade ago when I started my professional journey as a junior software engineer. Writing code that will exercise and check my code for correctness blew my mind. It still blows my mind, but for different reasons.

Developing my approach to work has been another eluding thing. For a good part of my career, I have struggled to figure out how to approach projects. Often folks think having experience is enough to develop a framework to approach work. But they’re usually wrong. Hard skills and abilities are useless without a tactical approach to problem-solving.

In continuation, I will share my story of how I built a problem-solving framework. My goal is for you to use its tenets and tactics to aid you in your everyday work.

Misconceptions #

In the first few years of my career, I didn’t think much about “an approach” to my work. I was trying to stay afloat. My leaders would assign me Jira tickets, and I would try to solve them. Write the code, open a pull request, get a review, and merge it. Sprint by sprint, rinse, and repeat.

Early on, I realized that folks around me were productive and delivered high quality on time. Which was great – I was in good company to learn. I always wrote that off to their experience, talent, intelligence, education, or any combination.

As time progressed, I saw less experienced folks that were productive and produced excellent code. Collaboration with them was a breeze. They had a good understanding of the domain. They asked thoughtful questions. And their code was great, at least by my standards.

As much as these were intelligent folks, it was not raw intelligence that made them efficient. Not that it didn’t help them, but I found out soon enough that they had developed their way of approaching problems. Finding this out was the reflection point for me. I realized that to get to the next level, I need a problem-solving framework that I can reuse in my everyday work.

Problem clarity #

Getting a simple math assignment at school such as “3 - 2 * 5” was simple enough. But can you imagine solving it without knowing what each sign means or their order of precedence? That would be equal to starting to code right out of the gate when presented with a problem. Yes, even if it’s just a bit of prototype code that you will throw away (you hope!).

Understanding the problem that you are solving is as important as the solution itself. By spending time to get clarity, you will save time building the solution. Unfortunately, I have been guilty of doing the opposite early in my career. Jumping to prototyping and making assumptions about the problem is too easy.

If you are a builder at heart like myself, you are excited and impatient to jump on the code. And I appreciate it when I see the enthusiasm to produce an artifact of one’s work. But I soon found out that there’s a way to get both: the dopamine hit from building without having to jump the gun and begin coding.

It’s called sketching.

Sketching to the rescue #

These days, when presented with a problem, I take a simple step: open a blank document. I usually use Dropbox Paper at work and Google docs for personal writing (such as this). I put the problem statement at the top, jotting down everything I know about the problem. Then, I link various supporting resources: code in version control, (pseudo-)code snippets, documentation, examples of similar issues, etc.

In this context, I define “sketching” as explaining the problem that I am about to solve through writing. The goal is to organize my thoughts and understanding before jumping on the solution.

Sketching turns into magic when I write about the problem and not the solution. It’s like my thoughts stare back at me. They force me to read them and assess my reasoning of the problem. So sketching is introspection on how well I understand the problem. And as we discussed earlier, no problem can be solved without first understanding its ins and outs.

Also, I think about the second-order effects of the problem. For example, if I dig into what new sales report the users asked for, I like to think about who else will use the new report beyond the user. The accountants of the users? Or an auditor? I branch off and explore. Exploration allows me to understand the problem from different angles better.

Cross-pollinate #

When I reach a point where I am happy with my sketch of the problem, once again, I get the urge to jump on the solution and code away. The improved clarity of the problem that I now have gives me more confidence (as it should!). But I still think it’s not a good point to start coding. Granted, it wouldn’t be ideal if I jumped on a prototype at this point, but it would be better than before.

With such writing exercises, it’s too easy to go too deep or wide, so it’s a good practice to get calibrated by others. The peer feedback aids with prioritization. It allows me to focus on the right thing within the current time/resource constraints. Some aspects of the problem will have higher priority/impact in the short term. And others, while still relevant, I can address later.

Once I have a draft problem exploration document, the best course of action is to share it with a group of folks. I make sure the individuals in this group are familiar with the problem space. I still don’t know the solution, so I prefer not to talk to folks close to potential solution space. My goal is to share my understanding of the problem and get feedback on it, not to get my problem solved.

Use the hivemind #

Many folks struggle at this point. Exposing one’s (lack of) understanding of a problem can make one feel vulnerable. I have had impostor-like feelings when I am supposed to share my work, which has held me back from sharing many times. So many questions have popped up in my mind, like “what if my boss realizes I know nothing about the problem?” or “what if they realize I am a fraud?”.

I don’t have a pill to get rid of those thoughts, but a perspective that I try to keep: the only way to get rid of my ignorance is by exposing it. And the best audience to do that is my closest allies, the folks I work with every day. They are the hive mind that I use to surface any ignorance on the problem space and allow them to poke holes in it.

My approach has simplified over the years. I say something like, “Hey, I put down some thoughts on the project we are kicking off; I’d appreciate it if you can check it out when you have X minutes”.

Embrace feedback #

I find writing very revealing: my thoughts on “paper” for the world to consume. Thus it’s hard when I get feedback that I don’t like - it is fair, but it stings. When receiving unfavorable feedback, my go-to action was to defend my writing. Still, lately, I’ve tried something different. I try to critique my writing too and dig deeper into others’ feedback.

Defending your work is typical and expected. But my intention in sharing the sketch is to get feedback on what I got wrong in it or find the blind spots that I have. The reason behind sketching the problem’s is to surface my misunderstanding and get rid of it.

Hence, I always try to embrace the feedback on the sketch. Also, I ask “why” back to the reviewers so I can learn more, and use their input to build my critique of the writing further.

Iterate #

For one to decide on an iterative approach, they have to be at peace that their work won’t be great in the first few attempts. It’s like building software. People realized that delivering software as one big chunk is dangerous and expensive. Monolithic deliverables do not provide room and time to fold in feedback. But, the beauty of iterative work is that it shortens the feedback loop and allows feeding feedback back into the development process.

I try to keep the same principle when sketching out problems: it’s better to have a half-baked sketch done now than later. I can share it with others earlier and learn from their feedback. The alternative is to obsess over details and lose the opportunity to improve.

Folks that are earlier in their career tend to overcomplicate their life by trying to get a polished writeup before sharing. It’s an anti-pattern. I prefer to share a quick draft and receive some harsh feedback. That’s better than spending days on a sketch before another pair of eyes reads it.

A good take on this is the one from Facebook’s early Little Red Book:

Fast is better than slow. While slow is adding unnecessary embellishments, fast is out in the world. And that means that fast can learn from experience while slow can only theorize.

Prototype #

In line with the same “fast is better than slow” mantra, I try not to dwell on the problem for too long. Instead, I decide that I understand the problem enough to move on to the prototype. When I begin prototyping, I try to be very cautious because solving the wrong(ly shaped) problem is very costly.

Jay Shah, one of the reviewers of this essay, asked me, “what is a good indicator that you’re ready to prototype?” – a question I loved. Of course, there’s always the risk of not knowing enough to begin prototyping. But there are also tactics to de-risk that. To start prototyping, a two-part indicator I use is:

1. I shared my problem sketch with folks who have a vested interest in the team solving the right problem. Examples can vary based on the organization’s layout and size. It can be one of these folks: tech lead, product manager, engineering manager, and other engineers.
2. I have addressed all blocking feedback from the reviewers. I can address nits or tangential commentary during implementation (or never).

Another tactic that I tend to use if I want to be extra-careful: if my sketch of the prototype is sound, I am ready to begin prototyping.

Sketching the prototype #

At this point, I try to think of the solution space for the first time, so I finally get to scratch that itch. A prototype sketch does not have to be very detailed. I try to keep sketches concise with enough detail to show the nitty-gritty of the implementation. I sketch out only the part where the rubber meets the road in the project.

For example, if I am to implement a new sales report, I will sketch out three aspects of it:

• Sketch the module that generates the report. Settle on a reliminary name, entry interface, dependencies, main algorithm (e.g., loops over each sale, extracts amounts, dumps them in a dictionary, etc.)
• Sketch the integration of the module with the rest of the product, e.g., how and where the integration invokes the new module
• (If a front-end component is involved) Sketch of plumbing between the front-end and the back-end

My perspective is skewed towards a backend-first approach, as that’s where my experience lies. But the principles apply to other areas too.

Request feedback #

I usually build the prototype with other engineers. So this time, getting feedback from them is enough. Sometimes getting a review from a neighboring team is also good. I make sure to include other teams if we end up changing parts of the codebase they own. If the changes I will make in the other team’s code are more extensive, like changing an API, then their feedback is blocking. In such cases, no approval means no further progress.

Once I gather feedback from all parties, I am ready to proceed with prototyping. Sometimes a teammate or someone external leaves blocking feedback that’s not easy to resolve. In such cases, I have found that the best course of action is to get on a call or sit next to each other (remember those times?) and work through it. My experience shows that getting feedback over text can feel distant and inhumane. I prefer to discuss and collaborate so both parties can walk away feeling like winners.

Prototype away #

Once I reach this part, I know my approach is sound, realistic, and cohesive with the codebase I will build in. I continue to prototype my way into the problem. I deal with every aspect of the problem in the current scope (remember: some parts of the problem might be out of scope, as discussed earlier).

I tend to keep my prototypes short but holistic, and I tend not to over-engineer. The least amount of code that I can write to prove that the approach will work – the better. Prototyping meets code-golf.

Once I am confident of the feasibility and the approach, I throw away the code. I can then work with other engineers/the team to build out the first iteration of a well designed, documented, tested, and deployable version of the solution.

Principles #

If I sublimated my whole approach to exploring, sketching, and solving a problem, I would put it as follows:

1. Strive for problem clarity: you can’t produce an excellent solution to a problem you don’t understand well
2. Write, write, write: writing is a thinking tool – use it!
3. **Sketch early and often: **it’s cheaper to calibrate on a one-pager that outlines the perception/thinking. Doing it on the first pull request is more expensive.
4. Use the hivemind: ask for feedback, learn what you don’t know, get comfortable with the uncomfortable and steal ideas
5. Prototype: check the feasibility of the solution and learn from the feedback you get from the codebase
6. Iterate: only Bob Ross gets it perfect on the first try; everyone else takes a lot of shots before they nail it.