In today's episode, Parag0n asks a question on Reddit about architecting a large software system for the first time. (Original question on Reddit: https://www.reddit.com/r/cscareerquestions/comments/4nt759/expected_to_plan_and_architect_a_project_and_im/)
In today's episode, Parag0n asks a question about architecting a large software system for the first time.
Mentioned or relevant to today's episode:
Please take a moment and subscribe and review the show! Click here to review Developer Tea in iTunes.
Hey, everyone, welcome to Developer Tea. My name is Jonathan Cutrell, and in today's episode, I'm answering a question that I found on Reddit about architecting projects. I've mentioned this before, but I am a user on Reddit. And in fact, I haven't hidden my name. I don't have a special username. My username is actually my full name, Jonathan Cutrell. So anybody who's on there who knows about the show has seen me on there, but it doesn't seem like a lot of my listeners are actually on Reddit. But if you are, then look out for me in the comments section. I am fairly active, not every day, but I'm on Reddit. And I do like to kind of go around and offer advice when I see it applicable. And I found this question, and I think it's a really interesting question that a lot of junior developers, maybe intermediate developers, are actually facing pretty regularly. And the question comes from a user with username Parag0n, the O in Parag0n as a zero. And the question is simple. He says, I was recently hired as a contractor to work on a long-term project with a fairly large and well-known finance company. The team consists of myself, and one of the contractor and a senior level developer that is not entirely devoted to the project as his plate is already pretty full as it is. The project manager has made it pretty clear that he expects us the contractors to take the initiative in planning and eventually implementing the entire thing from start to finish. The problem is that I have next to no experience with planning a large-scale enterprise-level project like this. Before now, I've been a pretty head-down type developer who chimes in every now and then when I feel like my input is needed. Other than that, I just enjoy coding and don't really have an appetite for the business side of things. I guess my question is, where do I even start? I'm a serious sufferer of analysis paralysis, and my partner, unfortunately, is even less experienced than me and is very passive. Any advice you guys give me about where to start and how to show my lead that I am making progress would be greatly appreciated. Parag0n, this is an incredibly interesting question. I wish I knew your real name. Maybe I can go to your profile and find out. But this question is interesting to me for a lot of reasons. Number one, you have some interesting personal dynamics on your team with the slightly disconnected senior developer and the project manager who is leaning on the junior developers as maybe a test of your tenacity and your will, power, and maybe even some of your skills. I want to first of all encourage you and say that plenty of people have been in your position before and have succeeded. And when I say succeeded, I mean that they have either continued in their job or they've moved up. They've been promoted in their job because they did a good job at this particular type of problem, architecting a very large-scale system. However, I want to give you three pieces of advice today that hopefully will help you in your path to architecting this larger system. And the first piece of advice that I have for you is don't worry about failure. I know that sounds kind of antithetical. It sounds like your analysis brain is probably kind of rejecting that sentiment of not worrying about failure. That's the number one thing to worry about, right? But as I've talked about on the show in the past, failure is how you learn and the most important thing that you can do with failure is take it in stride and continue. Because here's the reality. For anybody who is in a similar position, make sure you understand this. There's not somebody behind the curtain that has the correct answer. This isn't a coding interview question, right? They don't have the correct answer that you're trying to approximate. This is a real-life project. And real-life projects require some discovery. And if you're in the position and someone is giving you something, they're trusting you with this, then they believe in you. They at least believe in you enough to entrust you with a project. So you have to start believing in yourself as well, right? You have to at least match their confidence level in yourself. And what you have to understand is that confidence means being okay with failure because you realize that failure is a normal part of the learning process. If your company is so strict on you that in your first large-scale project, they expect you to have a 0% failure rate throughout the project. Then you probably are going to end up leaving that company or they're going to end up firing you anyway, right? Even if it isn't today, even if it's not on this project, you're a human and the other Developer That you're working with, even the senior developer, I would say go and talk to that person because they probably have some insight into this as well. The amount of failure tolerance that is at your particular company, but it's very common to fail in these large-scale projects in something or another, right? And you take that failure and you learn from it and you apply what you learned as you move forward. So that's the first piece of advice is don't be afraid of failure as you go through the process of architecting your first large project because it is debilitating and unproductive to spend your energy being afraid of something that is likely. Let's think about that for a second. It's unproductive to be afraid of something that is likely to happen. Instead, use that same energy to prepare for the failure, plan for the failure, plan it into your process, tell the person that is doing that project management, have a discussion about what your procedure is if and when failure does occur. And that leads me to my second recommendation for you today and that is to talk early and often. Really, this is talking about discussing the problem set with the other developers, especially the other developers, but also checking in with your project manager, making sure that they know the plan as you move through the development process. You don't need to view this as a, as a, some kind of like mission that they're sending you on where you have to go it alone. That's not the way you need to view this project. Have regular check-in meetings with the people that you're working with and with the people who have stake in this project. They're called stakeholders for a reason, right? If you don't have these check-ins, then you may go in the wrong direction and keep going in that wrong direction for a long time. And you invest a lot of time and money and ultimately, you're completely in the wrong place. If you consider that if you are meeting with these people regularly, giving them status updates, explaining your plan, explaining the architecture that you have in mind, and they are approving those ideas, if you have that in place, then it's very unlikely that you're going to be blindsided by something in the future, right? It's very unlikely that if you continue through these status updates with the project manager, with the other developers, that all of a sudden, you're going to be blindsided by something unexpected. So instead of trying to work in a vacuum, instead of trying to go through that process alone and trying to guess what the next step is, trying to make the right decisions well down the line, instead of doing that, go back and review. Go back and review with your project manager, go back and review with your other developers, do consistent code reviews, and that way, you aren't going way off on a path that you don't need to be on. This is how projects go awry. And when you are planning your project, make sure you understand that some of these meetings are going to be a little bit of overhead for you. Don't plan that the timeline has no space for this collaboration time. Don't plan a timeline that has no space for these check-ins, for these reviews, for revising your direction. Instead, plan shorter increments that you can look back at and review them with the people who you're working with. It's a very simple process. Talk early and often. Ask a lot of questions early and often. This is going to help you stay on the right path and you can make minor course adjustments as you go rather than going way off the path and having to make major course adjustments late in the game. My third piece of advice for you, Parag0n, is to view this project rather than viewing it as one huge project, view it as a bunch of small micro services, if you want to call them that, view it as a bunch of small applications or a bunch of small pieces of code that are all tied together by an integration application. So think about every small problem that you can identify in the application as its own thing. So this means creating small applications, testing them on their own, testing them in such a way that they are solid on their own. They could be deployed onto their own server even if the application is sufficiently large then that may be possible. And the reason that I'm telling you to think this way is because our brains honestly can't think any other way. Our brains can only comprehend so much at one point in time. So the best way to work with large projects like this is to break them down into their component parts. Even if you do build what has been called recently a monolith, even if you don't have actual true micro services that have inputs and outputs and they're discrete from one another, you're still going to think about the code in separate portions. You're not going to be able to think about a large enough architecture all at once. You just simply don't have the working memory to hold all of that in your mind. So you should validate that limitation by allowing yourself to only think about a discrete part of the application at a time. So for example, you should be thinking about the login portion of the application as its own thing, its own piece of the puzzle. So maybe you have a account management service where a user creates an account and they log in and that's the only thing that that service provides. And this methodology has deep roots in coding concepts that we've been using for years and years. The single responsibility principle has been strongly proven over and over again to be a valuable methodology for creating code. So if you take that to its extreme level, then you can start to see how these individual services can kind of play well together. Now, the important part here is that you don't ignore the importance of the cohesiveness or the cohesion provided by that parent integration application, the overarching application. It's incredibly important that each of these pieces of your application are somewhat congruent with one another. That for example, the user experience is not really horrible in one and really refined in another. It's important to keep these things congruent and have the same quality standards for each of these individual services inside of the application. And a lot of the time that comes down to the interface, the actual integration which ends up being that front end piece, whether you're writing an iPhone application or a web application, a lot of the time, these individual services are brought together to be shown on the front end. And so make sure that your front end, whatever that is, whatever that integration portion is, most likely some kind of front end code, make sure that that maintains a level of continuity, a level of congruence with the other portions of the application. As a user moves throughout the application, I can tell you from my own experience that I have seen this fall apart in financial applications that I've personally used online, some of the financial providers that I use, their online applications, even though they are functionally accurate, even though they work decently well, they're not congruent across each of their microservices. I can tell what they've spent a lot of time on and what they haven't spent much time on making those user experiences in particular congruent with one another. So number three, once again, was break every problem down to its smallest piece and create some kind of integration amongst each of those services. But with the caveat that you need to make sure that when you do that integration, you maintain continuity, you maintain congruence with those different services inside of the application. Now, I know I said that I only had three pieces of advice, but I do have a fourth kind of a bonus piece of advice. And that is to rely on metaphors. I know that planning something very large can be a little bit daunting, but one of the best places that you can start to conceptualize a large concept, one of the places that you can kind of begin to see the shape of a problem is when you start wrapping that problem around things that you already are familiar with. In other words, seeing them through the lens of a metaphor. Now, I'm not gonna spend the time and go into detail today because the episode is already running a little bit long, but we do have another episode about metaphors specifically that I'll link to in the show notes at spec.fm. Thank you so much for listening to Developer Tea. I hope this Reddit user actually goes and listens to this episode. Of course, I will include a link to the original Reddit question in the show notes at spec.fm. Thank you so much for listening. If you're enjoying Developer Tea, make sure you subscribe in whatever podcasting app you use. It takes just a few seconds, and it'll make sure that you don't miss out on future episodes of Developer Tea. Thank you so much for listening, and until next time, enjoy your tea.