This text is a write-up of a chat I gave at MinneBar 2022. As a substitute of studying this, you possibly can additionally watch the recording or view the slides.
The title of this speak is “sustaining software program correctness.” However what precisely do I imply by “correctness”? Let me set the scene with an instance.
Years in the past, when Trello Android adopted RxJava, we additionally adopted a reminiscence leak drawback. Earlier than RxJava, we’d have, say, a button and a click on listener; when that button would go away so would its click on listener. However with RxJava, we now have a button click on stream and a subscription, and that subscription may leak reminiscence.
We may keep away from the leak by unsubscribing from every subscription, however manually managing all these subscriptions was a ache, so I wrote RxLifecycle to deal with that for me. I’ve since disavowed RxLifecycle as a consequence of its quite a few shortcomings, considered one of which was that you simply needed to keep in mind to use it accurately to each subscription:
observable
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.bindToLifecycle() // Overlook this and leak reminiscence!
.subscribe()
Should you put bindToLifecycle() earlier than subscribeOn() and observeOn() it’d fail. Furthermore, for those who outright overlook so as to add bindToLifecycle() it doesn’t work, both!
There have been a whole bunch (maybe hundreds) of subscriptions in our codebase. Did everybody keep in mind so as to add that line of code each time, and in the proper place? No, after all not! Individuals forgot consistently, and whereas code evaluate caught it typically, it didn’t at all times, resulting in reminiscence leaks.
It’s straightforward in charge individuals for messing this up, however if truth be told the design of RxLifecycle itself was at fault. Relying on individuals to “simply do it proper” will finally fail.
Let’s generalize this story.
Suppose you’ve simply created a brand new structure, library, or course of. Over time you discover some points that stem from individuals incorrectly utilizing your creation. If individuals would simply use every part accurately there wouldn’t be any issues, however to your horror everybody continues to make errors and trigger your software program to fail.
That is what I name the correctness dilemma: it’s straightforward to create however laborious to keep up. Getting individuals to align on a code fashion, correctly contribute to an OSS mission, or persistently releasing good builds – all of those processes are straightforward to give you, however errors finally creep in when individuals do not use them accurately.
The core mistake is designing with out conserving human fallibility in thoughts. Anticipating individuals to be good is just not a tenable answer.
Should you pay no consideration to this side of software program design (like I did for a lot of my profession), you’re setting your self up for long run failure. Nonetheless, as soon as I began specializing in this drawback, I found many good (and usually straightforward) options. All it’s important to do is attempt, just a bit bit, and typically you’ll arrange a product that lasts endlessly.
How will we design for correctness?
Human error is an issue in any business, however I feel that within the software program business we have now a singular superpower that lets us sidestep this drawback: we are able to readily flip human processes into software program processes. We are able to take unreliable chores accomplished by individuals and switch them into reliable code, and sooner than anybody else as a result of we’ve bought all of the software program builders.
What will we do with this energy to keep away from human fallibility? We constrain. The important thing thought is that the much less freedom you give, the extra seemingly you’ll preserve correctness. In case you have the liberty to do something, then you’ve got the liberty to make each mistake. Should you’re constrained to solely do the right factor, then you don’t have any alternative however to do the proper factor!
There are all kinds of methods we are able to make use of for correctness, laying on a spectrum between flexibility and rigidity:
Let’s have a look at every technique in flip.
Institutional Data
In any other case referred to as “stuff in your head.”
That is much less of a method and extra of a place to begin. Every thing has to start out someplace, and often that’s within the collective consciousness of you and your teammates.
Ideas are nice! Pondering comes naturally to most individuals and have many benefits:
Ideas are extraordinarily low cost; the going charge has been unaffected by inflation, so it’s nonetheless only a penny for a thought. Brainstorming is predicated on how low cost ideas are; “The place ought to this button go?” you may ask, and also you’ll have fifteen totally different doable places within the span of some minutes.
Ideas are extraordinarily versatile. You possibly can pitch a brand new course of to your crew to check out for every week, see the way it goes, then abandon it if it fails. “Let’s attempt posting a fast standing message every morning”, you may recommend, and when everybody inevitably hates it then you possibly can rapidly give it up every week later.
Institutional information can clarify and summarize code. Would you slightly learn by each line of code, or have somebody focus on its construction and targets? Trello Android may function offline, which suggests writing adjustments to the consumer’s database then syncing these adjustments with the server – I’ve simply now described tens of hundreds of traces of code in a single sentence.
Institutional information can clarify the “why” of issues. By itself, code can solely describe the way it will get issues accomplished, however not why. Any hack you write to resolve an answer in a roundabout method ought to embrace a touch upon why the hack was crucial, lest future generations marvel why you wrote such wacky code. There might need been a collection of experiments that decided that is one of the best answer, although that’s not apparent.
Institutional information can describe human issues. There’s solely a lot you are able to do with code. Your trip coverage can’t be absolutely encoded as a result of staff get to decide on after they take trip, not computer systems!
There’s quite a bit to love about pondering, however relating to correctness, institutional information is the worst. Low cost and versatile doesn’t make for a robust correctness basis:
Institutional information could be misremembered, forgotten, or depart the corporate. I are inclined to overlook most issues I did after only a few months. Coworkers with knowledgeable information can stop anytime they need.
Institutional information is laborious to share. Each new teammate must be taught each little bit of institutional information by another person throughout onboarding. Everytime you give you a brand new thought, it’s important to talk it to each current teammate, too. Scale is unattainable.
Institutional information could be tough to speak. The sport “phone” relies on simply how laborious it’s to cross alongside easy messages. Now think about enjoying phone with some tough technical idea.
Institutional information doesn’t remind individuals to do one thing. Do you want somebody to press a button each week to deploy the newest construct to manufacturing? What if the one who does it… simply forgets? What in the event that they’re on trip and nobody else remembers that somebody has to push the button?
Like I stated, institutional information is nice and essential – it’s the place to begin, and an affordable, versatile method to experiment. However any institutional information that’s frequently used ought to be codified in a roundabout way. Which leads us to…
Documentation
I’m certain that somebody was screaming at their monitor whereas studying the final part being like “Documentation! Duh! That’s the reply!”
Documentation is institutional information that’s written down. That makes it tougher to overlook and simpler to transmit.
Documentation has a lot of some great benefits of institutional information – although not fairly as low cost or versatile, additionally it is in a position to summarize code and describe human issues. Additionally it is a lot simpler to broadcast documentation; you don’t have to sit down down and have a dialog with each one who must study.
There’s additionally a pair bonuses to visible information. Documentation can use footage or video. A great circulate chart or structure abstract is value 1000 phrases – I may spend a bunch of time speaking about how Trello Android’s offline structure works, or you possibly can have a look at the circulate charts on this article. I personally discover that video can click on with me simpler than simply speaking; I believe this is the reason the trendy video essay exists (over written articles).
Documentation may create checklists for advanced processes. We automated a lot of it, however the technique of releasing a brand new model of Trello Android nonetheless concerned many unavoidably handbook steps (e.g. writing launch notes or checking crash stories for brand spanking new points). A great guidelines may help reduce down on human error.
Regardless of documentation’s advantages, there’s a motive this speak was initially titled “documentation is just not sufficient.”
Right here’s a typical scenario we’d run into at work: we’d give you a brand new crew course of or structure, and folks would say “that is nice, however we’ve bought to write down it down so individuals gained’t make errors sooner or later.” We’d take the time to write down some nice documentation… solely to find that errors stored occurring. What offers?
Nicely, it seems there are lots of issues that may come up with documentation:
Documentation could be badly written or misunderstood. A doc can clarify an idea poorly or inaccurately, or the reader may merely misapprehend its that means. There’s additionally no method to double-check that the data was transmitted successfully; speaking to a different particular person permits for clarifying questions, however studying documentation is a one-way transmission.
Documentation could be poorly maintained and go old-fashioned. Maybe your doc was correct when first written, however years later, it’s a web page of lies. Conserving documentation up-to-date is pricey and laborious, for those who even keep in mind to return and replace it.
Documentation could be laborious to search out or just ignored. Even when the doc is ideal, you want to have the ability to discover it! Possibly you recognize it’s someplace on Confluence however who is aware of the place. Even worse, individuals won’t even know they should learn some documentation! “I’m sorry I took down the server, I didn’t know that you simply could not reduce releases at 11PM as a result of I by no means noticed the discharge course of doc.”
Documentation can’t function a reminder. Very like with institutional information, there’s no method for documentation to let you know to do one thing at a sure time. Checklists get you barely nearer, however there’s no assure that an individual will keep in mind to test the guidelines! Trello Android had a launch guidelines, however oftentimes the discharge would roll round and we’d uncover that somebody forgot to test it, and now we are able to’t translate the discharge notes in time.
Documentation is important. Some ideas can solely be documented, not codified (like high-level structure explanations). And in the end, software program improvement is about working with people. People are messy, and solely written language can deal with that messiness. Nonetheless, it’s just one step above institutional information when it comes to correctness.
Affordances
Let’s take a detour into the dictionary.
An affordance is “the standard or property of an object that defines its doable makes use of or makes clear the way it can or ought to be used.”
I used to be first launched to this idea by “The Design of On a regular basis Issues” by Don Norman, which fits into element learning seemingly banal design selections which have enormous impacts on utilization.
A basic instance of fine and dangerous affordances are doorways. Good doorways have an apparent method to open them. Crash bar doorways are a very good instance of that; there’s no universe during which you’d suppose to pull these doorways open.
The other is what is called a Norman door (named after the aforementioned Don Norman). Norman doorways that invite you to do the mistaken factor, for instance by having a deal with that begs to be pulled however, actually, ought to be pushed.
Right here’s why I discover all this attention-grabbing: We are able to use affordances in software program to invisibly information individuals in the direction of correctness in software program. Should you make “doing the proper factor” pure, individuals will simply do it with out even realizing they’re being guided.
Right here’s an instance of an affordant API: in Android, there’s nobody stopping you from opening a connection to a database everytime you need. A dozen builders every doing their very own customized DB transactions could be a nightmare, so as a substitute, on Trello Android we added a “modification” API that will replace the DB on request. The modification API was straightforward – you’d simply say “create a card” and it’d go do it. That’s quite a bit less complicated than opening your personal connection, organising a SQL question, and committing it – thus we by no means needed to fear about anybody doing it manually. Why would you, when utilizing the modification API was there?
What about bettering non-software conditions? One instance that involves thoughts is submitting bug stories. The tougher it’s to file a bug report, the much less seemingly you’re to get one (which, hey, possibly that’s a characteristic for you, however not for me). The teams that put the onus on the filer to determine precisely the place and file a bug tended to not hear essential suggestions, whereas the groups that stated “we settle for all bugs, we’ll filter out what’s not essential” bought a lot of suggestions on a regular basis.
If, for some motive, you possibly can’t make the “proper” method of doing issues any extra affordant, you possibly can as a substitute do the other and make the mistaken method un-affordant (aka laborious and obtuse). Is there an escape hatch API that most individuals shouldn’t use? Disguise it in order that solely those that want it might probably even discover it. Getting too many developer job purposes? Add a easy algorithm filter to the beginning of your interview pipeline.
I consider this idea like how governments can form financial coverage by subsidies and taxes: make what you need individuals to do low cost; make what you don’t need individuals to do costly.
Although not precisely an affordance, I additionally take into account peer strain a associated method to invisibly nudge individuals in the proper route. I don’t suppose I’m alone once I say that the very first thing I do in a codebase is go searching and attempt to copy the native fashion and logic. If somebody asks me so as to add a button that makes a community request, I’m going to search out one other button that does it first, copy and paste, then edit. If there are 50 alternative ways to write down that code, nicely, I hope I discovered the proper one to repeat; if there’s only one, then I’m going to repeat the write technique. Consistency creates a flywheel for itself.
I like affordances as a result of they information individuals with out them being consciously conscious of it. Loads of the correctness methods I’ll focus on later are extra heavy handed and obtrusive; affordances are mild and invisible.
Their essential draw back is that affordances and peer strain can solely information, not prohibit. Usually these methods are helpful if you can’t cease somebody from doing the mistaken factor as a result of the coding language/framework is just too permissive, you must present exceptions for uncommon instances, otherwise you’re coping with human processes (and something can go off the rails there).
Software program Checks
Software program checks are when code can test itself for correctness.
Should you’re something like me, you’ve simply began skimming this part since you suppose I’m gonna be speaking about unit checks. Nicely… okay, sure, I’m, however software program checks are a lot extra than unit checks. Unit checks are only one type of a software program test, however there are lots of others, such because the compiler checking grammar.
What pursuits me right here is the timing of every software program test. These checks can occur as early as if you’re writing code to as late if you’re working the app.
The sooner you will get suggestions, the higher. Quick suggestions creates a good loop – you overlook a semicolon, the IDE warns you, you repair it earlier than even compiling. In contrast, sluggish suggestions is painful – you’ve simply launched the newest model of your app and oops, it’s crashing for 25% of customers, it’ll be at the least a day earlier than you possibly can roll out a repair, and also you’ll must undo some structure selections alongside the best way.
Let’s have a look at the timing of software program checks, from slowest to quickest:
The slowest software program test is a runtime test, whereby you test for correctness as this system is working. Accumulating analytics/crash information out of your software program because it runs is nice for locating issues. For instance, in OkHttp, every Name can solely be used as soon as; attempt to reuse it and also you get an exception. This test is unattainable to make earlier than working the software program.
There are huge drawbacks to runtime checks: your customers find yourself being your testers (which gained’t make them blissful) and there’s a protracted turnaround from discovering an issue to deploying a repair (which additionally gained’t make your customers blissful). It’s additionally an inconsistent method to check your code – there is perhaps a bug on a code path that’s solely accessed as soon as a month, making the suggestions loop even slower. Runtime checks are value embracing as a final resort, however counting on them alone is poor apply.
The subsequent slowest software program test is a handbook check, the place you manually execute code that runs a test. These could be unit checks, integration checks, regression checks, and many others. There could be a whole lot of worth in writing these checks, however it’s important to foster a tradition for testing (because it takes time & effort to write down and confirm the correctness of checks). I feel it’s value investing in these kinds of checks; in the long term, good checks not solely prevent effort but in addition pressure you to architect your code in (what I take into account) a usually superior method.
One step up from handbook checks are automated checks, that are simply handbook checks that run routinely. The core drawback with handbook checks is that it requires somebody to recollect to run them. Why not make a pc keep in mind to do it as a substitute? Bonus factors if failed checks stop one thing dangerous from taking place (e.g. blocking code merges that break the construct).
Subsequent up are compile time checks, whereby the compilation step checks for errors. Usually that is concerning the compiler imposing its personal guidelines, similar to static kind security, however you possibly can combine a lot extra into this step. You possibly can have checks for code fashion, linting, protection, and even run some automated checks throughout compilation.
Lastly, the quickest suggestions is given at design time, the place your editor itself tells you that you simply made a mistake while you’re writing code. As a substitute of discovering out you mis-named a variable throughout compilation, the editor can immediately let you know that there’s a typo. Or if you’re writing an article, the spellchecker can discover errors earlier than you publish the article on-line. Very like compile time checks, whereas these are usually about grammatical errors, you possibly can typically insert your personal design time fashion/lint/and many others. checks.
Whereas quick suggestions is best, the sooner timings are inclined to constrain what you possibly can check. Design-time checks can solely particular bits of logic, whereas runtime checks can cowl principally something your software program can do. In my expertise, whereas it’s simpler to implement runtime checks, it’s usually value placing in a bit of additional effort to make these checks go sooner (and be run extra persistently).
Constraints
Constraints make it in order that the one path is the right one, such that it’s unattainable to do the mistaken factor. Let’s have a look at a number of instances:
Enums vs. strings. Should you can constrain to only a few choices (as a substitute of any string) it makes your life simpler. For instance, individuals are usually tempted to make use of stringly-typing when deciphering information from server APIs (e.g. “card”, “board”, “listing”). However strings could be something, together with information that your software program is just not in a position to deal with. By utilizing an enum as a substitute (CARD, BOARD, LIST) you possibly can constrain the remainder of your utility to only the legitimate choices.
Stateless capabilities vs. stateful courses. Something with state runs the chance of ending up in a foul state, the place two variables are in stark disagreement with one another. Should you can execute the identical logic in a self-contained, stateless operate, there’s no threat that some long-lived variables can find yourself out of alignment with one another.
Pull requests vs. merging to essential. Should you let anybody merge code to essential, then you definitely’ll find yourself with failing checks and damaged builds. By requiring individuals to undergo a pull request – thus permitting steady integration to run – you possibly can pressure higher habits in your codebase.
Not solely can constraints assure correctness, in addition they restrict the logical headspace you must wrap your thoughts round a subject. As a substitute of needing to contemplate each string, you possibly can take into account a restricted variety of enums. In the identical vein, it additionally limits the variety of checks you must cowl your logic.
Automation
If you automate, a pc does every part for you. This is sort of a constraint however higher as a result of individuals don’t even must do something. You solely have to write down the automation as soon as, then the computer systems will take over doing all your busywork.
One efficient use of this technique is code technology. A basic instance are Java POJOs, which don’t include an equals(), hashCode(), or toString() implementations. Within the outdated days, you used to must generate these by hand; these implementations would rapidly go stale as you modified the POJO’s fields. Now, we have now libraries like AutoValue (which generate implementations primarily based on annotations) or languages like Kotlin (which generate implementations as a language characteristic).
Steady integration is one other nice automation technique. Having bother remembering to run all of your checks earlier than merging new code? Simply get CI to pressure you to do it by not permitting a merge till you cross all of the checks. You possibly can even have CI do computerized deployments, such that you simply barely must do something after merging code earlier than releasing it.
There are two essential drawbacks of automation. The primary is that it’s costly to write down and preserve, so it’s important to test that the payoff is value the price. The second drawback is that automation can do the mistaken factor again and again, so it’s important to watch out to test that you simply applied the automation accurately within the first place.
Now that we’ve reviewed the methods, permit me to display how we use them in the actual world.
Earlier than fixing any given drawback, it is best to take a step again and determine which of those methods to use (if any) earlier than committing to an answer. You’ll most likely find yourself with a mixture of methods, not only one. For instance, it’s hardly ever the case that you could simply implement constraints or automation with out additionally documenting what you probably did.
There are a number of meta-considerations to bear in mind as nicely:
First, whereas inflexible options (like constraints or automation) are higher for correctness, they’re worse for flexibility. They’re costly to vary after implementation and unforgiving of exceptions. Thus, you must stability correctness and suppleness for every scenario. Generally, I development in the direction of early flexibility, then transferring in the direction of correctness as crucial.
Second, you may implement correctness badly. You possibly can have flakey software program checks, overbearing code contribution processes, tough automation upkeep, or no escape hatches for brand spanking new options or exceptions. Correctness is an funding, and you must ensure you can afford to speculate and preserve.
Final, you want buy-in out of your teammates. I are inclined to make the error of pondering that as a result of I like an answer that everybody else may also prefer it, however that’s undoubtedly not at all times the case. Should you get settlement from others, correctness is simpler to implement (particularly for crew processes); individuals will go together with your plans, and even pitch in concepts to enhance it.
Disagreements, however, can result in toxicity, similar to individuals ignoring or purposefully undermining your creation. At my first job they tried to implement a code fashion checker that prevented merges, however did not have a plan for repair outdated recordsdata. There was no computerized formatter (as a result of it was a customized markup language), so nobody ever wished to repair the large recordsdata; as a substitute everybody simply stored utilizing a workaround to keep away from the code fashion checker! Whoops!
Taking a while to collect proof then presenting the case to your coworkers could make a world of distinction.
Now, let’s have a look at a number of examples and analyze them…
Code Type
For instance, how do you get everybody to persistently use areas over tabs?
❌ Institutional information – Dangerous; this doesn’t stop individuals from going off the code fashion in any respect.
❌ Documentation – Simply as dangerous as institutional information, however written down.
✅ Affordances – Semi-effective. You possibly can configure your editor to at all times use areas as a substitute of tabs. Even higher, some IDEs allow you to test a code fashion definition into supply management so everyone seems to be on the identical web page style-wise. Nonetheless, when it comes to correctness, it guides however doesn’t prohibit.
✅ Software program checks – Utilizing lint or code fashion checkers to confirm code fashion is a good use of CPU cycles. Individuals can’t merge code that goes off fashion with this in place.
❌ Constraints – Not likely doable from what I can inform. I’m unsure the way you’d implement this – ship everybody keyboards with out the tab key?
❌ Automation – You might have some hook routinely rewrite tabs to areas, however actually this provides me the heebie jeebies a bit!
In the long run, I like imposing your fashion with software program checks, however making it simpler to keep away from failures with affordances.
Code Contribution to an OSS Undertaking
How do individuals contribute code to an open supply codebase? Should you’ve bought a selected course of (like code opinions, working checks, deploying) how do you guarantee these occur when a random particular person donates code?
❌ Institutional information – Inconceivable for strangers.
✅ Documentation – Should you write stable directions, you possibly can create a extra welcoming surroundings for anybody to contribute code. Nonetheless, documentation alone is not going to lead to a dependable course of, as a result of not everybody reads the handbook.
✅ Affordances – There’s lots you are able to do right here, like templates for explaining your code contribution, or giving individuals clear buttons for various contributor actions (like signing the contributor license settlement).
✅ Software program checks – Having loads of software program checks in place makes it a lot simpler for individuals to contribute code that doesn’t break the present mission.
✅ Constraints – Repository hosts allow you to put all kinds of good constraints on code contribution: stop merging on to essential, require code opinions, require contributor licenses, require CI to cross earlier than merging.
✅ Automation – CI is important as a result of it feeds info into the constraints you’ve arrange.
For this, I exploit a mixture of all totally different methods to attempt to get individuals to do the proper factor.
Cleansing Streams
Let’s revisit the story from the start of this text – clear up sources in reactive streams of knowledge (particularly with RxJava).
❌ Institutional information – You possibly can educate individuals to wash up streams, however they’ll overlook.
❌ Documentation – No extra appropriate than institutional information, simply simpler to unfold the data.
✅ Affordances – We used an RxJava software referred to as CompositeDisposable to wash up a bunch of streams without delay. AutoDispose provides simpler methods to wash up streams routinely as nicely. Nonetheless, all these options nonetheless require remembering to make use of them.
✅ Software program checks – We added RxLint to confirm that we truly deal with the returned stream subscription. Nonetheless, this doesn’t assure you keep away from a leak, simply that you simply made an try to keep away from it. Should you’re utilizing AutoDispose, it offers a lint test to verify it’s getting used.
✅ Constraints – I’m fairly excited by Kotlin coroutines’ scopes right here. As a substitute of placing the onus on the developer to recollect to wash up, a coroutine scope requires that you simply outline the lifespan of the coroutine.
❌ Automation – Understanding when a stream of knowledge is now not wanted is one thing solely people can decide.
What technique you employ right here is determined by the library. The very best answer IMO are constraints, the place the library itself forces you to keep away from leaks. Should you’re utilizing a library that may’t implement it (like RxJava), then affordances and software program checks are the best way to go.
Clearly, not each choice is obtainable to each drawback – you possibly can’t automate your method out of all software program improvement! Nonetheless, at its core, the much less individuals must make selections, the higher for correctness. Free individuals’s minds up for what actually issues – growing software program, slightly than wrestling with avoidable errors.