To Startup Teams: Always Write Tests, Especially for MVPs

Photo by Farzad Nazifi on Unsplash

When your startup is experimenting with an idea, you need to find if people would pay for what you offer. At this stage, what you want is most likely a Minimum Viable Product (MVP) that works, not perfectly, but one that works well enough to be delivered to the market right away to find its product/market fit.

There are two major fates of the MVP:

1. If the product doesn’t have a decent product/market fit, it will be abandoned, which is mostly the case for startups.
2. If the product finds its product/market fit, it will be iterated. New features will be built on top of it. Eventually, your MVP will grow out of its minimum phase and flourish into a sophisticated product.

In either case, contrary to common industrial practice and belief, it is beneficial to write tests for the code base. Before making the argument for writing tests for MVPs, let’s walk through how teams usually build MVPs.

How most MVPs are built

1. Someone had a business idea
2. Someone wanted to see if the market needed this product
3. Someone might come up with a Proof-Of-Concept (POC) internally in the team and decided that it was doable technically (or this didn’t happen at all)
4. Someone might build the prototype for stakeholders, investors, and a small group of potential customers to test the feasibility of the business idea (or this didn’t happen at all)
5. Developer(s) built the “MVP” in a short time and did not plan to maintain the code base
6. The “MVP” was launched to the market

In order to launch the product in the shortest possible time, tests are omitted to save time. It seems reasonable to spend all the development time solely on client-facing features, and everything else unseen by the clients should be done in the fastest way or skipped despite the difficulty of future maintenance or feature addition, since most MVPs would end up being abandoned anyway.

This decision is certainly made after serious consideration of multiple factors and the perceived advantages:

1. To minimize the opportunity cost due to the low success rate of new products, and
2. The impact of the time saved from writing tests on the chance of success

There is only one obvious disadvantage — if the product becomes successful, the hacky codebase would need to be refactored properly, or a completely new version has to be built from scratch without taking shortcuts.

Wrong assumptions lead to wrong decisions

Most teams and companies made the decision to build the initial product in a hacky way consciously. They acknowledged that they were taking on some technical debt in exchange for a shorter delivery time, agreeing that the initial product codebase would either be refactored or discarded in the near future. The decision-making process was logical, and all of it seemed to make perfect sense according to the scenario. Unfortunately, there were too many wrongly-made assumptions, which caused the decision-makers to be biased and not well-informed, and ultimately come up with sub-optimal solutions.

Assumption 1: Writing tests increases development time.

It is true that writing tests would require developers to spend part of their work time on writing tests instead of only on building features, but it does not necessarily amount to a longer total development time than without writing tests, because the tests would make feature development easier and reduce time spent on debugging.

A higher test coverage prevents code changes from breaking the features and ensures the features have the expected behaviour. It gives you confidence in every line of code covered by the tests because developers know which part to fix when the tests throw an error. The benefits of writing tests are especially obvious when Behaviour-Driven Development (BDD) or Test-Driven Development (TDD) is practised.

Before launching your product, you would need to test the user flow vigorously in one way or another to minimize complaints. Write end-to-end tests instead of performing Quality Assurance (QA) manually. End-to-end tests are tests for real-world scenarios your users would go through. Once you have written the tests correctly, you can run them automatically during development. If any feature or function fails to work as expected, you do not have to wait for another person to tell you after a manual check, because you would know immediately by running the end-to-end tests.

As for testing features and functions, there are unit tests. They are tests that pinpoint the root cause of unexpected output or behaviour. Even with very limited resources, you should always dedicate some resources to writing unit tests for crucial features and functions.

With a decent amount of unit tests, the number of integration tests for the MVP can be minimal under a low resource environment. The unavoidable integration tests for startup products are usually tests for database integration.

Overall, writing tests should not take too much time because tests should not be difficult to write. If the right tool is chosen, it should be easy and straightforward even for developers without any experience in testing. If the team slows down because of inexperience, it is still not a reason for omitting tests, because the development team will have to learn how to write tests sooner or later if you are serious about your product’s future.

Writing test is an investment you must make for your product in the long run — you can either hire senior developers who know how to write tests (and more expensive), or junior developers who have the potential to learn (and need more time to be trained to a decent level). You cannot totally skip this investment and still expect your product to be easily maintainable and highly scalable.

Assumption 2: MVPs are likely to fail, so hacking things together to test the waters is acceptable.

It is an undeniable fact that the success rate of MVPs is low. It is also true that you are building an MVP. However, it does not mean that you should anticipate failure or build accordingly with such an expectation.

You set out the journey to chase success, not failure. So why do you create your product and work with the assumption that it would fail? You might argue that you are simply being practical and realistic, so you want to maximize your resources and avoid investing too much into the business idea before it has proven its worth. But if you had not done your best, how would you know it failed because of the business idea itself, and not because of how you executed and materialized your idea?

If you want your clients to think highly of your product, you must be the first to think so. Build your MVP in the way you want your clients to regard it. If you want people to perceive your product as a high-calibre revolutionary product, allocate your resources in the way you expect a would-be high-calibre product deserves to receive under your circumstances.

Assumption 3: Allocating less resources would not decrease your chance of success.

One of the common mistakes many teams made was that they dedicated insufficient amount of resources. Very often, the first thing to cut down from the task list and production pipeline is writing tests because they cannot see its value immediately. Strangely, these teams still expect to get incredibly positive results from their hacky product. For a lucky few, the product became successful even though it was under-resourced. Most of these cases could be attributed to the creativity of the business idea. If no one else has tackled this problem before, or the product provides a better alternative than any existing solution, it would be no surprise for the product to achieve success even with an under-resourced strategy.

Unbeknownst to most starry-eyed onlookers, this mirage of success is extremely dangerous to both the successful team and aspiring entrepreneurs. Rarely would the entrepreneurs think about what their success means to competitors. Such neglect causes pioneers to lose their competitive edge in a short time.

Once these under-resourced innovative products are launched to the market, there would always be other entrepreneurs who get inspired and think they could do a better job than the original team. And they are probably right, because the groundbreaking product was built upon bad technical choices. If the copycats produce their product with the expectation of being successful (since the first team has already tested its product/market fit), and hence a better resource allocation, they might gain more market share with a less fragile product while the pioneers are busy fixing technical issues.

Assumption 4: The MVP will be refactored when it becomes successful.

Firefighting is exhausting. If you ask any development team for their opinion on the hacky products, most would give you a disgusted look, rant about the incompetence of the previous development team (if it was someone else who developed the product), and insisted that they would rather rebuild the whole thing from scratch than refactor it bit by bit.

Technical debts are certainly payable, but the repayment is usurious. The team which promised to take on technical debts is usually not the team ended up maintaining the code base. After all, who would want to put off a bonfire they started? Of course they would have left before they were asked to do something about it.

New developers who took on the responsibility for maintenance and feature addition might eventually get tired of the refactoring work because it is impossible to trust a code base with very low code coverage. Changing a certain function might break multiple seemingly unrelated features. There is no way for the developers to develop quickly in such an error-prone environment. It is demotivating to the developers when most of their code would fail or cause bugs due to some unforeseeable legacy technical debts. In this situation, the most reasonable solution is to demand a total revamp for the product.

If the MVP must require a revamp in order to follow the product roadmap, does it qualify as an MVP? No, definitely not. An MVP is a product that could go a long way. It should only be revamped when it reaches its scalability bottleneck after rounds of product iteration. An initial product which is expected to be rewritten entirely after its launch is a prototype. It is never meant to be launched to the market, but to test out the reception of the business idea, the interaction and experience of the potential product, and to find out what to refine.

Assumption 5: Rebuilding the product soon after launch makes sense.

Rebuilding the product at an early stage rarely makes sense. Below are three common cases of how teams proceed with their unmaintainable product.

Case 1: Fix bugs (or not) and build features on the product; build a new one at the same time

In this case, you need a team with at least one designer, one developer, and one project manager, to build features on top of the current product, and another team to work on the revamp. If you have that much resources from the very beginning, why not build a proper MVP in the first place? Launching a prototype to the market and taking the risk of others copying your business idea do not seem very clever.

Case 2: Fix bugs (or not) and build features on the product; build a new one when there are more resources

This is similar to the first case, yet there are fewer resources and thus the revamp cannot happen right away. The risk of being overtaken by a stronger competitor is higher than in the first case. It is also possible that the resources never increased with time, and at some point the team would face the situation in the third case.

Case 3: Stop fixing bugs or building features on the product at all; start building a new product immediately

If a total revamp is expected from the very beginning, and the product will not get any development resources after launch, then it means the product launched was a prototype. As explained earlier, giving your clients a prototype would bring you more competitions. Your clients might also be dissatisfied with the product because all their requests for improvement and new features are ignored.

While your product has not built a loyal group of customers, it has already shown people what product they would like to use. When your competitor has built a product with similar functions, your clients would not hesitate to try it out and compare the two. Given the unsatisfactory experience they had with you, it would be difficult to win the competition.

In all three cases, the resources put into developing the product were not maximized because the initial code base was planned to be abandoned. All effort put into the first round of product development was wasted if the product was completely rebuilt.

Assumption 6: If the MVP fails, the code base should be abandoned.

To most people, it seems apparent that if the product did not find its product/market fit, the only chance the team has on succeeding is to come up with a new idea and then build the new product from scratch again. This was true perhaps a few years ago. Since then, many people had encountered the same problem — not being able to utilize their previous work — and so nowadays developers would build a component library, then use the components to build the product.

The component library code base is decoupled from business logic. It can be reused as long as the visual design and interactive behaviour for the smallest units of the product (e.g. a dropdown, an autocomplete field, a text field, etc…) do not change much. If the designers and the developers had built a component library together for the MVP, and the MVP failed somehow, the component library could still be reused for building the next product to shorten the production time immensely.

Another possible way to reuse part of the original work is to change the text or the user flow of the product to test out a new business idea. Since most products include some common functionalities like a login system, a shopping cart, and so on, these parts can be kept and modified for another business idea if the code base was built to be maintainable.

Summary

Throughout the whole decision-making process, imagine your product to be crazily successful after launch. Challenge your team with all assumptions made to ensure you are making the right assumptions, and thus the right decisions. Think of how much resources a successful product deserves to receive. A winning team should invest as much as they can and aim for success.

Teams often focus on making the logically best decisions based on what they think, but have overlooked the validity of the ground for their discussion. It is of utmost importance to spend more time on thinking and debating before acting, because the decisions you made on resource allocation would either bring you victory or come back to haunt you.

No matter how much resources you have, writing tests should be on your task list because the benefits of testing could not be replaced by anything else. If you cut down the resources to a point where the MVP produced is not built properly, you are under-investing in the product, which actually counteracts all your effort in making it successful.

Building a new product is a high-risk investment that potentially brings a very high return. You cannot be scared of making the gamble when you are already in the game. Remember — always write tests, especially for MVPs.

Follow me if you like this article! I would share my thoughts on front-end experiments, startups, and management whenever I have time 😎

Thanks to Frankie Yan for showing me how important writing tests is, and how to write tests properly.