Poster
4 types of experiments in software development
PoC or MVP? Are they even different?
Software development is about knowledge. Developers have to learn parts of the domain and write down their findings in a programming language. The most effective way to gain knowledge from the real world is through experimentation. Therefore talented developers utilize several types of experiments in their work.
However, developers work closely with customers, juniors, users, and project managers who are not trained in these experiments and their differences. Therefore it can be difficult to communicate which questions different types of experiments can answer.
Product owner: Great, can you just put this in production?
Developer: No; it’s just a proof of concept.
Project manager: Just do it anyway.
Yet, we must stick to our experiments' format; otherwise, we compromise the outcome, invalidate the experiment, and devalue future experiments of this type.
This post describes the 4 most common types of experiments that developers use: Spikes, proof of concepts, prototypes, and minimal viable products. We pay special attention to how they are different, what their focus should be, and what the pitfalls are. Hopefully, this can help developers have a clear way of communicating about them no matter who the recipient is.
Spike
The first type of experiment is called a spike. Spikes help us determine what we should build. The spike rules are simple: build something quick and dirty, but promise to delete it once the spike is complete. It cannot make its way into the main branch!
We do this to gain knowledge about what we are about to build. Even having something rudimentary running, it is much easier to discuss with the feature requester. Another advantage of a spike is that while building it, we learn about the areas where extra care needs to be taken, and we gain confidence to build it better when we do it for real.
In short, we build a spike to explore our understanding of the task we are about to implement. This is why it is sometimes called a coding proof of concept. Once the spike is complete, we codify our gained knowledge in the form of a specification (preferably an automated test). Then delete the code.
We must delete the code because the benefit of a spike is that we can build it quickly. If we start thinking about making it maintainable, how to make it scale, then we slow down. As the goal is to find out what we should build, there is a high probability that whatever we build is not the right thing. Therefore we should waste as little time on it as possible. We can only do this comfortably if we trust that the code will not be kept afterward. If we start keeping the code from some spikes, developers will have that in the backs of their minds as they make future spikes and thus start building the spikes better, slowing them down, and wasting valuable time.
A spike should be the shortest possible route
from “I think I know what to build and how”
to “I know what to build and how.”
Proof of concept
Next up is a proof of concept (PoC). These experiments are typically larger and aim at determining whether something is possible or not. Do we have sufficient skills to develop X? Is it possible at all? How much is it going to cost? We are validating the feasibility of a feature or product. Thus the outcome of a PoC is a go/no-go decision.
In case the outcome is no-go, we need to discard all the code we have built for the PoC; this is very similar to what we described above about the spike. Therefore we mustn’t waste time making to perfect. In fact, it should only be exactly good enough to make the decision. Like the spike, we should not release a PoC. Instead, we document the decision and the learnings from building it. Armed with this, we are in a much better position to build the production version.
This means there are two paths forward when we have a go-decision. Generally, our product has some external interface like a GUI or an API. In this case, we need prototypes (described below) to test how the product should look and how the user experience should be. Also, knowing that something is possible, we still need to validate that it is useful/valuable. Such a hypothesis is what a minimal viable product (described below) is for. We might even need to run these next steps in parallel.
Prototype
At present, the best method for accessing this is putting it in front of people and recording how they respond. Ideally, we would give them access to the full completed system and get the most accurate feedback. However, often this is not the most cost-beneficial approach since we risk spending a lot of time building something incorrectly. Instead, we can utilize prototypes to guide our process. Prototypes come in two flavors: Low fidelity (lo-fi) and High fidelity (hi-fi), corresponding to whether it is interactive or not.
A lo-fi prototype is typically a collection of slides, pieces of paper or cardboard, or similar. On these pages are wireframes and rough sketches of the positions of different input fields such as buttons. Since this is made on paper, there is no interactivity, so the user is asked to pretend to click the button, after which a person provides a new page roughly outlining the effect of the button press. This can help us gauge what is intuitive and what is challenging and requires adjustment.
Once the rough outline is done, we can start polishing details, such as the concrete look. We now move to a hi-fi prototype where the buttons automatically take you to the next page. The pages are still static; there is no back-end, no business logic, no persistence. While there is no functionality, these provide a very close experience to the final product but are much cheaper to build and maintain.
Having a clear vision of how the interface should look and feel, we can create reusable templates. For once, it makes sense to try to add as many tools as possible into these templates, as use cases will inevitably arise that we have not foreseen. Thus providing templates for each use case we know and interchangeable modules that can be used with all. A good starting point for GUI templates is to make sure they cover all the HTML input types (buttons, checkboxes, …), contain an easy to use color palette, and contain guidelines for handling unforeseen use cases.
Minimal viable product
The final type of experiment we will go over intends to validate whether something is useful (or valuable). This is important because it does not matter how well we build and design something if no one uses it. Again the only method to assess this is to give it to some real users and see whether they use it.
Again we mustn't spend too long going down a dead-end path. Therefore we should build as little as possible that we believe will provide value. This is what the words Minimal Viable Product (MVP) mean. “Viable” because it has enough features to be useful, “minimal” because it has only those features. We should build only enough features to make the users start using it. Once they are, we can start adding more features and fleshing out the system.
Other than building too much, the only pitfall is not releasing it to the real users. Countless times I have seen product owners insist that they can predict what the users will want, but in practice, not only users themselves can predict what they want, so how could anyone else? Build the MVP, pick some real users, train them properly in the intended way to use the product, and see what happens.
Conclusion
When starting something new, we can either start by investigating whether something is possible or how we can build it, depending on how large the experiment is, we either use a spike or a PoC. Common for both is that the outcome is knowledge, not code.
The next step is to validate the business case and start delivering value. We do this with a minimal viable product, where we build just enough to provide value. Concurrently we can also start designing the user experience using prototypes. The outcome of which is templates that we can use to get a consistent look and feel across the system easily.
This post intends to help you to communicate this information to others. Therefore I have also made it into a poster. So if you are looking for something useful to decorate your office walls with, you can pick it up over at Zazzle:
If you are a programmer and want to improve your craft, you can also check out my book about improving code quality:
