Lessons learned from a successful project

As consultants, we are always focussed on the next thing to improve, so we easily to forget to celebrate our successes. We should pay special attention to our achievements. On average 29% of IT projects are delivered successfully (source ). When projects are performed in an agile manner this number rises to 39%. Success is defined as on time, budget and target. For most people in IT, this is a number that has been plaguing our industry for years. At customers, I see people become numb and shrug whenever a project misses a deadline, is over budget, or under-delivered. As a consultant, it’s your responsibility to do everything you can to make your project part of the successful segment. While doing so, we are striving to eliminate every damaging factor. Part of our engineering culture is to always focus on the parts that we can improve. This feels natural to us. However, it’s also important to emphasize the things that are actually successful! Articles and books like the Phoenix Project add to the body of knowledge of things that work, and things that don’t.

In this blog post, I will discuss how factors like our product development process, technology stack, and a great culture fit made our project a success.

The project

For the past 14 months, my team and I were focussed on a green-field-ish project for a company in the entertainment industry (music festivals). I say “green-field-ish” because – as you often see – there already was a complete software solution in place that supported this process. The old software is based on an ERP system which was heavily modified to fit this administrative workflow. The customer decided that this customisation was not sufficient, and the business case justified the development of tailored software.

The size of a project influences the chance of success (according to Standish metrics ). So in order to accurately weigh our results, I will present some facts. The project budget is close to a million euro. Attention varied, but at its busiest we had six engineers working daily on the project creating about 100K lines of code (I know, it’s an imperfect metric). It involved three web clients, a mobile app connected to the Google Cloud, and an ETL sync to a reporting environment in AWS.

We had a carte blanche to pick any technology suitable for the project. Technology-wise we were presented with the challenge to build offline support for web clients. The customer has a support organisation for packed software only, so keeping the operations-side minimal was a desire. The business case lent itself to a Serverless approach, since the workload irregularly varies between no-usage, and peak-usage. This way, costs would directly relate to usage, with the benefit of a low-ops approach.

The project entailed administrative software for the entertainment event industry, where the show always goes on. So there was minimal leeway for overrun.

So was the project a success?

After having supported the larger events throughout the summer, each subsequent event with even more features, we can look back on a fully featured, fully functional product. We passed the pen-test with ‘state-of-the-art’ recommendation. During operation, the biggest service interruptions took only minutes. The pagers kept mostly silent, and on office hours, we could tweak performance and UX and quickly solve the majority of UX and functional issues. On the busiest days of the software, we were able to continuously deploy 10+ times to production without downtime. Everybody involved in the project is happy about the results, and the personal relations are very good.

Success factors

Reflecting on the project, there are a couple of key learnings.

Process agility

The project was executed in-house, where we controlled the complete process. Early on in the project, we started the quoting process by doing a product vision session. In a four-hour workshop we explained our way of working, safe-guarded the vision on the project, did a high-level requirements gathering, did an initial pass over the project non-functionals and identified three prototypes. We storymapped the first sprint releases to agree up on priorities. The prototypes consisted of the two technological hurdles, and one prototype was actually a walking skeleton of the applications to establish a baseline with the customer about our way of working.

We started the project with a small multi-disciplinary team. We did an EventStorming session to get almost ten stakeholders from the customer side aligned and to bootstrap development. The customer had never worked in an agile project, so in parallel, we started onboarding our primary product-owning delegates on our agile way of working.

Personally, I have tried to do the above for all projects for some time now, and with this recipe, I believe you are minimizing the overhead in bootstrapping a project and maximize short-term results and early feedback. After doing this quite rigorous at the start of the project, we adjusted to a by-the-book Scrum way of working.

After six weeks, we presented two technical prototypes and a functional one and re-oriented the project accordingly. UX came in and helped to prototype almost every visual feature for the three web applications and the mobile app. UX validated important click-able prototypes with target users. We aimed to keep UX one sprint ahead of the development team, so stories in refinement were augmented with designs. The (remote) customer was involved in almost all daily standups. Sprint planning/refinement were weekly on location, and sprint review/retrospective was performed bi-weekly. Every feature is demoed in an acceptance environment. After completion and testing, the production environment is updated multiple times a sprint.

The most important thing in our process is the rapid validation of features. UX performed functional/user validation. We released often, and we had really short feedback loops on new features. It happened a lot that a feature was already accepted and heavily used before the sprint closed. This process is neither unique nor special, it’s just rigorous. It kept the team in a fast delivery cadence. With the exception of maybe one or two sprints with a lot of support work – the burn rate was predictable for 30 sprints with increasing velocity and confidence.

The technological stack

We kept things simple(-ish): a Serverless, public cloud backend combined with a SPA frontend (Angular). Our backend was built with Firebase, a combination of a storage solution and a function runtime for the more exotic operations. Having worked in the Java ecosystem, we always had tons of tools, where one could always find the best (or most fun) tool for the job. However, each tool introduces operational complexity, implementation complexity, (re-)learning, documentation, onboarding. By keeping the stack as simple as possible, we kept onboarding easy, which helped us scale our team even during the busiest months.

From a technological perspective, the fixed stack forced us to be more creative at times, but we could accompany every requirement. We noticed that with a smaller toolset you establish common patterns a lot faster, which sped up our work. The use of Serverless components from the Google Cloud meant little time was spent on setting up our environment, or on operations and monitoring. An entire environment from scratch was up and running well within the day.

Prioritisation and culture

The key differentiating factor in this project is the product ownership. The customer had a team of four that had worked with the previous software for years. Let’s call them the product representatives. The product ownership administrative role and project management role were kept in the development team, but roadmapping, prioritisation, release planning, stakeholder management, and reviewing / accepting work were handled by the customer representatives. The product representatives were available each day and actively participated in the daily standup and reviewed work on daily bases. So this was not special, this is pretty straightforward for most projects.

What made the project successful turned out to be the credo of entertainment: the show must go on!

A good agile culture fit

Somebody on the customer-side remarked at the kick-off of this project: “Be aware, we might not be as agile as you hope us to be.” While not unusual for us, practice proved the exact opposite to be the case. The product representatives were very well seasoned in picking the right battles, killing darlings, managing setbacks, stakeholder involvement, and pushing for the right priorities. All fundamental skills that are very hard to teach a product owner.

Prioritisation and skin in the game

The show MUST go on!!! In agile, we mostly work with fixed resources, and fixed time in an effort to create the scarcity which enables us to prioritise features properly, as this is the only variable. This means output is not guaranteed in terms of feature completeness. This was quite a scary thought, because there was a roadmap with fixed events which each required some degree of feature coverage. Many projects are plagued by business politics, with lots of stakeholders that all want to set the agenda. Sprints become diluted and full of compromises.

In this project, the product representatives had skin in the game (book )!!! They are operationally responsible for running the software, so the project success would make their day, or make life hell. In many organisations product owners are a proxy of the stakeholder, but their daily work is not much governed by the outcome of a sprint. In our case, the product representatives themselves and dozens of co-workers are immediately affected. They proved to be able to make a good feedback loop with the right priorities, which is the ultimate influencer in making this project a success.