Agile methods are as diverse in their application as the projects and industries in which they are used. Originally developed in the automotive industry by the Japanese manufacturer Toyota in the form of the Kanban method to increase production efficiency, a diverse landscape of agile methods has developed over many years. Today, these are an integral part of modern software projects. The Scrum framework has become the long-standing industry standard in software development
Variety of Frameworks
In the practical implementation of agile working methods, various frameworks are used, which are also continuously developed. In addition to Scrum, there are many other agile frameworks such as Objectives and Key Results (OKRs). These two frameworks, in particular, place very different emphases. Scrum focuses more on optimizing operational collaboration (e.g., with the role of the Scrum Master, retrospectives, and the daily scrum), whereas OKRs have a strategic focus that centers less on the “how” and more on the “why.” The idea is that combining both frameworks makes teamwork more efficient and measurable within an overall context that keeps the bigger picture in mind.
Agility is not a Framework
Agility is not just a collection of frameworks or a toolbox; it is a mindset born from the realization that complexity is unpredictable and, therefore, not controllable in the long term (see, e.g., Cone of Uncertainty). Within the agile mindset, flexibility, adaptability, and a culture of continuous improvement are promoted. This thinking enables organizations to respond quickly to changes and drive innovative solutions. Frameworks, their artifacts, and roles help practically implement an agile mindset and approach. They provide a possible means to do so but do not claim to be suitable in every situation or organization.
Frameworks provide good guidelines for collaboration and, above all, create conditions that can serve as a starting aid for companies that have not yet worked according to agile principles. They pave the way for an agile mindset and the corresponding principles to develop, including loosening previous structures and entrenched working methods. Spaces and opportunities arise to focus not only on what is being worked on but also on how it is being worked on, allowing teams to learn and continuously improve their collaboration.
Complexity in Developing Data Products
source systems, challenges in data quality, and the availability of internal and external resources. Agile methods help manage this complexity by promoting iterative development, cross-functional teams, and continuous learning. Through the iterative approach, where requirements are implemented step by step and regular user feedback (if it’s a product accessible to the market, user feedback from the market is the ultimate test) is collected, it is also avoided that recorded requirements and user expectations drift too far apart, resulting in a data product that the user does not use at all in the end. This would be fatal because the active use of a data product is ultimately the step crucial for generating added value from a use case.
Where does complexity come from?
| Unavailable and decentralized data | Frequent changes in requirements |
| Lengthy decision-making processes | Unavailability of essential resources |
Actively Managing Complexity
The difficulty in measuring the complexity of a project in advance or expressing it in absolute terms leads to many problems. For instance, assessing complexity largely depends on the project team’s experience, which itself is a factor influencing complexity. The positive consequence is that each project team makes an assessment of complexity that suits them. To actively manage complexity based on this assessment, especially when the scope or requirements are not final, a simple means is to increase the frequency of feedback loops. In the context of the Scrum framework, you would shorten the sprint length (e.g., from four to two weeks) if it is estimated that complexity would otherwise be too high. This way, the fuzziness between the developed product and user expectations is actively managed through additional feedback loops.
User-Centeredness and Design Thinking
Involving users is crucial for the acceptance and success of a data product. By collecting early and regular feedback, developer teams can ensure that the final product meets the actual needs of users and thus provides real added value. This starts with gathering the initial requirements because, to ensure a user uses the data product, it must not only provide added value to the organization but also to the user, ideally making the user’s work easier. This is achieved through a design thinking approach, where a solution is conceived that includes both the users’ problems and pains and the potential added value for the company. Such an approach not only saves resources but also increases the likelihood that the data product will be positively received and used for value creation.
Transparency Builds Trust
Just like the topic of digitization, data in the context of a transformation towards data excellence can also evoke fears among some employees. These fears are detrimental to the acceptance of both the transformation process and the development efforts for individual use cases, including user acceptance. Therefore, it is important to make the process transparent and clearly communicate that there will be changes, including how people work together, but also that the goal is to address employees’ pains (e.g., boring repetitive tasks) within data use cases.
Lessons Learned
Agile methods are an important tool on the path to success. In essence, agile working represents a change in the mindset of how we collaborate. It helps companies become and remain flexible and innovative. The entirety of agile frameworks and methods is so vast that we have focused on prominent examples in this presentation. In all the mentioned frameworks, we have extensive expertise and practical experience in our team. Our team of organizational designers for agile companies, Scrum Masters, Product Owners, and agile coaches accompanies you on the path to transforming into a data-excellent organization and realizing data use cases.
