At the end of the movie The Candidate, Robert Redford is sitting in a hotel room surrounded by cheering staffers after his character has won the election for the US Senate. Looking a little perplexed and forlorn, he turns to his advisor and asks, “What do we do now?”
Many executives who have led their businesses through successful agile programs can probably relate to Redford’s character. They have overseen sizable improvements in software product development thanks to agile; our Developer Velocity research shows that adoption of agile practices at the team level can be one of the most critical dimensions for companies that are in the early part of their journey.
But many of these businesses have run into a ceiling where incremental gains are minimal. The same Developer Velocity research, for example, showed that while third-quartile companies in terms of overall software-development performance scored 41 percent higher on agile practices than fourth-quartile companies, the differences between companies in the first and second quartiles dropped to less than 20 percent. In other words, once a business hits a certain level of excellence, improvements to how teams work in agile alone drive diminishing returns.
For companies that have realized many of the initial gains from adopting agile, there are valuable lessons to be learned from how tech companies develop products. The industry’s intense competition and pace of change have forced tech companies to develop a set of capabilities that take the fullest advantage of agile, of which the following are the most important:
- grounding every decision on customer value through world-class product management and experience design and adopting an operating model built on products and platforms
- creating a software-engineering culture that nurtures and celebrates technical craftmanship, empowers teams, and provides them with high levels of psychological safety in addition to supporting developers with automation and world-class tools
- embedding data and analytics at every level of product development
While these concepts might not be new, the difference between many tech companies and their peers in other sectors is the degree to which these capabilities have been embedded in how they deliver products and services. In our experience, many companies keep product-management and design capabilities on their organization’s periphery, and/or they create boundaries around the roles. One North American insurer, for example, limits the work of its professional experience designers to digital channels only. This makes it virtually impossible to put in place an engineering culture of hands-on problem solving focused on continually improving customer issues.
In addition, we see companies that do not have a deep understanding of the core skills needed. It is not uncommon, for example, for a company to insist that it is customer oriented or has a strong analytics function only to find out that, under the surface, it lacks the requisite skills and processes to deliver on those capabilities.
And, finally, many companies “cherry-pick” a subset of those capabilities to focus on, failing to understand that they are mutually reinforcing. Companies may have strong customer-oriented capabilities, for example, but lack the engineering culture and talent to build products or services for those customers. Our Developer Velocity research shows that product, engineering, and design work together at top companies; for example, companies in the top quartile of one capability tend to be in the top quartile in others. They also reinforce each other: impact from being good at all capabilities is greater than being amazing in one and subpar in another.
Until companies address these issues, they are likely to languish on an “agile plateau,” improving their methodology but not their business outcomes.
Would you like to learn more about McKinsey Digital?
Professionalize and scale capabilities that focus on the customer
Professionalize product management
World-class product management has a broad remit: understand user and business needs and where the market is heading, and then translate that understanding into actionable strategies, road maps, and team backlogs. For this reason, therefore, product management in leading technology organizations is a highly professional function staffed with individuals who have deep technical skills, business acumen, and fluency in design-driven processes built around the end customer. In fact, this unique combination of technology and business skills is increasingly vital to businesses in the tech sector, putting some product managers on the path to the CEO position, Satya Nadella and Sundar Pichai being two of the most well known.
This reality is news to many companies, which often regard product management as the responsibility of a senior product owner. For this reason, they often pick either a project manager or someone from the business without previous product-management experience to fill this role, with little training or support. (For more, see the sidebar, “Product-management roles and responsibilities.”)
Given the importance of the role, we recommend that companies put in place a “professionalized” product-management function. That includes creating roles and levels, corresponding pay grades, certifications, and a career path that provides product managers (PMs) with expanding responsibilities, such as from managing a single feature of a product to managing a complete portfolio of products. Top companies also actively source top talent externally as well as internally, since people in R&D and development who have an interest in business make good candidates for product-management roles. One B2B service provider, for example, had an ineffective product-management function that was mostly staffed by former business analysts or part-time business leaders. It decided to professionalize product management and created a career path, including competitive pay and career-growth opportunities, that was coveted by internal high-potential employees as well as external talent. With a stronger product-management team in place, the business experienced a dramatic increase in the ROI of its digital products and services.
An additional set of key enablers is required to make PMs successful. This includes a well-defined product-development life cycle that is anchored by a set of signature artifacts, such as the PR FAQ; consistent planning and prioritization processes at multiple levels—annual, quarterly, and sprints; tools to support project management, such as planning, product analytics, and so on; and outcome-based goal setting and governance models, such as using objectives and key results. Additionally, PMs should partner closely with a set of critical roles such as engineering managers, technical program managers, and product marketing managers to jointly deliver the business outcomes. A talent or capacity gap in any of these roles often results in PMs filling the “whitespace” and could detract from their fulfilment of other more-core responsibilities.
Embed experience design into every aspect of product development
Experience design is a specific capability focused on understanding user preferences and usage patterns and creating experiences that delight them. The value of experience design is well established, with organizations that have invested in design exceeding industry peers by as much as 5 percent per year in growth of shareholder return.
What differentiates best-in-class organizations is that they embed design in every aspect of the product or service development. As a core part of the agile team, experience designers participate in development processes by, for example, driving dedicated design sprints and ensuring that core product artifacts, such as personas and customer journeys, are created and used throughout product development. This commitment leads to greater adoption of the products or services created, simpler applications and experiences, and a substantial reduction of low-value features.
As an example, a group life insurer decided to bring design experts onto a team working on connecting to its customers’ systems to accelerate onboarding. Despite the technical nature of the problem, they conducted user research and realized that poor system connectivity when onboarding clients was just one of many related problems leading to a poor user experience. Rather than approaching it as a technical issue, the team focused on addressing the full onboarding journey, including workflow, connectivity, and user communications. The results were impressive. The team created a market-leading experience that enabled their first multimillion-dollar sale only four months after it was launched and continued to accelerate sales and increase customer satisfaction.
The increasing commitment to design is becoming more institutionalized at many organizations and a core part of their brand positioning.
Developer Velocity: How software excellence fuels business performance
Create a software-engineering culture
Anchor DevSecOps in developer experience
DevSecOps teams are development, security, and operations professionals working together to release resilient software rapidly. They typically use advanced tools and the latest agile methodologies, such as deploying highly automated continuous integration/continuous delivery (CI/CD) pipelines, managing infrastructure as code, and implementing consistent tool sets. While many organizations have invested in DevSecOps capabilities, few have established the operational infrastructure of interconnected tools to support collaborative and cohesive project management, development, testing, deployment, monitoring, compliance, and security.
Best-in-class organizations develop not just DevSecOps tools but also the processes that allow those tools to work well together. Their goal is to remove friction from the day-to-day life of a developer rather than simply to optimize for cost or efficiency. Additionally, top tech organizations take a strategic view of where to standardize, thoughtfully balancing areas that are critical to cross-team collaboration, such as planning tools and deployment pipelines, with those where developers and teams need autonomy to choose tools or cloud services. Amazon, for example, offers its development teams a well-established CI/CD infrastructure that enables code reuse and increases delivery velocity.
One regional bank is supporting its DevSecOps teams by both standardizing and automating processes wherever it can in order to relieve the burden on developers. It is standardizing all development, for example, into a single CI/CD pipeline that uses a range of automated information security and compliance tests. In addition, it is adding the provisioning of certified infrastructure into the pipeline, which allows it to dramatically accelerate deployment cycles without compromising security.
Build a culture of empowered engineers
Leading digital innovators have a culture that celebrates developer skills and development craftmanship by giving engineers the freedom to pursue their craft. One way to support this kind of engineer-first culture is to adopt open- and inner-source practices, the biggest drivers of additional value in our Developer Velocity research.
Building an open-source culture is about more than using open-source software (OSS) within the code; it extends to encouraging contribution and participation in the open-source community as well as adopting a similar approach to sharing code internally (“inner sourcing”). These companies maintain a strategic approach to developing software, building up their core advantages and complementing them with open-source technologies or cloud services. They also proactively use OSS-focused security-management practices and automated tools that can scan open-source components and remediate vulnerabilities before deployment.
In addition, our research points to psychological safety (providing a supportive work environment) as a critical aspect of successful engineering cultures. Institutions with a high degree of psychological safety react faster, since employees feel comfortable raising concerns sooner, and learn more quickly, thanks to practices such as blameless postmortems. Furthermore, reputation travels fast across the engineering community, so psychologically safe organizations are more successful at attracting and retaining better talent.
This commitment to an engineer-first culture is also evident in leading companies’ relentless pursuit of top engineers, who could be multiple times more productive than their junior peers. Not only do they inject an engineer-first culture into the organization through habits and practices; they are also magnets for other technical talent. Top companies put in practices to harden this culture and retain top talent by establishing a career path that allows engineers to be promoted while continuing to be hands-on practitioners.
A large US bank made a commitment to building an engineering culture by deciding to adopt open source in parallel with its cloud-migration efforts. It looked to accelerate both usage and contribution, and it became very public about its participation in the OSS community in order to reach the right talent. This shift also relieved the concerns among many engineers that working on the company’s legacy and proprietary stack would not build transferable long-term skills. Today, the bank is seen as a leader in digital, with more than 80 percent of its customer interactions occurring through digital channels.
Embed data and analytics
The power of data and analytics to drive business value is widely accepted, but that value is often isolated in pockets of the company or relegated to a set of discrete use cases. Leading software innovators take a different approach to embed data and analytics everywhere they can across the development process.
They embed analytics into the business logic, often enabled by leveraging platform-as-a-service (PaaS) technologies such as Salesforce Einstein or TensorFlow. One leading retailer, for example, embedded machine learning (ML) into its personalization engine, which improved product recommendations for customers and increased e-commerce purchases by 40 percent.
These leading companies also build AI/ML capabilities into applications, including providing real-time and predictive insights, embedding predictive insights, and leveraging conversational AI. They also build advanced instrumentation into the product so that software teams can better track how customers use the product and continuously improve it. We also see these companies using analytics-driven automation to improve operations to, for example, detect abnormalities and address them directly in production (“self-healing”).
This central role of data and analytics doesn’t happen in a vacuum, of course. Leading tech companies make data scientists and data engineers integral parts of agile product teams and invest in a robust shared data-and-analytics infrastructure. Perhaps the biggest difference between top tech companies and their peers in other sectors, however, is the mindset. At most businesses, development teams tend to focus on the functional aspects of their systems—Can the system complete the activities it is tasked with completing?—with data and analytics an afterthought. Tech innovators, however, see data as ground zero for competitive advantage. They rely on data and analytics to drive decisions and educate the entire business on their power.
This shift in appreciating the importance of data and analytics also requires a momentous change of mindset. For example, a B2B e-commerce client has defined data as “the blood of the organization.” The data infrastructure is treated as a first-class citizen and receives consistent investment. At the same time, the company actively works to minimize “data debt.” It also routinely embeds data scientists in agile-delivery teams and provides those teams with rich analytical tools. These practices give these teams a solid grasp on the financial value they are creating and allows them to run hundreds of experiments.
Agility is necessary but not sufficient to build great software and digital experiences. Getting to that next horizon of excellence demands that a company have a clear understanding of its capability gaps and a commitment to improving all of them. Tech companies have shown the way. Now it’s up to incumbents in other sectors to make the leap.
