tl;dr: Agile is almost always misapplied today. Most teams aren’t actually learning, they’re just going through the motions. A quick test: ask your team for the three most surprising things they’ve learned in the last two sprints. If they can’t answer, you’re either using Agile where it’s not needed or using it badly. Don’t mistake tactical churn for strategy. Use Agile only when rapid learning is the goal, and reach for better tools when the job demands it:
For strategy: Stop pretending backlogs are roadmaps. Use a venture-style model to drive big bets and clarity.
For well-understood work: Skip the Agile theater and overhead. Shape Up + Lean, or just plan properly.
For optimization: Switch gears into growth hacking with metrics, hypotheses, and experiments.
The goal isn’t to “be Agile.” It’s to choose the right tool, at the right time, for the right job.
In the world of software development, Agile has long been hailed as the ultimate solution to our productivity woes. It’s the Swiss Army knife of methodologies, the panacea for all project management ills. But here’s a hard truth: Agile is a hammer and sometimes we need a different tool.
The Rise of Agile: A Brief History
To understand why Agile has become such a blunt instrument, we need to look at its origins. Back in the 1990s, when grunge was king and the internet was just learning to crawl, software development was dominated by the waterfall model. Imagine building software like constructing a skyscraper: meticulous planning, rigid phases, and a prayer that nothing changes along the way.
Waterfall promised thorough upfront planning and cost estimation. It was the comfort blanket for businesses making investment decisions. Should we build new product Y or expand features of product X? Just look at the detailed Gantt chart and you’ll know!
But there was a tiny problem: software isn’t a skyscraper that can be fully architected up front. Requirements change, technologies evolve, and markets shift faster than you can say “Y2K bug”. Learning must be baked into parts of the process. Waterfall projects often ended up delivering outdated solutions to problems that no longer existed.
Enter Agile, the rebellious teenager of software methodologies. Born from the Agile Manifesto in 2001, it promised flexibility, customer collaboration, and the ability to respond to change. It was a breath of fresh air in a world of suffocating processes.
The Agile Advantage: When the Sledgehammer Hits the Nail
Let’s be clear: Agile, when applied correctly, can be incredibly powerful. Imagine a team working on a new feature for a rapidly evolving market. Requirements are fuzzy, user needs are still being discovered, and the competition is breathing down your neck.
In this scenario, Scrum (the most popular Agile framework) shines. Two-week sprints allow for rapid learning. Daily stand-ups keep everyone aligned. Sprint reviews provide regular checkpoints with stakeholders. It’s a beautiful dance of collaboration and adaptation.
But here’s the rub: not all projects are created equal. Sometimes, the rapid changes in requirements stem from internal dysfunction, unclear company direction, or simply a lack of proper planning. And this is where our Agile hammer starts to crack the foundation instead of building it.
When Agile Goes Awry: The Pitfalls of Misapplication
In my years as a product leader across companies of all sizes, I’ve seen Agile misapplied over and over again. Here are the common pitfalls:
The Two-Week Tunnel Vision: Teams become so focused on the sprint cycle that they lose sight of the bigger picture. They’re driving on a backcountry road at night with only a flashlight to guide their way.
Agile Theater: Daily stand-ups become status reports. Sprint planning becomes a rushed affair of throwing tickets into a sprint. Retrospectives become venting sessions with no real improvements.
The Planning Paradox: Scrum was designed for execution, not strategic planning. Yet, many organizations try to shoehorn their entire product strategy into this framework. It’s like trying to write a novel using only Post-it notes.
One Size Fits None: Agile is often applied indiscriminately, whether the project involves cutting-edge innovation or routine maintenance. We’re using a hammer to both hang a picture and plaster the wall.
The Fatal Flaws: Strategic Myopia and Misplaced Flexibility
The most damaging aspect of Agile’s misapplication comes in two flavors:
Failure Mode 1: Agile for Strategic Planning
One of the worst agile failures I’ve seen is a lack of strategic planning. Teams continually gather input from customers and stakeholders and capture it to their backlog. Then this backlog is used to create the team’s roadmap. This is backwards. Annual or quarterly planning requires a broader perspective, one that Agile’s iterative approach simply isn’t designed to provide.
But the problem goes deeper than just misapplying Agile. The root issue lies in how organizations approach strategic planning altogether. Typical quarterly planning processes cascade down the org chart, with each layer of management asking the one below it, “what’s your strategy and roadmap.” Ultimately this ends with teams themselves, unarmed and unprepared, being asked to answer these questions with little or no company direction or strategy. Here’s where things go off the rails:
Unarmed Teams: Teams are asked to strategize without being equipped with the right tools or context. They’re expected to produce a roadmap out of thin air.
The Agile Crutch: In the absence of better options, teams fall back on what they know – Agile. They squint at their backlog, trying to divine a strategic roadmap from a list of tasks. And what’s worse, most agile approaches dictate full team participation in all activities, translating into a massive waste of time creating a fake roadmap.
Perfunctory Planning: With limited time and resources, this “planning” becomes a box-ticking exercise. Teams want to minimize the time wasted on it and so treat it as an afterthought rather than the critical activity that it is.
The Management Cop-Out: Managers and leaders often contribute little more than a crappy template and a litany of Google Docs comments. Their primary role becomes concatenating the results from their teams, adding little strategic value.
Misaligned Outcomes: The result? A collection of bottom-up tactical plans masquerading as strategy, often misaligned with the company’s broader goals and market realities.
This approach fundamentally misunderstands the nature of strategic planning. It’s like asking a chef to plan next month’s menu by looking at today’s grocery list.
Failure Mode 2: Agile for Well-Understood Deliverables
Not every project is a journey into the unknown. Sometimes, the requirements are clear, the technology is familiar, and the path is well-trodden. In these cases, the overhead of Agile ceremonies can actually slow things down.
The misapplication of Agile here is more than just a minor inefficiency, it’s a fundamental mismatch that creates a host of issues:
Limited Visibility: Agile, particularly Scrum, typically provides visibility only into the next two-week sprint. For well-defined projects, this artificial shortsightedness is unnecessary and counterproductive. We’re choosing to wear blinders when we could have a clear view of the road ahead.
False Sense of Flexibility: We treat these projects as if they’re in constant flux, when in reality, they’re not. This mindset often leads to unnecessary pivots and changes, introducing instability where stability would be more beneficial.
The Planning Crutch: Agile can become an excuse to avoid thorough upfront planning. We tell ourselves, “We’ll figure it out as we go along,” even when we have enough information to make solid plans from the start. If you start building a deck without a rough plan, just be ready for many trips back to the lumber yard.
Misallocated Effort: The time and energy spent on Agile ceremonies (daily stand-ups, sprint planning, retrospectives) for well-understood work is often disproportionate to the value they provide. We don’t need a committee meeting to change the batteries on our smoke detector.
Lost Predictability: One of the key benefits of working on well-understood problems is the ability to predict timelines and outcomes more accurately. By defaulting to Agile, we forfeit this advantage, choosing perpetual uncertainty over achievable predictability.
It’s important to note that advocating for more upfront planning doesn’t mean a full return to Waterfall. There’s a middle ground where we push ourselves to ask and answer tough questions early on, without falling into the trap of trying to plan every detail in advance.
By recognizing when a project is well-understood, we can choose a more appropriate methodology that allows for clearer long-term visibility, more accurate predictions, and less ceremonial overhead. This approach not only saves time and resources but also provides the broader organization with the predictability it needs for effective planning and coordination.
The Way Forward: Precision Tools for Precision Work
So, if Agile isn’t always the answer, what is? We need a more nuanced approach, one that matches the methodology to the nature of the work. Based on the successes and failures that I’ve seen and lived, here’s what I propose:
The Venture Model for Strategic Planning
Seeing new product development through the eyes of an investor can provide a more effective framework for strategic planning and innovation. This approach:
Better characterizes the inherent risks and uncertainties
Allows for different criteria and metrics at various stages of product maturity
Provides a framework for making informed bets on innovation
Let’s dive deeper into how the venture model works and how companies can adapt it:
Key Aspects of the Venture Model
Clear Stage Awareness: Venture-backed companies know exactly what stage they’re at (pre-seed, seed, Series A, Series B, etc.) and what their next focus should be. This clarity drives their strategy and execution.
Fixed Investments: Unlike the open-ended budgets often seen in corporate projects, ventures receive fixed investments tied to specific milestones.
Pitching for Resources: Teams must pitch to raise money at key inflection points, forcing them to articulate their progress and future plans clearly.
Embracing Failure: Ventures that don’t prove themselves out at a given stage are allowed to fail, ensuring resources are allocated efficiently.
Lean Teams: Venture teams are typically just big enough for the task at hand. Roles often blur as people do whatever it takes to move the project forward.
Aligned Incentives: Team members are heavily invested in a successful outcome, with personal rewards tied directly to the venture’s success.
High Autonomy: While investors (acting as board members) can provide input, they don’t dictate day-to-day operations. Control is exercised primarily through funding decisions and the ability to replace leadership.
Adapting the Venture Model for Companies
When implementing a venture model within a company, it’s crucial to recognize and leverage the unique advantages of the corporate environment while being aware of potential limitations:
Simulating Investor Breadth: While startups can pitch to hundreds of potential investors, internal ventures have a limited pool. Companies can simulate this breadth by empowering multiple executives with independent investment budgets.
Expanding Pitch Exposure: Real VCs hear thousands of pitches annually. To broaden perspective, companies could consider allowing outsiders to pitch ideas as well. This could be more insightful and more interesting than traditional job interviews.
Leveraging Horizontal Assets: Unlike standalone startups, internal ventures can tap into valuable company-wide assets:
Recognized Brand
Existing Distribution channels
Customer Data
Compliance support
Ties into existing customer workflows
Network effects across users and partners
Amortized overhead (e.g. facilities, HR, legal)
It’s crucial that the entire portfolio of internal ventures leverages these strengths.
Utilizing Horizontal Support Teams: Companies have an advantage over traditional VC networks in their ability to provide robust internal support. Organize these horizontal teams as services that internal ventures can easily plug into, similar to best-in-class SaaS solutions.
By adopting this venture model approach, companies can foster a more dynamic, accountable, and innovative environment for strategic planning and new product development. It allows for clearer goal-setting, more efficient resource allocation, and a balance between autonomy and accountability that often eludes traditional corporate structures.
Learn more: I haven’t found a great source for details on the Venture Model inside companies. For now, learn all you can about pitching startup ideas and running lean.
Shape Up + Lean When It’s Time to Execute in a Clear Direction
For projects with clearer parameters, a combination of Shape Up and Lean principles can provide a more effective approach than traditional Agile methods.
Shape Up: A Brief Overview
Shape Up is a methodology that emphasizes thoughtful upfront planning (shaping) followed by focused execution. It was developed by 37signals as an alternative to traditional agile methodologies. Here are the key aspects of Shape Up:
Shaping is Distinct from Implementing: Shaping is a separate activity that happens before any project begins. It’s about defining the problem, roughing out a solution, and setting boundaries.
Blurred Lines Between What and How: Unlike traditional approaches where product managers define “what” and developers figure out “how,” Shape Up encourages a more integrated approach where these lines blur during the shaping process.
Dedicated Time for Shaping: Shaping drives the process and is not an afterthought. It gets dedicated time and attention, ensuring that projects are well-defined before they begin.
Small Team for Shaping: The entire team doesn’t participate in shaping. Instead, an individual or a very small group does this work, allowing for more focused and efficient planning.
Complete Projects, Not Tasks: Teams are given complete projects, not a list of tasks. The shaping process provides clear business challenges, boundaries, and guidelines without prescribing an overly detailed solution.
Projects as Commitments: Projects that don’t get completed in the allotted time (typically weeks) don’t automatically continue. This creates a sense of urgency and helps prevent scope creep.
Flexible Implementation: While Shape Up provides a framework for planning and scoping, teams have the flexibility to choose their working methodology for the implementation phase.
Integrating Lean Principles
As noted, once a project has been shaped, teams can adopt any methodology they choose. But one especially effective approach is to use Lean principles for the implementation phase. Lean, with its focus on eliminating waste and continuous improvement, complements the Shape Up methodology well. Here’s how they can work together:
Kanban for Workflow Management: Use a Kanban board to visualize and manage the workflow of the shaped project. This aligns with Lean’s principle of visualizing work and limiting work in progress.
Continuous Flow: Instead of fixed sprints, adopt a continuous flow of work, pulling new tasks as capacity becomes available. This reduces the overhead of sprint planning and aligns with the Shape Up ethos of giving teams complete projects.
Just-in-Time Planning: While the overall project is shaped upfront, detailed task planning can happen just-in-time, aligning with Lean principles and allowing for adaptability within the project boundaries.
Focus on Value: Both Shape Up and Lean emphasize delivering value to the customer and the business. Use this shared focus to guide decision-making throughout the project.
Continuous Improvement: Be judicious with team retrospectives to identify areas for process improvement. Lean principles would encourage just enough reflection at just the right time and no more. Rather than wait for or plan separate retros, encourage process improvement at any time.
By combining Shape Up’s thoughtful upfront planning with Lean’s efficient execution principles, teams can benefit from:
Clearer long-term visibility (typically 6-8 weeks)
More accurate predictions of project outcomes
Reduced ceremonial overhead compared to traditional Agile methods
Improved focus on delivering value
Greater flexibility in day-to-day work management
This approach not only saves time and resources but also provides the broader organization with the predictability it needs for effective planning and coordination, all while maintaining the flexibility to adapt to changes within the project scope.
Learn more: Shape Up from 37signals and a good article on using Trello to implement Kanban.
Growth hacking for product optimization
When your main objective is to improve adoption or usage of your product, it’s time to switch gears into growth hacking mode. This approach is fundamentally different from new product or feature development, focusing instead on measurable outcomes, hypotheses, and experiments.
Understanding the Product Funnel and AARRR Metrics
At the core of growth hacking is the product funnel, often represented by the “Startup Metrics for Pirates” or AARRR:
Acquisition: How do users find you?
Activation: Do users have a great first experience?
Retention: Do users come back?
Referral: Do users tell others?
Revenue: Can you monetize?
These metrics provide a framework for understanding and optimizing each stage of the user journey.
The Growth Hacking Process
Here’s how to approach growth hacking effectively:
Measure First: Before you can improve, you need to know where you stand. Establish baseline metrics for each stage of your funnel.
Create Hypotheses: Formulate ideas about what might improve your metrics. Crucially, define desired outcomes upfront to avoid bias. A good hypothesis includes both the expected positive impact and potential negative effects on other metrics.
Prioritize: Use your baseline measurements to prioritize hypotheses that are likely to have the highest impact. Focus on the areas of your funnel that need the most improvement or that align with your current business goals.
Run Experiments: Design and execute tangible tests for your hypotheses. This is where the rubber meets the road in growth hacking.
Analyze and Iterate: Based on the results of your experiments, refine your approach and start the cycle again.
Tips for Effective Growth Hacking
Step Back from Your Product: Adopt an outside-in, customer-first view. What’s on your users’ minds? What do they want, regardless of what you want them to want?
Use Statistics Wisely: Learn enough about statistics to run experiments correctly, or find someone (or a tool) who can. If you can’t run quantitative experiments, qualitative ones can still provide insights.
Balance Experimentation and Intuition: Just because you can experiment doesn’t mean you should. Use experience and intuition to short-circuit endless experimentation. Sometimes, you can turn a complex A/B/C/D test into a simple A/B test, saving time and resources.
Know When to Switch Gears: If you find yourself unable to run meaningful experiments, it might be a sign that you’re using the wrong tool at the wrong time. Consider if you’re in a phase where Agile for learning might be more appropriate than optimization experiments.
When to Use Growth Hacking
Growth hacking is most effective when:
You have a product with a clear value proposition
You’re looking to optimize specific stages of your funnel
You have enough users to run statistically significant experiments
You’re able to implement and measure changes quickly
By adopting this data-driven, experiment-focused approach, you can systematically improve your product’s performance and drive growth. Remember, the key is to stay focused on measurable outcomes and to always keep your users’ needs at the forefront of your optimization efforts.
Learn more: Step-by-step overview of growth hacking. Or, for more depth, the free and paid materials from Reforge are very good.
And Yes, Agile for Accelerated Learning
When the thing you need most is rapid learning in the face of a complex and quick-moving market, an agile approach can be well-suited to the task. However, the term “Agile” has been so overused and misapplied that it’s almost lost all meaning. There are entire books, consultancies, and corporate transformations dedicated to Agile adoption, often with conflicting advice.
Assessing Your Current Approach
If your current approach is working well and genuinely results in rapid learning, stick with it. Make minor tweaks and improvements as most agile methodologies would suggest. But how do you know if you’re getting the learning you need?
Here’s a quick test: Ask one of your agile teams to name the three most surprising things they’ve learned in the past two sprints. If you get nothing substantial back, it could mean one of two things:
The team doesn’t actually need to learn and is using the agile sledgehammer unnecessarily.
They’re not using agile effectively for learning.
Starting Fresh with Scrum
If you’re not getting the learning you need, consider adopting a new methodology wholesale. For a clean slate, I recommend Scrum. It’s concrete, clear, and well-established. Scrum provides a framework and terminology that your teams can quickly understand, with built-in mechanisms for continual improvement.
Key elements of Scrum include:
Defined roles: Product Owner, Scrum Master, Scrum Team
Clear artifacts: Product Backlog, Sprint Backlog, Shippable Product
Standardized Rituals: Sprint Planning, Daily Standups, Sprint Review
Continuous learning: Sprint Retrospectives, Improvement and customization
When adopting Scrum, resist the urge to cherry-pick practices at first. Give teams the opportunity to experience all aspects of vanilla Scrum before they drop or alter some parts. This may seem counterintuitive, but it’s actually easier to start with the whole thing and then make thoughtful changes, rather than trying to build up bit by bit.
Tips for Effective Agile Learning
Customer Collaboration: Ensure you have a real customer working closely with your agile team. This direct connection is crucial for rapid learning and validation.
Measure Learning, Not Just Delivery: Focus on the speed of learning versus the speed of delivery. Agile is about delivering software that aligns with real customer needs, not just about quickly pushing out features.
Embrace Uncertainty: Accept that in complex, fast-moving markets, you don’t have all the answers. Use agile practices to explore and learn rapidly.
Frequent Reflection: Use sprint retrospectives not just to improve processes, but to explicitly discuss and document learnings about the market, customers, and product.
A Word of Caution
Please, if you take nothing else from this, don’t force all of your teams to start practicing Scrum (or any other agile methodology) today. Acknowledge that different teams are at different phases in their own projects and need to choose the right tools for their specific situations.
Instead:
Arm your teams with a variety of tools in their planning toolbox, including the ones we’ve discussed in this article.
Help them gain the wisdom to know when to switch tools.
Foster a culture where it’s okay to experiment with different methodologies and learn from the results.
Remember, the goal is not to be “Agile” for the sake of it, but to create an environment where teams can learn rapidly and deliver value effectively. Sometimes that means using Scrum, sometimes it doesn’t. The key is having the flexibility and insight to choose the right approach for each unique situation.
Learn more: The Scrum Framework, in detail.
Conclusion: The Right Tool for the Right Job
Agile isn’t inherently flawed; it’s how we’ve been wielding it that’s the problem. It’s time to expand our toolkit. By adopting a more nuanced approach, we can enjoy the benefits of Agile where it truly shines while using more appropriate methods for strategic planning and well-defined projects.
Remember, in the world of product development, one size does not fit all. Sometimes you need a sledgehammer, sometimes a screwdriver, and often, you need both. The key is knowing when to use each.
So the next time someone suggests “Let’s do this project Agile,” take a step back and consider: What’s the right tool for each phase of this project? Your future self (and your team, and your stakeholders) will thank you for choosing the right tools for the right jobs, rather than forcing the entire project into a single methodology.