Last year a company reached out to me for advice on their software development practices. They admired my organization’s ability to build engaging and usable products, and asked if I could meet with them to explain how our software engineering team worked to accomplish this.

I was asked to open the meeting with an overview of our processes and practices. While I could tell that their team was very engaged, they also seemed a little nervous — opening yourself up to criticism from an outsider isn’t easy. A few minutes into the overview I mentioned “Agile”. Instantly the tension in the room lifted and everyone seemed suddenly at ease. “We use Agile too!” The meeting proceeded and spirits continued to improve as they recognized key terms from my answers — automated testing, retrospectives, test-driven development, daily stand-ups — that they, too, used.

A few months later this organization put out a website and a native app to support a heavily promoted event. It was a disaster. It had broken images and videos; it loaded slowly and had a non-intuitive UX. The native app was even worse. It barely functioned at all and content that was supposed to update in real-time remained stagnant from a few days before the event to a few days after the event (maybe forever — I stopped checking after a while). They had released broken software to their users after spending a considerable amount of resources building and promoting it.

I thought back to the meeting and how it clearly didn’t help them. It made me realize that explicitly talking about Agile methodologies was the wrong approach. Ask a bunch of software engineering managers if their Agile practices are working. They will all say yes. Everyone thinks they work with agility, so asking the question is as meaningless as asking a bunch of people if they are smart or if they have a good sense of humor.

As I thought more about it, it occurred to me that a focus on the outcomes of a good Agile process would have been much more productive. Here are five outcomes that I believe show you are truly working with agility.

1) The process is always changing

Agile is about constant improvement and adapting to change. These only happen when every member of the team feels empowered to suggest improvements. If the process remains stagnant for a few months you have either failed to foster a “safe to try” environment, you lack a forum for people to suggest change, or your team does not feel invested in the process.

Our software engineering teams always have weekly retrospectives. Thirty minutes are set aside every Friday afternoon to discuss what was good, bad, or confusing; and what actions should be taken the following week to make improvements. We are constantly applied to the task of making small changes to our process. Half the time we decide to revert the change. The other half of the time we stick with the change and continue to build from there. Week-to-week things don’t change very much, but year-to-year the cumulative improvements to our process are noticeable, as is the team’s increased effectiveness.

2) Clarity is pushed up the chain

Too often it’s the software engineer that exposes incorrect assumptions, unanswered questions, or other issues that act as blockers to building something. Such flaws often permeate through the entire system right up until the last possible moment, which is, of course, the worst possible moment. Waiting until someone starts to write code to discover these shortcomings is a clear sign the process has failed.

Well-implemented Agile practices help prevent this by pushing clarity up the chain. A test-driven development approach allows an engineer to discover issues before they begin writing code. But this clarity can be pushed even further up the chain. By writing user stories with clear and detailed acceptance criteria, this clarity can be achieved as the features of the product are still being defined.

For example, our organization involves the engineers as stories are put into the backlog. Sometimes we discuss implementation, but we always make sure all the engineers have an opportunity to challenge the story. This small investment of time upfront saves a lot of time that would be spent uncovering issues later.

We also include the engineering team in a process called “inception” that we use to start new projects. Getting engineers involved at the outset not only makes them more invested in the project, but it also allows them to make valuable contributions to the “why” as well as the “how” by contextualizing those contributions from the start.

3) Agile principles spread to other parts of the organization

If any team or department in an organization is highly productive, other people take notice. They start to ask questions and become curious about how they, too, can start to work with agility. At our organization people consistently notice the engineering team’s ability to make small chunks of progress very quickly and with confidence. Other departments ask us to run workshops to surface their own team’s true priorities. Our marketing team talks about minimum viable products. And our CEO went from asking us to build new products and features to asking us if we can run A/B experiments to test his ideas. A department run with agility becomes an excellent incubator.

4) Usable software is pushed to production daily

Software should be pushed into production every single day. Even if your industry has regulations that prevent this, code that is deemed ready for production by the engineer who wrote it should be committed every day.

This will produce hard evidence that your work is being done in small, logical chunks. We make sure that every user story could be pushed to production on its own even if it’s part of a very large project. We do this with feature flags or by simply knowing that the code will not get called. It also means your user stories aren’t too big. When the engineers point stories (1,2,3,5,8), a value of 8 almost always means that the story should be broken into multiple stories. If it’s common for engineers to work on the same story for multiple days, you are not breaking up your work into small enough chunks. And when work isn’t broken down into small enough pieces, complexity gets overlooked and becomes an issue later.

5) Less talking, more doing

It’s all too common for people to try to solve a complex issue by “getting into a room together.” Most of these meetings probably happen without any kind of structure directed toward meaningful decision-making. We’ve all participated in such meetings. Everyone going into a meeting should be able to explain the goal of that meeting.

Too many emails is also a bad sign. At a previous job, I witnessed an email chain for a big project between 20 people that topped out at over 100 individual emails. The subtext of every email was, “I just did this thing, and I don’t know what’s supposed to happen next.” Too much communication without clear results is a strong indicator that your process is not working.

When done well, working with agility can be a powerful and transformative force. But software engineering managers all too often implement something basic, check the “Agile” box, and walk away. They forget that working with agility means testing, learning, and iterating not only the product, but the process itself. It’s about responding to change. If you’re not doing the responding part, you are not doing Agile. The five outcomes above are good indicators that you are truly working with agility.