I've been using and loving Feature Flags for a long time. And, historically, I've embraced feature flags as a construct that makes product development faster, easier, and safer. But, last week, on the Deploy Friday podcast, I started to formulate some novel thoughts on what makes feature flags so exhilarating: they shift the balance of power away from Designers, which is probably a good thing.
Before we had feature flags, there wasn't a great way to incrementally build parts of your product: code was either deployed to production or it wasn't. Since the "deployment" of code to production was tantamount to the "release" of a feature, code couldn't be deployed until it was "feature complete".
So in a pre-feature-flag era, code was deployed to a staging or Quality Assurance (QA) server of some kind where it could be reviewed by your product team, your QA engineers, and various other stakeholders that had a keen interest in the product experience. This is historically where a lot of "fine tuning" and "polish" would be discussed:
- Adjust the animation timing.
- Move this up 3 pixels.
- Change the language on that button.
- Pull these two elements into better alignment.
- Track that click-event.
And so the iteration would begin: deploy to staging, gather feedback, apply feedback, re-deploy to staging. And so on.
For engineers, this dance is the stuff of legend. Every engineer carries with them war stories from the trenches: the designer who refused to let anything be deployed; the executive who was prepared to die over the wording of a page title; the marketing director who needed every single interaction to be categorized and tracked and posted to several different analytics services.
In those days, since the new code was so far away from the customers, it all felt like part of the "design and development phase"; and, that all of this fine-tuning and hand-wringing was being done in an effort to create the best product experience possible.
In those days, designers were the gate-keepers of quality.
But, feature flags changed everything. Once we had feature flags, we no longer needed a staging server: incomplete code could be deployed to production without being released to the users. So all that iteration - all that polishing and fine-tuning and gathering of feedback - that's all now done in production, behind a feature flag.
Now, the new code is collocated with the customers. In fact, the only thing that's keeping the customers from accessing the new code is a toggle. A toggle that, when flipped, instantly synchronizes across all production environments and grants customers access to the new feature.
This change in the location of the code dramatically changes the conversation. Whereas in the past, when the code only ever lived locally or in Staging, everything felt "pre-production". But, once the code is literally on the production servers, it takes on a new gravity. It put "value" at only an arm's reach away from the customer.
And when you start to realize that the only thing between the customer and this new customer-value is a toggle, you start to seriously question, why we ain't flipping that toggle on immediately?!
Because, isn't "some value" now better than "all the value" later?
It turns out, yes. In almost every case, yes.
So now, people who don't want to turn on the feature flag, they aren't "gate-keeps of quality" like they used to be, they're the people who don't want to deliver value; they're the people who have the power to improve the lives of our customers with a single click; but who choose not to.
Nobody wants to be "that person".
And so, the power dynamic changes; and the perspective shifts; and your team begins to bias towards actions and embrace workflows around iterative product development. Which is - without a doubt - the key to successful product development.
Now, to be clear, Designers are critical to creating a quality product. I am in no way trying to diminish the role of design. In fact, I would kill to have unfettered access to a designer on my little product team. But, Engineering, Product, and Design (EPD) are three legs of a stool. And that stool is only structurally sound when everything is in balance - when the forces of each leg are properly tempered by the opposing forces of the other legs. And I believe that feature flags are key to restoring that balance.
I am fully on board with this mentality.
Does the code work? Can the customer use it? If yes then push live. Then iterate on that to improve it, now with customer feedback most likely.
A non-web example of that I saw was in the show Long Way Up, they were using pre-production Rivian trucks so they didn't have the fancy touch screens. Instead the engineers had given them toggles, buttons and switches to control functions that later would be done via the touch screen. Was it beautiful? No, it had little labels stuck on next to toggles and the like, but it worked.
The crazy part of it all is that if you ask Customers, "hey, do you want it to work, or do you want it fancy?", they will (in my experience) be like, "Please just let me have it ASAP! I need it to do my work!". They don't care about the bells and the whistles.
Exactly, they have work to do.
Whoa, whoa, whoa. heh
I had this fight when I was at Cox Automotive. We had designed a home page, created faster ways for people to do things. However, we had designed this "widget" if you will and tested it. In the meantime, the FED had implemented it. It did not test great and did not scale at all. We redesigned it and voila it tested well and scaled.
The Engineering Senior Director was adamant to turn it on. It gives people a better way to get in and do things. Sure, it did, but the other solution we came up with was way better, and here is my sticking point, was going to be different enough to cause some confusion when it was changed.
It isn't about prettier, fancier or bedazzled. A UX'ers job is always about giving the user what they need in a way that also works with business and is a good technical solution. Jumping the gun on functionality isn't always in the user's best interest if the said solution is not fully tested.
I get your "I need to do my work" comment. I also stand with the UX'ers job should also never be about bells and whistles, but making sure the assumption made about the design is validated with actual users. There should also ALWAYS be a future state and an iteration plan to get there. Add a bit here or there but still, have that plan in mind so there is no frankensteining of the product.
I was kidding about the whoa. Without a plan, bells and whistles are meaningless.
I think focusing on UX is a solid plan most of the time. I think a lot of what I react to (and why I've developed this feeling) is the focus on design in the absence of UX. Or rather, the use / misuse of the term UX for when people mean "bells and whistles". I know that you and I have butted-heads in the past on what UX means (as I tend to throw it around more casually). But, I think we both agree that UX should never mean "I think that button should have a 6px radius, not a 4px radius" 😱
That's why I think all the departments need to temper each other, so no one department runs-amok.
Don't think we've ever actually butted heads. If a designer is focused on the look and feel instead of what the problem is for the users, and how that measurement of success should look, that designer should be fired. As a matter of fact, I'd say a few things as I look at Epics and features should happen that often don't.
- A Needs Statement should always accompany and the Epic statement should be worded as what problem is being solved for the user. (I often have to help Product with this)
- For both Epics and Features, alongside of business measurements of success, there should be a measurement of success for the user. Actually measurable. Time-bound, whatever.
Something like the user will save x amount of time, or be able to do something they weren't able to do before. Along with how it will be measured. A tool like Auryc, Google Analytics, usability testing, whatever.
- I actually meant an actual Design Plan/Brief.
As you know Ben, I am passionate about collaboration. There is never a time I should be surprised at a Sprint demo, nor should the engineering team be surprised by a design. If we aren't working together, that is how bad things happen.
That all makes a lot of sense to me. I especially like the idea of a Needs statement, especially since so much of what I do is based on Zendesk tickets and feature-requests; so, it's usually based on actual points-of-friction from the user. Which, I think, would make the needs statement much easier to codify.
Measuring stuff is also cool. We were actually just talking about that (internally) yesterday. Specifically with the fact that we use Feature Flags to turn on new feature, but never really to test new concepts. This is something we would like to be better at (having experiments and then actually testing to see if the new experiment is better than the old approach). Mostly, we just decide to do something, then do it.
What are your triggers for moving code from Staging to Prod, in a generic sense? I don't think you're advocating that code be collocated in both environments prior to QA being completed, but you did not explicitly state that in your post. A pedantic point, I know. My thoughts immediately went to situations where a security flaw could exist behind a feature flag in Prod, but is still somehow exploitable.