How we (actually) deprecated our old tech
Building something new? Easy. But what about actually deprecating the old feature/product/etc?
Eesh.
Here’s how Klaviyo’s Editors team successfully migrated 68 million templates and deprecated our old tech.
Why we wanted to do this
Simply put, our new email template editor (now our only editor) serves customers better. The new editor was released in June 2021 and supports custom fonts, has new blocks that integrate with other parts of the product, includes the ability to undo/redo, and you can even favorite blocks to reuse across other templates.
Even better (as an engineering manager), this one is maintainable and extensible. Woo!
What often happens is companies end up supporting two products. Companies can’t figure out how to happily migrate customers (who might not want change) over to the new product. This can create a handful of situations, two of which I’ll highlight:
Focus on growing both products
The engineers are split between two teams, one team supporting the previous product, one the new. Inevitably, this creates duplicative and diverging editors (or whatever the product is). The weight of supporting two editors is felt on every part of the organization. Two sets of support docs have to be written. Customers have to figure out which support docs to read and the support team will need to be well versed in what each editor supports.
- What if a customer wants a feature from the new editor? Do they wait? Can they migrate? Can they have some templates in one email editor and some in the others?
- And what about how email templates integrate with Apple, Gmail, Yahoo, etc. on ios/android/macOS/Windows? There’s going to be different issues with each editor’s email templates when clients release new versions.
Focus on growing the new product, maintain the old
The company keeps around both products but are really focused on growing the newer one. The “classic” technology only gets love when it’s on fire, otherwise all 8 engineers (in our fictitious scenario) are focused on the newer technology. Customers still on the “classic” technology feel the lack of support. It might not be today or tomorrow, but they will eventually have a problem no one is willing to help with. What then? I’ve been there in previous jobs and seen the pain it causes customers to know “you don’t care anymore.”
Hopefully I’ve convinced you that, usually, the best path forward is to migrate users (even if it’s over an extended period of time) to your new technology/feature/product.
Now how to do it?
Quick Note
The very kind person editing this blog article told me (twice) that it was hard to understand the complexity of the work we were doing and I was burying it.
This migration was extraordinarily intricate.
The real complexity of the work wasn’t in migrating a user from one UI to another. The complexity is we changed the templating structure in each email editor. The html/css that was output was fundamentally different between the two editors. Every single email had to be converted from one templating format to another and still look identical in every email client on every hardware and on multiple versions.
On top of that, we have a hybrid feature in our editors where users can import their own html templates. We then allow them to use their own custom templates in our editors and add custom blocks. Amazing! But now we’re converting code we don’t own, that wraps our code, and trying to do it perfectly in all email clients.
Phew!
Lesson 1. Break out the work.
When I joined the project, the team was a year in on trying to move users from the classic editor to the new editor. Templates could be migrated from the classic editor to the new editor and (most) high priority bugs had been fixed. The project had been underscoped and the team was in the midst of many, many necessary bug sprints. The project was considered far behind schedule, and the team + leadership desired a higher level of predictability.
Think smaller
The first goal was to properly assess work and break it out into small projects. If something is behind schedule or seems impossible, always always always look at if you can break the work into smaller deliverables.
What were the smallest pieces could we deliver to users? Tracking those enabled us to communicate better with leadership, track our progress, figure out WHO to communicate with and when, and (an important piece!) stay motivated.
“Deprecate the classic editor” became six projects that we delivered over the course of six months:
- Auto Convert Non-Hybrid* Templates on Clone (*hybrid explained above): When a user cloned a template for any purpose, we automatically converted the template from the classic editor to the new editor. At this point, the template would still be revertable.
- Opt-In Non-Hybrid Templates: Think of this as a beta. A user could choose to opt in all of their templates to the new editor.
- Automatic Conversion of Non-Hybrid Templates: On a set date, all of Klaviyo’s non-hybrid templates were converted from the classic editor to the new editor.
- Net New Hybrid Templates Use New Editor: This project meant any new custom code templates (aka hybrid) opened within our new editor by default.
- Hybrid Editor - Manual Conversion: Users could choose to convert their templates one by one to the new editor.
- Hybrid Editor - Automatic Conversion: On a set date, all of Klaviyo’s hybrid templates were converted from the classic editor to the new editor.
And then we get to delete the classic editor code! (And this is also was broken out into four projects).
Lesson 2. Evaluate what is truly high priority.
At what point do you stop working on bugs? At what point do you stop focusing on new features?
Sometimes you gotta get yourself out of the quicksand. And an old product that is no longer being worked on, has its own quirks (erm… bugs) that customers have learned to work around, and interacts with email clients is absolutely quicksand.
We made a choice to only focus on two types of new editor issues: high priority and conversion bugs. This freed up a substantial amount of our time to focus on getting out of the quicksand.
Every new feature was weighed against the specific classic editor deprecation project. Is updating our image cropping functionality substantially more beneficial than Auto Convert Non-Hybrid Templates on Clone?
This allowed us to make informed trade offs along the way and move fast. When a project is too large, it’s easy to lose focus and say “Oh deprecation has been a year? And might be a year or more? Yea let’s pause and add this feature.”
Staying the deprecation course is absolutely not easy. The product manager and I had to say no many times along the way and advocate for clarity of vision, trusting that we could get this done quickly if we were all focused. It was worth it!
Lesson 3. Communicate. Communicate. Communicate.
With each project, we had to figure out how to communicate with customers. At the end of the day, Klaviyo is forcing this change. But how can we make it as enjoyable as possible?
For Auto Convert Non-Hybrid Templates on Clone, we notified users in a toast every time a clone happened.
For Opt-In Non-Hybrid Templates, we had a banner on our templates page that asked users to opt in.
For Automatic Conversion of Non-Hybrid Templates, we emailed users, put a banner on the top of the website, and gave three weeks of warning.
We had to figure out how to best reach users. Even with this communication, we still received feedback saying “But you never told us you were getting rid of the Classic Editor!”
Change is hard.
Lesson 4. Evaluate progress each step of the way.
How did we know if we could proceed with the next step?
The team had already built an internal tool called render diff. This tool did visual comparisons by taking classic editor templates, converting the template to the new editor, and comparing the visual differences. We had a sample set of 10K templates and had almost perfect accuracy.
With Auto Convert Non-Hybrid Templates on Clone, we tracked how many times users reverted from new editor to classic editor. Whenever a user reverted a template, the template id and reason for reversion was recorded in our analytics service. From there, we extracted all template ids for users that answered “Something is wrong with my template” into a Google Sheet.
These template ids could then be manually rendered by our engineering team. Was there a visible difference between the classic editor and new editor? We found and fixed quite a few bugs this way. But, I want to note, very small bugs, we left.
Why leave small bugs?
We knew that emails were converting very well. And each fix had a substantial amount of effort involved. Will fixing the converted template have an impact on templates already in the new editor? And let me tell you, Apple ios/mac/cloud, Gmail ios/android/cloud, Outlook.com, Outlook, Opera Mail, Yahoo Mail do not play nice together. How does the fix render in each email client?
You have to take educated risks. In this case, small bugs were easier and safer to fix once users were on the same base template structure.
We then did Opt-In Non-Hybrid Templates so that users who were excited to work in the new editor exclusively could beta test. We prioritized any issues that came out of this immediately. Fortunately for us, there weren’t many at all which gave us a ton of confidence to move forward with Automatic Conversion of Non-Hybrid Templates.
Lesson 5. There will be chaos (aka email clients are in the dark ages).
All multi quarter, long running projects are going to have an “oh shit” moment. Take a break, evaluate the real risk and fallout, and figure out next steps.
For us, it was the week before we did the automatic conversion of 68 million templates… Webkit did a release that caused 1px white lines to appear on “display: inline-block” images. Webkit is used for both Apple mail and Gmail.
This was affecting a handful of large accounts and wasn’t happening with classic editor.
Dangit.
Here, we made the decision to ask our lead engineer to go heads down on the problem for a week. Was there a solution? Would anything work? If it did work, would the issue be fixed for all email clients?
We decided it was worth it to proceed if there wasn’t a safe fix (highly likely given how many email clients / versions / hardware options there are). This was a very hard decision we desperately didn’t want to make or follow through with.
Last minute, thank goodness!, our lead found a fix.
Let’s goooo
And that’s it! To recap:
- Break out the work
- Evaluate what is truly high priority
- Communicate. Communicate. Communicate.
- Evaluate progress each step of the way
- There will be moments of chaos
As of July, we are back to a single email template editor in Klaviyo 🎉! No more split personalities. Our Editor team engineers can now put their full effort into features and bug fixes that benefit all users. Was it worth doing a crazy difficult deprecation project? Yes! Moving to one editor was the best choice for Klaviyo. Do the best you can, pick the best path for your users, and stay the course. Good luck!
