ITECH SOLUTIONS took the challenge of a security breach that impacted half our clients and turned it into an opportunity. It was painful, but through that pain we refocused our business on quality of customer vs. quantity, redoubled our security focus, reduced our risk, and reemerged with some new lines of business.
The breach happened in 2018, a year we expected to hit $1.8 million in revenue after a period of rapid growth and customer acquisition. We were in the process of migrating from an RMM tool that didn’t support multifactor authentication to one that did, but got attacked on a Friday night before we completed it.
On Monday morning, customer calls started coming in; one by one, they had gotten hit with a variant of WannaCry ransomware. At that point, we were completely oblivious to the idea that we were the source of the breach, because two years ago you didn’t hear much about MSPs being an attack vector. Then we noticed a commonality: All the clients getting hit were still on our old RMM. We checked the SQL logs and discovered one of our employees had been phished. The hacker had used that account all weekend to log in and out of the client accounts. Once we realized the old RMM was the vector of attack, we immediately cut off access to it.
We created an incident response plan on the fly, restored from backup everyone we could, and procured Bitcoin for the two clients we couldn’t restore after negotiating with the hacker. The remediation took about two weeks, and we incurred some hard cash losses. Fortunately, because of the deep customer relationships we have, we only lost three clients, two of which we had already been in the process of parting ways with.
My knee-jerk reaction was to become an MSSP so this didn’t happen again, but the expense and level of expertise required didn’t make sense for our business. So after some peer networking and vendor research, we built a security-first MSP stack that minimizes shared risk. Most important, we recognized that we needed a single source of truth to understand which users, which devices, and which networks we’d be responsible for protecting. We embraced the Microsoft stack to help us do so, and now require all clients to be on Azure Active Directory and Intune. We also require everyone to have a unified threat management firewall in place, and to have cyber insurance to cover losses we’re not responsible for. We did not take on any new clients for two years as we worked to get our existing ones up to par with this enhanced security stack.
We learned two key lessons from this experience. First, communication and transparency with clients are key. They need to know they can trust you, so don’t hide anything and keep them updated on the remediation. It will help you retain them.
Second, get your insurance company involved immediately and let them guide the process. We didn’t have the benefit of that in 2018, but the industry has changed and they now have experts to help with a breach.
In fact, we leveraged our experience to develop a new line of business working with insurance companies to help their clients that have been hacked.
Silver Lining
Another new line of business arising from the aftermath is co-managed IT. We already had one fairly large co-managed IT client and liked that collaborative relationship of working with and supporting an existing IT department, so decided to grow that offering. These types of clients understand the value of security, and it’s a great way to scale our business as we can take on a lot more endpoints and users under management without having to staff up.
Finally, ITECH Solutions shifted its growth paradigm. Instead of the model we were following to constantly acquire new customers to grow, we actually pruned our client list to those that value both security and our services. In doing so, we minimized risk and boosted our monthly recurring revenue.
While WannaCry made me wanna cry when it happened, we came out of it stronger.
Photography/Brian Weiss by Carlos Alvarado Photography
Photography/ITECH Solutions team by Tayler Enerle Photography
