Not really;) There’s also the apology email issued by Etsy’s CEO for the recent payment processing problem:
I’d like to take a moment to address the recent payment processing issues Etsy experienced. From July 1st through July 24th, Etsy’s payments processor Worldpay experienced a long, frustrating service disruption. As a result of this outage, many Etsy sellers were unable to fulfill their obligations to buyers in a timely manner, leaving those buyers with a less than optimal shopping experience. We know that to our members, Etsy and our technology partners are one and the same, and simply put, we let our community down. Our community of sellers depend on Etsy’s platform to grow their businesses and build positive relationships with buyers, and this disruption stood in the way of building those relationships. For that, we deeply apologize and commit to doing better in the future.
We take our responsibilities to our community extremely seriously, and I want to take this time to be transparent about what happened, what we did to address the issue, and what we’re doing to prevent this from happening again.
On Friday, July 1st we started to see payments for new orders get stuck in a processing state. Our engineering team and payments operations team were alerted immediately to assess the situation. We quickly discovered that our third-party payments processor, Worldpay, was experiencing issues with one of their gateways which prevented transactions from completing. During the first few days of this outage, it appeared that resolution was just around the corner, as payments were intermittently completing. But what we thought would be resolved within a few hours, ultimately took 24 days.
While Worldpay attempted to identify and resolve the issue, our team determined that we could not wait for a fix, so we acted quickly to mitigate the issue. We architected a new payments process to allow for orders to complete even as Worldpay’s outage persisted, and implemented it within two days. While it doesn’t make things better for members who were impacted, I am thankful that we were able to limit a large majority of the user-facing issues that could have stemmed from this outage. Had we not proactively engineered a workaround leveraging the flexible infrastructure we have built over the past several years, what was for most members several days of processing delays would have been a 24-day ordeal of exponentially greater magnitude to us and our community. Our solution did not fix everything, however, and because Worldpay was still experiencing an outage, our community encountered residual issues, primarily with refund processing. (Because we ultimately do not control the last step in refunds, our technical solution did not resolve those delays).
As all of this was unfolding, we did our best to communicate proactively and transparently through daily and sometimes twice daily updates in our public forums and through direct emails to impacted members. Our communications were limited at times by the information we received from our partner, our dedication to convey accurate information, and the volume of inquiries we received. We dedicated a payments task force to be on call for any impacted buyer or seller to help them resolve their issues. We had teams working around the clock to serve our community. We read every one of your forums posts, tweets and emails.
On day 24 of the outage, we received confirmation that Worldpay had instituted a permanent fix for their gateway, and transactions began processing normally on their end. We have been closely monitoring the status since then and have been working through our backlog of refunds. We are now able to confidently say that all new payments are functioning normally. There are still some residual refund processing issues for a set of past orders. We are aware of each and every order affected, and, in partnership with Worldpay, we are committed to resolving these issues as quickly as possible.
We know the impacts this outage had. Generic apologies are insufficient, but I will say that we did not deliver for our buyers and sellers, we are deeply sorry for the difficulties our community endured, and we have not taken this lightly. Etsy lives and breathes its sellers and we feel it deeply when we let you down. We are committed to preventing this from happening again. We’ve already begun making our platform stronger by adding redundancy and resiliency. We’ll be making more changes over the next few months, devoting all of the resources it takes to ensure that checking out on Etsy is reliable, convenient and secure.
Thank you for bearing with us during this time and for being part of our community.
I dunno. After over a decade in retail I feel I can spot the ivory tower putting on the B.S., but this came close to being an actual apology. Would’ve been nicer if they have given all affected sellers some sort of credit, but that’s probably asking for too much.
In any case, this is why it’s important to have more than one selling platform. To not put all your eggs in one basket. I would come up with more aphorisms but I’ve been useless today, owing to an impromptu ER trip with one of my kids last night (everyone is now fine).
Have a great weekend, Art On, and be awesome.