Spain's Difficult Road To Black Friday
Spain's Difficult Road To Black Friday
As we become more cashless by the day, card payments are now critical infrastructure
Thanks for reading Payments Culture. Read more about Payments Culture on our About page. If you enjoy this post, please consider sharing with friends, colleagues, or on social media.
Meco inbox is a distraction-free space for reading and discovering newsletters, separate from your main inbox. You can add your newsletters in seconds, for free ✨
Spain is an amazing country. Sangria, Football, Beaches, Churros. Those who have been to Spain will know it’s far more than just these clichés. But in the run up to Black Friday in Spain something went majorly wrong.
Before going into further details, let’s understand why is Black Friday so important?
Black Friday is a big deal for shopping. It’s an annual event of discounts and special offers which takes place on the Friday after Thanksgiving in the US. Although Europe doesn’t celebrate Thanksgiving - except for American expats - Black Friday also occurs in Europe on the same day. In 2023 it was on November the 24th, and this year it’ll be on November the 29th.
Black Friday has become such a huge deal that it’s spilled over to the following Monday, so-called Cyber Monday. As the name implies, Cyber Monday has been positioned as a day to shop online, with many one-day-only discounts and special offers available. Yet many retailers now have a whole long weekend of discounts, whether in-store or online.
Payments company Stripe put out a press release following Black Friday. This included some history on the origins of the event:
The term Black Friday was coined in Philadelphia in the 1960s, to characterize the way physical infrastructure struggled under heightened volume as crowds surged downtown at the start of the Christmas season… Today, nearly half of all shopping around the weekend after Thanksgiving takes place online, which is why since 2005, the period has been known as Black Friday and Cyber Monday.
The rest of the press release mentions the record-breaking volumes processed by Stripe during the long-weekend:
From Black Friday through Cyber Monday (BFCM), Stripe processed more than 300 million transactions with a total payment volume of more than $18.6 billion. It was the largest ever four-day period on Stripe. [At peak there were] 93,304 transactions per minute, while maintaining API uptime greater than 99.999%
Stripe's systems were available more than 99.999% of the time during BFCM - essentially zero downtime. Software engineers refer to this 99.999% API uptime as "five nines", and when it comes to payment processing, an availability standard such as this is often included in contractual agreements. High levels of uptime can be significant for large businesses with many stores, extended opening hours, or those that trade online 24/7/365. Any downtime is likely to equate to lost sales.
These days, "four nines", or 99.99%, is the minimum uptime many large companies expect over a 12-month period. Over 12 months 99.99% uptime allows for just 52 minutes per year where a payments processor may be unable to process transactions. It could be manageable if any downtime were spread evenly during the year. But any downtime near or during Black Friday is a big issue.
So What Happened In Spain?
Spain has a unique payments market. There’s a payment gateway called Redsys that connects to over 60 financial institutions. With more than 80% market share in electronic payment transactions. This includes payment processors who connect to Redsys to facilitate card payments, as well as directly with banks for ATM transactions. The company is owned jointly by major Spanish banks such as BBVA, Caixabank, and Santander - each of which own 25% of the company.
The problems started on the 17th of November when Redsys suffered a significant outage. That day, more than 2.5% of the electronic payment transactions in Spain were impacted. From the newspaper El Mundo:
The service outage occurred at a critical time for shoppers, in the lead up to Black Friday, which marks the beginning of the peak sales period of the year.
Furthermore, the incident took place on Saturday, very close to noon with many people finishing purchases, or having something to eat in bars and restaurants.
La caída del servicio ha tenido lugar en un momento crítico para el consumo, en la antesala del llamado Black Friday, que da inicio al pico de mayor consumo del año.
Además, la incidencia ha tenido lugar en sábado, muy cerca del mediodía con mucha gente ultimando compras, o tomando algo en los bares y restaurantes.
The outage lasted for around one hour. But more was to follow.
On the night of the 23rd of November, another outage occurred at Redsys. This time, impacting around 1.7% of the day’s transactions. El Economista reported:
On Thursday night at the start of Black Friday sales, the company once again experienced an outage that left many businesses unable to operate.
En la noche de este mismo jueves, en el inicio de las ventas del Black Friday, la compañía volvió a sufrir una caída del servicio que dejó sin poder operar a gran parte de los comercios.
There were some reports of issues occuring on Black Friday itself. Both incidents led to an increase in calls to customer service from members of the public.
Many users were left frustrated as they attempted to make their first purchases on Black Friday… In banks, incidents related to duplicate or failed transactions have increased during both outages, as well as requests to customer service.
Muchos usuarios se quedaron colgados mientras intentaban realizar sus primeras compras en el Viernes Negro y, con ellos, también los comercios y negocios en los que trataban de hacer las adquisiciones. En los bancos, las incidencias sobre operaciones duplicadas o fallidas se han incrementado en ambas ocasiones, también las solicitudes de reembolso de dinero por los fallos, así como las consultas a los servicios de atención al cliente.
Redsys confirmed that the issues resulted from two different technical failures but emphasised that such outages are rare. It had been over 12 years since any similar incident occurred.
Representatives from large banks have come out in support of Redsys. Alberto Rivera, the CEO of Santander Spain - one of Redsys’s largest shareholders - noted that whilst Redsys must improve their systems to prevent reoccurrences, there’s no need to make a big deal out of the situation. It seems there is still a lot of confidence in the system.
What Can Businesses Do?
In an earlier post titled How To Deal With A Payments Outage, I looked at various steps businesses can take to help protect against their payment processor failing. One of the options mentioned was to offer a direct Account-to-Account payment solution alongside a card payment solution (sometimes called Real-Time Payments).
In this case, this wouldn’t have helped. Spain’s largest provider of Account-to-Account payments by far is Bizum. Bizum was also affected by the outages as they use Redsys to connect banks and enable payments to be sent directly from one to the other. With so many transactions sitting with one provider - does the Spanish payments market feature too much risk?
Businesses of all sizes should have a backup payment solution. But this example shows it can be challenging. In the short term, these incidents may encourage competitors to further their activities in Spain. For the incumbents, it's essential to improve service levels and prevent re-occurrence of these outages.
If you enjoyed this post, please consider liking or leaving a comment. It's much appreciated. And please consider subscribing to receive future posts via email or in the Substack app.
Like newsletters? Here are some other newsletters that our readers also enjoy 📚
How Payments Culture Can Help You ☕️
As well as writing this newsletter, Payments Culture also offers various services:
General consulting and advising in the Payments and Fintech domain.
Fractional support for your business - sales, strategy, and GTM analysis.
Or you may have a topic or collaboration suggestion for this newsletter.
For any of the above - email me or message me.