I know we are in good hands with Akamai. I’m not afraid of whatever comes our way now because I know that Akamai is handling way bigger attacks than we’ve ever had on our platform.
Marcin Bienek, Site Operations Manager, Kleinanzeigen
Marcin Bienek never planned on being a site operations manager. The job description really didn’t exist when he started his career at Nokia, maintaining databases that authorized customers to use the minutes they’d purchased for their cellphones.
In that pre-cloud, pre-smartphone world, all Marcin wanted to do was “really low-level C programming. If I could program, I don’t know, some credit-card reader, that was fun for me.” But careers have a funny way of advancing — and today, he runs the site operations for Kleinanzeigen, one of the most popular websites in Germany. Rather than tinkering with code in some quiet corner, he’s the man facing down one cyberthreat after another as head of a 10-person team that keeps the ecommerce enterprise humming.
His vocational concerns reflect the rising complexities and challenges that are endemic to modern IT operations. When he first joined Kleinanzeigen (German for “classifieds”), it was just him and a couple of other guys trying to write some programs and keep their 16 on-premises servers up to the task. But the online classifieds market experienced breakneck growth from the beginning — and they had to invent ways of coping with the challenges that accompanied it.
“You write an application, they put it in production, and it really takes off,” Bienek says. “Then there’s like a million people in Germany suddenly using it. And now you have to figure out how to cope with the load, how to make sure you have enough resources.”
On-the-fly solutions to new threats
Faced with 25% annual traffic growth year after year, Kleinanzeigen’s site ops team scurried to take advantage of any technological advantage they could find — first switching to virtualization with VMware servers, then Docker containers and orchestration, then building their own Kubernetes clusters and relying on thousands of servers to serve their customers, which often seems to be everyone in the entire country.
Over the years, the site has grown far beyond just a platform for free classified ads. Today, Kleinanzeigen’s text service has become the default communication vehicle between millions of Germans when it comes to secondhand trade.
“It’s like basically a kid gets born in Germany these days and the parents create a Kleinanzeigen account,” Bienek says. In more quantitative terms, the site averages about 40,000 HTTP requests every second, peaking at up to 75,000.
With the prominent place Kleinanzeigen occupies in the market, it attracts interest not just from legitimate consumer and commercial interests but also from those with criminal intent. To make effective use of the massive amounts of data the site contains (terabytes of information flood in and out almost daily), both the good guys and bad guys rely on bots. These automation tools are often run anonymously from sites that don’t identify their users or organizations, so Bienek and his team don’t have any way of immediately knowing the bots’ intent.
One approach is to block the anonymous bot by default, but that can be inappropriate in some circumstances. For instance, one user deployed a bot that was part of an app he’d created to help Kleinanzeigen shoppers find what they needed quickly. Because the bot’s purpose was not evident to the security team, it had been blocked. The user explained the bot’s intent to Bienek, who told him to check with Kleinanzeigen’s product team about incorporating the app. But such case-by-case manual attention doesn’t scale when trying to resolve the legitimacy of hundreds or even thousands of bots.
The site also was subject to password/account checkers that would snatch user identities so they could defraud others by selling products they didn’t own, collecting payments, and never shipping the items. Other misuses included price checkers and scrapers that would build databases of advertised items and resell the information without permission.
All of this resulted in tremendous headaches for both site ops and the customer service department. Initially, Bienek and his team tried to create an automated solution, but quickly realized it would require a full-time effort by several staff members to create and use an effective bot-fighting tool.
Starting with another outsourced solution
Bienek knew that mobile.de (a sister company) used Akamai Bot Manager, but his organization first chose a different solution. This alternative solution quickly showed itself to have major weak points. For example, using the tool to conduct a forensic investigation of an attack was time-consuming because it had not been designed to process the terabytes of information generated by a site as vast as Kleinanzeigen.
More significant to Kleinanzeigen’s core business was that the tool often had trouble determining which bots were malicious and should be blocked while simultaneously allowing legitimate traffic to the site. Because of its limited ability to fully analyze the source of bots and the like, the tool generated a lot of false positives — alarms that resulted in shutting off access to those who had done nothing wrong.
Switching to Akamai Bot Manager
Because of these flaws, Bienek proposed that they try a “proof of concept” trial with Bot Manager. Seeing the extensive amount of detailed, actionable information that the Akamai product could provide was eye-opening. “I was already sold,” he says. “We compared it with the tool we were using, and basically Akamai crushed it.”
In short, he realized that Bot Manager could help him cope with the pressure he receives on a daily basis as the site operations manager — from customers, from government regulators enforcing privacy laws, and from internal stakeholders who want to keep the site up and running no matter what new tricks cybercriminals are dreaming up to circumvent protections.
“I know we are in good hands with Akamai,” Bienek says. “I’m not afraid of whatever comes our way now because I know that Akamai is handling way bigger attacks than we ever had on our platform. And I know that I can call their engineers and they will help me figure out what’s happening.”
He admits that part of the attraction of his job had been the adrenaline rush it provided when he and his team got an incident alert and jumped into a full emergency response to solve it ASAP. But he knows there are better uses of his time now.
Bienek says, “At this point in my life and with this size of a platform, I would rather leave it to the professionals who do it for a living.”
About Kleinanzeigen
Kleinanzeigen is an online classifieds market that brings the joy of sustainable trade to everyone. Users buy and sell on Germany’s number one site for classifieds, mainly secondhand. In this way, they make an active contribution to more sustainability. On average, more than 50 million ads are available in numerous categories — from children’s supplies to electronics and real estate. Kleinanzeigen also offers small and medium-sized businesses the opportunity to present their services online. More than 36 million users per month make Kleinanzeigen one of the most widely used websites in Germany. The online classifieds market was launched as eBay Kleinanzeigen by eBay in September 2009. Since June 2021, Kleinanzeigen has been part of Adevinta, the world’s largest provider of online classifieds. In May 2023, it officially changed its name to Kleinanzeigen.
About Akamai
Akamai powers and protects life online. Leading companies worldwide choose Akamai to build, deliver, and secure their digital experiences — helping billions of people live, work, and play every day. With the world’s most distributed compute platform — from cloud to edge — we make it easy for customers to develop and run applications, while we keep experiences closer to users and threats farther away. Learn more about Akamai’s security, compute, and delivery solutions at akamai.com and akamai.com/blog, or follow Akamai Technologies on Twitter and LinkedIn.
