Reconstruction of a cyberattack on a Slovak manufacturing company with a 70-year successful history
It is Tuesday, April 13, 2021, 6 a.m. An employee of a manufacturing company is starting a typical workday. He switches on his computer – and immediately everything changes. The screen lights up with a message from a cyber attacker: “The computer is encrypted, contact me within 24 hours.” Simply put: the system has been hacked and the hacker wants a ransom for the return of the data.
“It was the blackest day in the 70-year history of our company,” the company’s managers agree today.
Not only in the world, but also in Slovakia, attacks on private companies and state institutions are a daily occurrence. In some places, cyber attackers are playing it safe, in others they are just trying it out and waiting to see which action will succeed. The damage to successfully attacked companies is consequently enormous. They lose hundreds of thousands of euros on at least four levels: in business interruption, in the need for technical recovery of systems, in contractual penalties for business partners, and not infrequently in the payment of ransom. In addition, the unintended consequence is usually a loss of customer confidence and reputation.
In the following article, we present a reconstruction of the attack on a Slovak manufacturing company, which is approaching a successful finale thanks to the rapid search for professional management solutions, high commitment of loyal employees, and last but not least, timely and open communication to clients.
The company’s first steps after the attack
Let’s go back to the fateful 13 April 2021. The employee who found the message from the attacker contacted the company’s IT manager. His first response on the phone was: “Which server is it?” They’d had experience with individual encryption before, and they’d always handled it gamely. However, the employee’s response was: “All of them, I’m afraid.”
What was originally a classic working day in a manufacturing company turned into a series of dramatic events, requiring the deployment of all in-house components, including top management.
First of all, all 38 servers, including computers and printers, had to be shut down from the data network. Those that had been rebooted or jammed by the virus had to be “hard” pulled out of the power cabinets. “That’s when we all understood that the remedy would take longer than the maintenance downtime,” recalls the IT manager.
Normally, their server was restored from backups within one or two days, but after the attack the data continued to be encrypted even after a test power-on. What to do about it?
Management’s immediate reaction was, understandably, anger against the attacker. “Why our company? For whom can we be interesting as a target,” recalls a member of the company’s management on a black day. ICT infrastructure, systems, equipment, information flow to production, email communication, everything they had directly tied to IT was suddenly not working. “Imagine having a network that’s shut down, you’ve got contaminated servers, and you’re finding out how much your backups have been affected.”
Every single person in the company felt like they were at ground zero. Only one thing was certain: the righteously annoyed members of the management team agreed and agreed that under no circumstances would they pay anyone anything.
“At the same time, we realised that anger will not help, that it is necessary to think constructively, to set a strategy and to solve the situation by gradual steps. It was clear to all of us that the most important thing was to maintain production, which is crucial in terms of the company’s production, revenues and liabilities. This was at the cost of having to go back to paper and pencil to manage operations,” recalls a member of the production company’s management on the first hours of a black day. “It was extremely difficult.”
First recommendations of specialists
The technicians first considered the possibility of connecting via a Windows device to a NAS in the server room. This is a storage device that allows companies to store and protect large volumes of data securely. However, there was also the possibility that malicious codes could be transmitted or encryption could be triggered. The company therefore turned to IT consultancy provider GAMO.
The consultant’s first recommendation was: Check the backups through the Linux appliance. “We went gradually, the piles of files we went through were always fully or partially encrypted. And we know that if something on a virtual server is partially encrypted, there may already be invisible malicious code embedded. In that case, we should not use anything from that server,” comments the IT manager of the company that was attacked at the time.
So they turned to the market leader in security solutions. Although it registered ransomware attacks on several Slovak companies, it did not have a concrete solution to the situation. “So the plan of restoring everything from backups did not work. And time was running out.”
One hope in saving some of the data was cassette tapes. “We backed them up on these media about six months ago. Hoping there might be usable information on them, we sent the tapes for the last five years to our consultant,” the IT manager continues.
These were laboriously categorised in GAMO using robotic equipment. The process of rolling through the tapes one by one and finding connections in the records took more than three weeks. In parallel, the company looked for other ways to solve the problem.
Fast transition to the cloud
Since the entire infrastructure of the attacked enterprise was encrypted, an environment was sought that would get the information system back on its feet quickly and securely. The ideal solution was the cloud. On the second day of consultation with the specialists, the company’s management chose this new way of operating and requested the service for all servers.
Step by step, the technicians went through what else could be salvaged. For example, the production system database from the previous day was found. The examination revealed that it was not contaminated, but only damaged by a severe server shutdown. The corrupted transactions were repaired, an MS SQL blank database was installed, and the import delivered a database with no extension, no encryption. “If the database was not in order and all the checks of all the drives inside did not match, the database system would not accept it. However, once they cleaned up the transactions and the database was imported correctly, it went through testing and was then sent to the cloud,” the enterprise IT manager explains.
“We put the rescued databases and data of the company’s most important systems on two disks,” says one of the company’s top executives about the moment of crisis.
They had to be quickly delivered from central Slovakia to Bratislava so that GAMO technicians could bring them to life in the cloud environment. “It was a strange feeling to suddenly have the entire wealth of a company you have been building for 70 years on two disks. It’s not a hall that you can build, it’s not a rack that you can buy. It’s two disks that you don’t know what to do with. The data was in there somewhere, but it had to be mined,” he recalls, adding that he had it “stored in his bed with him all night, instead of his wife, who was out of the house.”
He left for Bratislava very early in the morning. “My head was swirling with various thoughts and work obligations – but they were all overwhelmed by the knowledge that I was literally carrying in my car perhaps the only salvation for the company. And imagine carrying something of incalculable value to you, and then having those assets picked up at an arranged location by a man you’ve never seen before. Who puts them in his rucksack like they’re nothing – and walks away… You stand there wondering if you’ve just made the right decision,” smiles a top manager today.
There are several security measures in GAMO Cloud. The first is the data centre itself, from which the cloud services operate. Thanks to its construction and physical and object security, it looks like an impregnable fortress from the outside. In addition, the provider guarantees the safety and security of all data. It is built on VMware’s SDDC technology with maximum emphasis on security. Its security meets TIA 942 standards for Tier III. GAMO was the first Slovak cloud provider to receive the prestigious CSA Star certificate awarded for the most comprehensive security clearance. Here there is no need to deal with operation, maintenance, or continuous availability of servers.
Production sanitisation and supplier assistance
The management was looking at how and where to obtain data and databases for the three most important systems – Production System, Drawings and Production Preparation.
Parallel to those priorities, in collaboration with suppliers, they also acquired new and loaned equipment to process data from salvaged and intact databases. A stroke of good fortune was a vendor who had a five year old database from a legacy system, which allowed insight into the old entries. After technicians’ approval, the individual workflows and BOMs were printed on paper and pushed to production.
Three more computers and an offline printer were installed so that workers could copy drawings and designers could prepare paper production for the following week. It was also possible to salvage additional drawings from backup tapes, which were imported into SharePoint and then copied into Word.
However, this process was lengthy and accompanied by daily pressure.
Setting up the right communication
The manufacturing company quickly understood the importance of each and every decision and assembled a crisis team of experts. It consisted of the management of the company under attack and experts from GAMO, each with precisely defined responsibilities and tasks.
Since the Exchange server was down, merchants and buyers started communicating with customers through private emails and phone contacts. Apart from these, no other electronic communication existed.
“We already had a requirement ready to move from Exchange server to the cloud to Office 365. After these events, it was necessary for GAMO to speed up the implementation – the installation of the first set of emails in M365,” says the IT manager at the time. Within a week, they received the first set of emails and started communicating through them gradually. “That was a signal to our customers as well that we were going live. Gradually, we rolled out other servers in the cloud.”
They did not lose a single customer or supplier. “And this is mainly due to properly set up open communication inside and out. Had it not been for the willingness of the suppliers, the helpfulness of the customers and the quick connection with the specialists, things could have turned out very differently,” adds a member of the management of the production company.
Employee = the foundation of a good company
As far as the staff is concerned, there has been no underestimation of communication in a crisis situation either. The company had around 350 employees at the time, with whole families working there. The problem did not really arise, but it could have. An outage of the attendance and payroll systems or the internet banking that every manager relies on to process and pay wages can make life very difficult.
Moreover, failure to pay the money could have posed a real threat to the livelihoods of families in the region. “This is where the loyalty and cohesion of our staff has been demonstrated. They were willing to work overtime, for which I am extremely grateful. Our company has always taken a pro-family approach, and this has helped us a lot in this difficult situation,” says a member of the company’s management team.
He also appreciates the work of the payroll clerk, who has been with the company for 20 years. Although she had the payroll documents in the information system, she printed everything out for checking in the “old school” way.
“Another fortunate thing was that our company does not have a high turnover. A colleague from the team remembered how wages were paid before IT systems – using a coin bank,” explains a member of the company’s management team about the next step in the solution. But it was not at all easy to manage a coin bank for more than three hundred people, budget how many notes and coins were needed for payroll, and then report the cash withdrawals to the bank. “Then it was also necessary to divide the whole amount, to ‘suck up’ exactly what was due to a particular employee, and there was also a not insignificant problem with security. Our female employees were working on this day and night, and these were not small sums. These are also things one has to think about when the whole system goes down.”
Of course there were daily and regular meetings of the plant manager with the production managers for the information of all his people.
Communication with the attacker
Initially, the company flatly refused to pay the ransom. However, they realized the importance of some servers, so they tried to negotiate with the cyber attacker.
“We managed to save some data, we tried to negotiate just to pay for the ones that don’t work. We also asked the attacker to send us proof that he can decrypt them as well. The answers from him were very curt. He had our system perfectly mapped and if we asked him to decrypt a server for us to demo, he would supply the least important one, the test one, which would be encrypted back after a while,” recalls the IT manager.
In the end, the striker was paid nothing. “We found a person who managed to bypass the encryption of the backups on the other servers we needed.”
They also avoided a problem with the law. Cyber attackers usually demand ransom in the form of cryptocurrency, which is associated with the problem of entering the transaction into the books in a legal way. There is then the risk of a fine from the tax authorities, as it is not possible to prove what the money was used for. No attacker will issue an invoice for these services.
“The cyber attacker is a common criminal and the data is stolen from him. What he is after is the money of the attacked company. Counting on the company to get the data back is naive,” says Ľubomír Kopáček, a cyber security specialist.
The attacker’s priority is usually to encrypt the data, and nobody cares if he can decrypt it. Decrypting a piece of data “for show” is completely untrustworthy and tells us nothing. “Another argument for not paying is that if you pay and do not do a perfect ‘clean-up’ of the compromised systems at the same time, you can count on the same incident happening again soon,” concludes the specialist.
“We agree that care must be taken to thoroughly inspect and clean systems. We strictly followed the procedure of restoring “broken” server backups, with great care we first checked the copied system databases and then used them again,” adds the IT manager of the manufacturing company.
Even if you don’t pay, the attack is worth thousands
Although the company in question did not pay anything to the criminal, the cost of the cyber attack is estimated at around half a million EUR. The technical costs alone, including services, cost the company 142 thousand EUR. The rest was mainly made up of wages and the work of people who had to work overtime. They also had to recruit around 30 new workers to make up for production time delays.
The extra amount does not include stress, logistics and only seemingly partial “little things”. For example, it was not easy to get hold of technology in the metallic period. In the spring of 2021, there were several attacks going on simultaneously, and the system of an order going out one day and a courier bringing the technology the next day did not apply. “At the first moment, we were trying to buy NASko. When it was finally in stock, we sent a car to Bratislava – no quotes, just phone calls – to get one of the last ones. That was on a Friday, and by Saturday there were no more devices available,” describes a stressful period for a member of the management.
After 4 months they are sleeping better, but they are not at the finish line
Although the company has managed to get a lot of things up and running successfully, not everything is installed and working yet, nor is the complete infrastructure deployed. However, three pilot production information systems have been salvaged.
All the other stages that the company intends to improve technically are in the process of being implemented. “It will take a long time. We need to replace the original firewalls with new and more secure ones, raise awareness for users about resilience and proper use of the devices, and so on. It’s a never-ending story. Technology is improving and attackers are always ahead,” concludes the IT manager.
To report or not to report an assault to the police?
The attacked company also consulted with GAMO about whether to report the attack. Here the answer is clear. It is more than unreasonable to listen to an attacker who also blackmails the subject by telling them not to contact the police or a third party.
GAMO’s cyberexpert Ľubomír Kopáček says: “It is definitely wise to report the incident. If the victim of an attack is regulated by the Cybersecurity Act and the incident is assessed as serious, it is even his or her legal obligation to do so.” He also recommends filing a criminal report. “Not that the police will be able to find out anything further or even investigate, but it is useful to have an official record of the whole matter.” This is important in case of further problems with, for example, the tax office, social security, health insurance or others. “Remaining silent during an incident is not only unwise but also illegal in most civilised countries and can cost the company additional money.”
In the European Union, there are various decrees and laws on how to deal with a security incident. For example, in the case of negligence in relation to a security incident (failure to report or deal with it), the Cybersecurity Act on Administrative Offences can impose a fine ranging from EUR 300 to 300 thousand.
Cloud = the right solution and the future
The attacked company was correct in its rejection of the attacker’s demands and in all points of crisis management. Not only did it turn to specialists and listen to their advice early on in the solution, but it also maintained a good reputation in its communications with both suppliers and employees.
However, her story is a prime example that even if you regularly back up your data on servers, you can’t be sure that it’s really safe. To keep your company protected too, you need expert solutions tailored to the specific needs of each individual entity. By moving to a secure cloud, you can be sure that it will be taken care of to the highest standard. This removes one of the most important tasks of modern business from the company’s management and allows them to focus 100 percent on their core business.
