Inside a 38,000-person remote work rollout at Goldman Sachs: sleepless nights, assembly lines, and an Amazon-like hub on a Manhattan trading floor

Atte Lahtiranta Goldman Sachs

  • Goldman Sachs had to bulk purchase monitors, hard drives, and internet-connected phones, put them together, and ship them to employees’ homes, according to CTO Atte Lahtiranta. 
  • The bank sent out thousands of these kits during the month of March as part of a plan to get 98% of employees working from home by early April. 
  • To create the kits, the bank turned a trading floor in its Manhattan headquarters and then a data center in northern New Jersey into pop-up logistics centers. 
  • With the first part of the operation complete, Lahtiranta has started thinking about new technologies like virtual reality and ensuring that employee wellness remains a focal point. 
  • Click here for more BI Prime stories.

Goldman Sachs’s six-story trading floor in Manhattan typically hosts traders and engineers, including those who work on an institutional trading platform known as Marquee. Stock and bond traders ply their wares on the floors below. 

In early March, the cavernous space hosted a different kind of activity. In the back, overlooking a Manhattan side street, Goldman’s technology and operations team was doing some of the early work to enable its 38,000-plus employees to work from home. 

The execs were assembling kits of technical gear – large computer monitors, internet-enabled phones, hard drives, and, in some cases, backup power sources and WiFi devices – needed for employees to work remotely. They were then packaging them up and shipping the gear to employees’ homes. 

A parallel effort was underway across the Hudson River at a facility in Jersey City. 

The kit building and other efforts to ready Goldman for an unprecedented shift to working from home fell on the shoulders of Chief Technology Officer Atte Lahtiranta, a newcomer who joined the bank in October from Big Tech.

That experience couldn’t prepare him for the speed and breadth of what was needed. His plans to take a year to get Goldman’s systems up to speed got compressed into just a few weeks. 

“None of us were necessarily experts in logistics,” Lahtiranta said in an exclusive interview with Business Insider. “There was a two-week period where nobody slept a wink.”

On March 8, the firm sent a staff memo enacting its business continuity plan, which involved moving some staff to backup sites and enabling others to work from home. On March 17, the firm encouraged “most of our people” to work from home. By April 3, CEO David Solomon boasted to CNBC that 98% of the firm’s workforce was working from home. 

Much like the advertising industry decades ago, when deals were clinched with liquid lunches off Madison Avenue, Wall Street has depended on face-to-face interactions for much of its 225-year history.  

Senior investment bankers fly to meet clients in Missouri or Milan, while junior employees put in 100-hour weeks at the office for the promise of facetime with senior managers. Traders congregate on desks just feet from each other where they can share information by shouting or gesturing. 

The coronavirus put an end to all that. In just weeks, the industry found itself scrambling to comply with government shelter-in-place orders as the first positive cases of coronavirus began showing up on US shores. 

Goldman’s approach is a microcosm of how corporate America sprang into action.  And as the first phase of the rollout winds down, Lahtiranta has started to worry about employee wellness and cutting-edge technologies like virtual reality. 

Know your role

The first thing Lahtiranta did was divide the company’s workforce into different roles based on the specific requirements of their remote working setup. Investment bankers, traders, engineers and coders, and operations and compliance people each had different needs.  

As the early days of the crisis unfolded, the first of those groups to get his attention were the firm’s traders, grappling with a surge in volatility not seen since the 2008 financial crisis. Doing so would require accelerating a number of technology trials already underway. 

Goldman purchased software licenses for a complex phone system known as a trading turret. Patterned after the hard-wired command centers used in the office to connect hundreds of phone lines to colleagues, clients, and exchanges, the software versions could accomplish much of the same functionality from home. 

The engineering team was still exploring the products’ full feature set when the pandemic forced it to accelerate its roll-out, according to Orla Dunne, Goldman’s London-based global co-head of engineering deployment, runtime and architecture. So far, the bank has installed more than 1,000.

The software turrets can record phone calls on the system’s back end rather than a trader’s computer or phone, making them less reliant on the speed or quality of broadband or mobile phone reception, she said. And it’s compliant with regulations. 

The bank stopped short of hard-wiring physical turrets at traders’ homes, which some peers have chosen to do. The reason, Dunne said, is because they wanted to be thoughtful about bandwidth consumption, knowing there would be incredible demands with entire families staying at home. (Some streaming companies have lowered bandwidth requirements or quality in order to continue serving customers.) 

Traders have also been using Zoom, the videoconferencing app that has become the de facto way for millions of people trapped at home to communicate. (Goldman helped take the company public last year.) In many cases traders are using video links that stay open all day for colleagues to remain in contact. A Zoom trial had been underway. 

As Lahtiranta has started thinking about the future, he says a virtual reality trading desk may be an option for staff who depend on human-to-human interaction to fine tune their intuition and make markets. 

“Is there a next step beyond the Zoom call?” he said. “If we go a little bit more, is it that maybe their social hours should be done in VR. We are not there now, but like I said, across the industry, these last few weeks for everybody has changed quite a lot.”

The CTO was coordinating all this from his home outside Silicon Valley, where his office has a wall very close to the color of Goldman blue, and white shelves filled with books on software coding. Like many, he is trying to home school his children. 

One day, as he was holding a meeting with his direct reports and facilities chief Roy Joseph, he asked a rhetorical question about what else could go wrong. Soon, the rooms of the managers dialing in from Salt Lake City started shaking as a magnitude 5.7 earthquake rippled through the region. Officials from the Utah division of emergency management said it was the largest to hit the state since 1992. No one from Goldman was hurt. 

Softphones

Next up for remote-work setups were Goldman’s investment bankers, the employees who advise on mergers and underwrite debt and equity issues.

The bankers got so-called softphones, which allow for more complicated routing than typical call forwarding techniques. Most importantly, the phones mean that clients can call a Goldman number and have that ring the mobile phones of multiple people, ensuring the call is always answered.

The work wasn’t trivial: Goldman’s telephone systems weren’t set up to handle call forwarding for thousands of people at the same time, and the company had to work closely with its telecom vendors to make the transition. 

An ongoing pilot using Cisco Jabber Mobile app for making calls and sending messages was also accelerated. The bankers also got access to Zoom. 

To have the hardware when Goldman needed it meant thinking ahead about the potential vulnerabilities in its supply chain. 

In a Feb. 25 email, Dunne suggested the engineering team start thinking about its ability to source the equipment it would need if large scale shelter-in-place orders were handed down. By then, Goldman’s Asia units had already gone through lockdowns and had some experience to share. 

Less than a week after Dunne’s email, northern Italy became the first country in Europe to get walloped by the spreading disease. 

“It was like the last day of February, I think, when I took the first call and we had to start shipping to Milan,” Dunne said. “The closer it gets to your doorstep, that’s when it really begins to become very real.”

Goldman was ultimately able to continue working with its vendors to get the hardware it needed. 

Ramping up output

Soon into the kit-making operation, execs realized they needed to increase their output. The operation at 200 West Street and 30 Hudson Street would eventually be merged into a sprawling data center in northern New Jersey, where teams set up huge assembly lines in the space that didn’t house the firm’s enterprise data servers. 

The facility made it easier to take deliveries for pallets of computer screens, for example, and sped up the operation, said Robert Naccarella, head of enterprise tech operations. 

In the beginning, Goldman’s employees were building a kit in about four hours but over time, the bank was able to achieve a pace of making more than 200 kits a day. 

“The first week was a little rough because things were taking long,” Naccarella said. So they started thinking about “something that we would build sequentially that would take four hours, how do we build 40 of them at a time in that same four hours? It took us one to two weeks to figure all of that out,” he said.  

To get them shipped to employees, they relied on everything from the US Postal Service, UPS, FedEx, and even black car service, Naccarella said. In some cases, they had to send it a second time because no one was home to sign for the package. 

“Who knew that we were going to have to kind of become a mini version of Amazon, taking in requests, getting suppliers of all the parts, assembling it, and shipping it out through different methods of transport,” he said. 

With much of the hard part done, Lahtiranta has started considering the long-term effects of working from home. Goldman, like many companies, now provides group fitness classes, read-aloud story time for the children of employees, and mindful breathing and meditation workshops. 

“Wellness is becoming the next big question,” he said.

“It took 10 years for Big Tech to figure out how to maintain working from home when it comes to work-life balance. Now put that on the constrained time frame of a couple weeks,” he said, adding “we have to be thinking about how we make certain, as a finance industry, that we take the learnings of the tech world and apply that super fast so that people can stay efficient, keep their mental balance and remain healthy.”

SEE ALSO: Wall Street’s disaster playbook never included work-from-home trading. Insiders explain how banks rapidly adjusted during one of the most chaotic markets in history.

SEE ALSO: The CEO of a cloud-based trading tech startup that saw a 30% surge in business last month says the coronavirus is a catalyst for overhauling how Wall Street works

SEE ALSO: ‘The best run I’ve ever had’: Inside Wall Street’s coronavirus-fueled trading frenzy, where historic shocks of volatility are creating massive paydays

Join the conversation about this story »

NOW WATCH: Tax Day is now July 15 — this is what it’s like to do your own taxes for the very first time