GOLDMAN SACHS
DEVELOPER BLOG
Insights from the Goldman Sachs engineering team
November 9, 2022
Integrating Legend with Databricks' Lakehouse for Financial Services
Ephrim Stanley, VP, Data Engineering
We are excited to announce the integration of the Databricks Lakehouse platform with Legend. This contribution from the Databricks team is a great example of the spirit of FINOS - collaboration and innovation in the financial services industry via open source software. Databricks is also a member of FINOS. In this blog post, we will start with a primer on Legend's relational data modeling and data access capabilities. We will then move onto a quick discussion about the use of data models to establish consistent data vocabularies across applications, platforms, and organizations. Finally, we will wrap up with a peek into work streams that are underway in the Legend project.
September 29, 2022
Sustainability in Action: Diving into ESG Data Engineering at Goldman Sachs
Nilabja Ghosh Chowdhury, VP Data Design and Curation; Abhishek Narang, MD Data Design and Curation
Goldman Sachs has a long-standing commitment to address the impacts of climate change and drive the transition to a low-carbon economy. Over 15 years ago, the firm led the industry as one of the first major banks to acknowledge the scale and urgency of climate change. Read more about how our developers are enabling that transition.
September 20, 2022
How Legend has Empowered Global Markets Engineering at Goldman Sachs and the Derivatives Industry as a Whole
Shilpi Singh, Mariania Cassimiro, Prashant Nirwal, Toh Newin, Tom Barkes, Eric Esterkin
Given the growing volume and complexity of the financial service data in combination with an increased demand for actionable data from stakeholders, the firm developed Legend, an open source data platform that breaks down data silos and connects business and technology teams resulting in improved data governance for automation, transparency and consistency.
August 18, 2022
Goldman Sachs' Open Source Program Office, One Year In
Rohan Deshpande, Managing Director, App Foundry and Open Source
Goldman Sachs runs on Open Source. In April 2021, we took the opportunity to refresh our open source strategy with input from engineers from across the firm. The first initiative we decided to take on was the creation of an Open Source Program Office (OSPO), which formally launched in August 2021.
August 5, 2022
How to Think Like a Programmer Part Three
Denis Akinmolasire
In 2020 the EMEA BE (Black Engineers) network created a "How to Think Like a Programmer" guide. Following a presentation that we did as part of the GS Decoded series we received feedback that we should look to externalize the content to make it available for university and colleague students to allow them to assimilate to industrial software engineering easier enabling a smoother transition once they complete their studies. Now in Part Three, we'll tackle production issues by recreating a scenario in a test environment.
July 28 2022
Legend + Snowflake Native Apps = Fast, Easy, Secure Access to Data
Neema Raphael, Chief Data Officer; Abhishek Narang, Managing Director, Data Engineering
Learn how engineers at Goldman Sachs and Snowflake partnered to unlock native Snowflake performance on top of Legend APIs.
July 14, 2022
Using Entitlements for Privileged Access to APIs and Applications in a Cloud Environment
Ashwin Balakrishna, Vice President, Identity and Access Management in Core Engineering and Balasubramanian Sakthivel, Vice President, Global Investment Research
As Goldman Sachs continues to expand its footprint on the public cloud, it is imperative that the firm protects its data. This commitment has led to the genesis of an Open standards-based Cloud Entitlements Service (OCES). It offers a firmwide policy-based, attribute-driven entitlements platform.
July 13, 2022
Software Craft Series: The Importance of Deleting Unused Code
Darshan Mehta, Vice President, Core Engineering
Following the initial post on the Importance of software craft at Goldman Sachs, this blog post covers an important aspect of software quality and its commercial impact on software development and delivery: deleting unused code.
June 30, 2022
How to Think Like a Programmer (Part Two)
Denis Olasehinde Akinmolasire, Vice President, Data Engineering
"I have a business problem, where do I start?" we introduced this blog series with the aim of helping developers that are new to the software engineering industry and highlight potential challenges that they may encounter as they begin their journey into this field. The next parts will focus on specific technical areas of software development. This part will cover how to model solutions using domain driven development.
June 23, 2022
Chaos Testing an Application on AWS
Sindhuja Durai, Vice President, Consumer Engineering
Well-maintained applications usually have generous test coverage, spread across unit tests, integration tests and performance test suites. Despite this coverage, incidents in production can be due to unfavorable operational scenarios ranging from infrastructure and network faults to unexpected traffic patterns. Read about our lessons learned from integrating chaos testing with the Continuous Integration/Continuous Delivery (CI/CD) process.
June 16, 2022
Building Automated Scalable Network Policy Management: A Look into PINACL
Kishore Kumar Anand, VP, Engineering and Irene Nydees, VP, Engineering
At Goldman Sachs, the infrastructure resources in our cloud can be managed using Infrastructure-as-Code (IaC). The engineering teams at Goldman Sachs request these resources, via code, on-demand and expect the provisioning to be completed in a matter of minutes. The various infrastructure resources include compute, network and storage along with other services. This post will explore how network connectivity requirements are fulfilled by PINACL.
June 7, 2022
The Evolution Of Digital Assets with Mathew McDermott, Global Head of Digital Assets, Goldman Sachs
Rob Underwood, Global Open Source Lead, and Felix Yip, Global Head of Digital Assets Engineering, Goldman Sachs
Recently, Mathew McDermott, Goldman Sachs' Global Head of Digital Assets, joined Jeremy Allaire, Circle's Co-Founder and CEO, on “The Money Movement” podcast. Mathew discusses with Jeremy the tokenization of assets, smart contracts, the role of open source, and more.
June 3, 2022
Key Takeaways from the Open Source Software Security Summit II
Monica Maher, Vice President, Cyber Defense; Tim Douglas, Managing Director, SDLC & Runtime Production Engineering; Rob Underwood, Vice President, Global Open Source Program Lead
The Summit brought together security executives from technology companies, government, and open source communities to announce and discuss the Open Source Software Security Mobilization Plan.
May 27, 2022
Join Goldman Sachs at Perspektywy Women in Tech Summit Warsaw
Lydia Whicher, Associate, EMEA Engineering
We are delighted to sponsor Perspektywy’s Women in Tech Summit in Warsaw (June 7-8), for the fifth year. As the largest technology conference for women in Europe, this year’s hybrid edition will gather more than 5,000 women to hear from industry leaders, share ideas, and grow their networks. Attendees will enjoy a packed agenda. If you are attending, we would love to invite you to join our sessions.
April 20, 2022
Forecasting Capacity Outages Using Machine Learning to Bolster Application Resiliency
Aditya Arora, Associate, Production Runtime Experience
Discover how a novel forecasting pipeline which includes a patent pending change point detection algorithm is used to predict capacity outages ahead of time.
April 8, 2022
Providing 99.9% Availability and Sub-Second Response Times with Sybase IQ Multiplexes by Using HAProxy
Karan Ahluwalia, Vice President, Controllers Engineering and Jack Griffith, Vice President, Controllers Engineering
Learn how our Controllers Engineering team improved resource utilization, saved time during database failovers, and enhanced developer efficiency by using HAProxy.
April 2, 2022
Increased Velocity, Reduced Risk: Non-Disruptive Network Upgrades at Scale
Josh Ricklis, Vice President, Network Engineering
Networks are expected to be available 24/7, even when undergoing maintenance. Explore how we keep our network devices patched without causing noticeable client impact.
March 24, 2022
Applications Now Open for Black Codher Bootcamp
Lydia Whicher, Associate, EMEA Engineering
Goldman Sachs Engineering teams in Birmingham (UK) have partnered with Niyo Network and Coding Black Females to select 10 aspiring Black women coders based in the West Midlands, to take part in a bootcamp sponsored by Goldman Sachs.
March 16, 2022
Building Multi-Region Resiliency with Amazon RDS and Amazon Aurora
John Gorry, Vice President, Transaction Banking Cloud Engineering; Naga Sai Aakarshit Batchu, Vice President, Transaction Banking Cloud Engineering; Ankit Kathuria, Vice President, Transaction Banking Security Engineering
Explore how Transaction Banking Engineering expanded native AWS-provided resiliency options by designing and building a mechanism for failing over their relational databases in a scalable and secure way between AWS regions, along with the database credentials which control application access.
March 11, 2022
Meet the Green Software Foundation Organizational Leads: Sarah Hsu of Goldman Sachs
Green Software Foundation
Sarah Hsu is an Associate at Goldman Sachs working on Site Reliability Engineering. She is also chairing the project on revising and extending the Principles of Green Software Engineering to reflect the evolution the field has seen since the original principles were created in 2019.
March 3, 2022
Startup Foundations: Exploring AWS Options for Serverless APIs
Mark Perkin, Vice President and Architect, GS Accelerate
X25 seeks to provide a digital-first, cloud-based solution to transform the financial product landscape. Explore how the X25 team laid the foundation of their startup by exploring an AWS serverless solution for their client-facing APIs.
February 23, 2022
Monorepos from a Risk Perspective
Karthik Ram, Vice President and Executive Director, Cloud Security and Architecture
The use of monorepos can help companies save time on management, testing, and debugging. Learn how to reap the benefits while still keeping your codebase secure.
February 16, 2022
Driving Developer Productivity via Automated Dependency Tracking
Ankhuri Dubey, Managing Director, CI/CD Engineering
With the proliferation of reusable libraries, one of the challenges that has come about is how to properly manage library dependencies. Learn how the Continuous Integration/Continuous Delivery (CI/CD) team launched support for dependency tracking.
February 9, 2022
Enabling Highly Available Trino Clusters at Goldman Sachs
Ramesh Bhanan, Vice President; Siddhant Chadha, Associate; Sumit Halder, Vice President; and Suman Baliganahalli Narayan Murthy, Vice President - Business Platform Engineering
Discover how the Business Platform Engineering team solved the challenge of making varied datasets from different sources available in a central location for analytics purposes.
January 31, 2022
The Importance of Software Craft and Quality to Goldman Sachs
Hoshil Sejpal, Vice President and Tech Fellow, Storage Engineering
The ability to release changes to software rapidly and reliably is critical to the function of software systems at Goldman Sachs. Learn how a focus on Software Craft ensures a high degree of quality, control, and assurance.
January 12, 2022
Infrastructure and the Command Chain Pattern
Eugeniu Zaharia, Vice President, Storage Engineering
Explore how the Storage Engineering team reduced the amount of time spent on support and eliminated the need for manual intervention in their production environment.
December 16, 2021
Harnessing the Power of Machine Learning to Improve Data Lake Client Happiness
Jaimita Bansal, Vice President, Data Lake Engineering
The Data Lake platform scales to run hundreds of thousands of ingestions and more than a million exports per day.
December 16, 2021
Goldman Sachs Wins Two Awards at Banking Tech Awards 2021
Zach Stevens - Transaction Banking; Eric Esterkin and Sarah Jankowski - Data Engineering
Awards included Best Fintech Partnership and Best Use of Cloud.
December 15, 2021
Six Degrees of Symphony - A Conversation with Ffion Acland about Open Source in Financial Markets
Christopher Satch, Vice President, Developer Experience
Listen to Ffion Acland, Global Co-Manager of Data Models and Governance, as she describes why open source is vital for financial services innovation.
December 8, 2021
Delivering Dynamic and Engaging Machine Learning-Driven Investment Research Experiences Using Amazon SageMaker
Fan Liu, Pravallika Etoori, Henna Dialani, and Alex Woolley - GIR Engineering
Learn how our Global Investment Research team used machine learning to improve the client experience.
December 2, 2021
Mobile CICD with EC2 macOS
Rohan Deshpande - Open Source and Mobile Platform; Anand Biligiri - Mobile Platform
How a streamlined, centralized, and automated process was created.
December 2, 2021
Observability at Scale
Graeme Bennett, Managing Director, Site Reliability Engineering
In 2020, the Site Reliability Engineering (SRE) team began a significant initiative to build a new generation of centralized observability platforms to further accelerate adoption of SRE practices across Goldman Sachs.
December 1, 2021
How to Think Like a Programmer (Part One)
Denis Olasehinde Akinmolasire, Vice President, Data Engineering
This series will aim to encourage readers to think about how to program applications that scale to industry standards, while learning helpful software engineering techniques along the way.
November 30, 2021
Open Source Strategy Forum NYC Recap
Rosalie Bartlett, Developer Advocacy Lead
The event brought together individuals from the financial services, technology, and open source communities. Several presentations were shared by Goldman Sachs.
November 27, 2021
Q and A with Pierre De Belen on our Open Source Platform, Legend
Pierre is the head architect of Goldman Sachs’ open source data management platform, Legend, which was recognized by the 2020 American Financial Technology Awards as Best Collaboration Initiative.
November 27, 2021
Case Study - Managing Page URLs in a Docs-as-Code Based Site Using a Rich UI
Neil Boddy, Vice President, Developer Relations
Learnings from creating a scalable solution for a Docs-as-Code site.
November 22, 2021
Our Open Source Program Office and the TODO Group
Rob Underwood, Bella Wiseman - Open Source Program Office
Why we created an Open Source Program Office (OSPO) and why we joined the TODO Group.
November 19, 2021
Engineering Principles of the GS Design System
Miriam Teitelbaum, Vice President, Developer Experience
The Design System defines the visual styles, interaction patterns, and user experience guidelines to build modern websites and products for desktop and mobile experiences.
November 19, 2021
Measuring Customer Happiness Across 1,000s of Independent Databases
Brett Carrington, Vice President, Middleware Engineering
How Goldman Sachs measures availability across thousands of databases.
November 19, 2021
Builds Today, Breaks Tomorrow: The Mystery of the Disappearing NPM Dependency
Bella Wiseman, Lead Engineer, Open Source Program Office
How a minor change in an open source library had unexpected impact to multiple apps.
November 19, 2021
Announcing CatchIT - Source Code Secret Scanner
Nima Darivandpour, Gunwant Singh, Aniruddha Bhattacharjee - Tech Risk Advisory
CatchIT was recently open sourced and contributed to FINOS (Fintech Open Source Foundation).
October 14, 2021
The Open Source Security Foundation
Atte Lahtiranta, CTO, Goldman Sachs
Announcing Goldman Sachs' commitment to the Open Source Security Foundation.
September 23, 2021
Hello World (from co-CIO Marco Argenti)
Marco Argenti, Co-CIO, Goldman Sachs
Announcing the new Goldman Sachs Developer Blog.
September 23, 2021
Building Platforms for Data Engineering
Neema Raphael, Pierre De Belen, Eric Esterkin, Sarah Jankowski, Beeke-Marie Nelke - Data Engineering
Legend is an open source data platform created by Goldman Sachs.