GOLDMAN SACHS
DEVELOPER BLOG
Insights from the Goldman Sachs engineering team
November 25, 2024
Scaling Open Policy Agent (OPA) to Offer a Centrally Managed Cloud Entitlements Service
Ashwin Balakrishna, Vice President; Kamlakant Shukla, Vice President
Cloud Entitlements Service (OCES) was launched in August 2021 as a Firmwide policy-based, attribute-driven, cloud-native entitlements platform. OCES uses open source Open Policy Agent (OPA) as the decision engine. This post will describe how we scaled OPA to offer a centralized managed service that supports a diverse set of customer needs across all of the firm's business lines.
October 4, 2024
Synthetic Data Generator for Financial Contracts
Jaimita Bansal, Vice President; Stefano Stefani, Partner and Tech Fellow
Over the Counter (OTC) derivative contracts from trades booked internally at Goldman Sachs are long, semi-structured and highly polymorphic. Given the dynamic nature of these contracts, the queries needed are fast-moving. We have created a synthetic data generator which produces anonymized data with the same statistical, polymorphic, and structural properties as production contractual data, while also preserving the primary key/foreign key relationships across datasets.
July 9, 2024
How Transaction Banking Implemented an Instant Payment Architecture for High-Throughput, Low-Latency and 24x7 Availability
Shahin M. Shahin, Vice President and Tech Fellow; Krishna Yalamanchi, Vice President
Payment processing sits at the heart of Goldman Sachs Transaction Banking (TxB). With instant payments, funds are required to move between sender and receiver within seconds. On top of this, processing must be available 24x7x365 and payments occur in high volumes at peak times. In this post, we dive into how TxB utilizes products and services offered by Amazon Web Services (AWS) to achieve a scalable architecture with high availability and millisecond latency per transaction.
June 5, 2024
Introducing Legend GraphQL
Abhishoya Lunavat, Associate; Anand Matta, Vice President
We are pleased to announced that Legend now provides a GraphQL interface for querying modelled data. GraphQL streamlines the process of accessing data from multiple sources, paving the way for efficient client-server interactions. In this post, we discuss building a GraphQL service with Legend, shedding light on its role in streamlining the SDLC while providing a governed server-side runtime.
May 9, 2024
Using NLP to Purposefully Articulate Software Changes
Richard Mwaba, Associate; Raj Patel, Vice President; Hoshil Sejpal, Vice President and Tech Fellow
Engineers at Goldman Sachs are making thousands of software changes daily. One responsibility we have is to review changes made by others. When confronted with a request with an unclear title, we are often left contemplating the purpose of the change. In this post, we examine guidelines that resulted in impactful change titles, why purposeful titles are important, and the tooling we’ve developed utilizing NLP to validate change titles to ensure they adhere to these practices.
October 23, 2023
SecDb Observability Journey
Bruno Salazar, Vice President and Tech Fellow, Engineering
SecDb is Goldman Sachs’ platform for trading, valuation, and risk management which has been running for over 30 years. With over 13,000 daily end users across the firm, an abrupt change to how SecDb operates could impact critical bank operations. Over the past three years the SecDb team has created a more resilient and reliable platform through an uplift to their monitoring and observability.
December 9, 2022
Supporting the Next Generation of Software Engineers
Marco Argenti, CIO and Irfan Hussain, Chief Operating and Strategy Officer, Engineering
Goldman Sachs, like nearly any organization nowadays, large or small, runs on software. Attracting and retaining great developers, from a broad set of backgrounds and life experiences, is critical to our Engineering Division and our firm overall. We have a vested interest in supporting the next generation of software engineers.
November 9, 2022
Integrating Legend with Databricks' Lakehouse for Financial Services
Ephrim Stanley, VP, Data Engineering
We are excited to announce the integration of the Databricks Lakehouse platform with Legend. This contribution from the Databricks team is a great example of the spirit of FINOS - collaboration and innovation in the financial services industry via open source software. Databricks is also a member of FINOS. In this blog post, we will start with a primer on Legend's relational data modeling and data access capabilities. We will then move onto a quick discussion about the use of data models to establish consistent data vocabularies across applications, platforms, and organizations. Finally, we will wrap up with a peek into work streams that are underway in the Legend project.
September 29, 2022
Sustainability in Action: Diving into ESG Data Engineering at Goldman Sachs
Nilabja Ghosh Chowdhury, VP Data Design and Curation; Abhishek Narang, MD Data Design and Curation
Goldman Sachs has a long-standing commitment to address the impacts of climate change and drive the transition to a low-carbon economy. Over 15 years ago, the firm led the industry as one of the first major banks to acknowledge the scale and urgency of climate change. Read more about how our developers are enabling that transition.
September 20, 2022
How Legend has Empowered Global Markets Engineering at Goldman Sachs and the Derivatives Industry as a Whole
Shilpi Singh, Mariania Cassimiro, Prashant Nirwal, Toh Newin, Tom Barkes, Eric Esterkin
Given the growing volume and complexity of the financial service data in combination with an increased demand for actionable data from stakeholders, the firm developed Legend, an open source data platform that breaks down data silos and connects business and technology teams resulting in improved data governance for automation, transparency and consistency.
August 18, 2022
Goldman Sachs' Open Source Program Office, One Year In
Rohan Deshpande, Managing Director, App Foundry and Open Source
Goldman Sachs runs on Open Source. In April 2021, we took the opportunity to refresh our open source strategy with input from engineers from across the firm. The first initiative we decided to take on was the creation of an Open Source Program Office (OSPO), which formally launched in August 2021.
August 5, 2022
How to Think Like a Programmer Part Three
Denis Akinmolasire
In 2020 the EMEA BE (Black Engineers) network created a "How to Think Like a Programmer" guide. Following a presentation that we did as part of the GS Decoded series we received feedback that we should look to externalize the content to make it available for university and colleague students to allow them to assimilate to industrial software engineering easier enabling a smoother transition once they complete their studies. Now in Part Three, we'll tackle production issues by recreating a scenario in a test environment.
July 28 2022
Legend + Snowflake Native Apps = Fast, Easy, Secure Access to Data
Neema Raphael, Chief Data Officer; Abhishek Narang, Managing Director, Data Engineering
Learn how engineers at Goldman Sachs and Snowflake partnered to unlock native Snowflake performance on top of Legend APIs.
July 14, 2022
Using Entitlements for Privileged Access to APIs and Applications in a Cloud Environment
Ashwin Balakrishna, Vice President, Identity and Access Management in Core Engineering and Balasubramanian Sakthivel, Vice President, Global Investment Research
As Goldman Sachs continues to expand its footprint on the public cloud, it is imperative that the firm protects its data. This commitment has led to the genesis of an Open standards-based Cloud Entitlements Service (OCES). It offers a firmwide policy-based, attribute-driven entitlements platform.
July 13, 2022
Software Craft Series: The Importance of Deleting Unused Code
Darshan Mehta, Vice President, Core Engineering
Following the initial post on the Importance of software craft at Goldman Sachs, this blog post covers an important aspect of software quality and its commercial impact on software development and delivery: deleting unused code.
June 30, 2022
How to Think Like a Programmer (Part Two)
Denis Olasehinde Akinmolasire, Vice President, Data Engineering
"I have a business problem, where do I start?" we introduced this blog series with the aim of helping developers that are new to the software engineering industry and highlight potential challenges that they may encounter as they begin their journey into this field. The next parts will focus on specific technical areas of software development. This part will cover how to model solutions using domain driven development.
June 23, 2022
Chaos Testing an Application on AWS
Sindhuja Durai, Vice President, Consumer Engineering
Well-maintained applications usually have generous test coverage, spread across unit tests, integration tests and performance test suites. Despite this coverage, incidents in production can be due to unfavorable operational scenarios ranging from infrastructure and network faults to unexpected traffic patterns. Read about our lessons learned from integrating chaos testing with the Continuous Integration/Continuous Delivery (CI/CD) process.
June 16, 2022
Building Automated Scalable Network Policy Management: A Look into PINACL
Kishore Kumar Anand, VP, Engineering and Irene Nydees, VP, Engineering
At Goldman Sachs, the infrastructure resources in our cloud can be managed using Infrastructure-as-Code (IaC). The engineering teams at Goldman Sachs request these resources, via code, on-demand and expect the provisioning to be completed in a matter of minutes. The various infrastructure resources include compute, network and storage along with other services. This post will explore how network connectivity requirements are fulfilled by PINACL.
June 7, 2022
The Evolution Of Digital Assets with Mathew McDermott, Global Head of Digital Assets, Goldman Sachs
Rob Underwood, Global Open Source Lead, and Felix Yip, Global Head of Digital Assets Engineering, Goldman Sachs
Recently, Mathew McDermott, Goldman Sachs' Global Head of Digital Assets, joined Jeremy Allaire, Circle's Co-Founder and CEO, on “The Money Movement” podcast. Mathew discusses with Jeremy the tokenization of assets, smart contracts, the role of open source, and more.
June 3, 2022
Key Takeaways from the Open Source Software Security Summit II
Monica Maher, Vice President, Cyber Defense; Tim Douglas, Managing Director, SDLC & Runtime Production Engineering; Rob Underwood, Vice President, Global Open Source Program Lead
The Summit brought together security executives from technology companies, government, and open source communities to announce and discuss the Open Source Software Security Mobilization Plan.
May 27, 2022
Join Goldman Sachs at Perspektywy Women in Tech Summit Warsaw
Lydia Whicher, Associate, EMEA Engineering
We are delighted to sponsor Perspektywy’s Women in Tech Summit in Warsaw (June 7-8), for the fifth year. As the largest technology conference for women in Europe, this year’s hybrid edition will gather more than 5,000 women to hear from industry leaders, share ideas, and grow their networks. Attendees will enjoy a packed agenda. If you are attending, we would love to invite you to join our sessions.
April 20, 2022
Forecasting Capacity Outages Using Machine Learning to Bolster Application Resiliency
Aditya Arora, Associate, Production Runtime Experience
Discover how a novel forecasting pipeline which includes a patent pending change point detection algorithm is used to predict capacity outages ahead of time.
April 8, 2022
Providing 99.9% Availability and Sub-Second Response Times with Sybase IQ Multiplexes by Using HAProxy
Karan Ahluwalia, Vice President, Controllers Engineering and Jack Griffith, Vice President, Controllers Engineering
Learn how our Controllers Engineering team improved resource utilization, saved time during database failovers, and enhanced developer efficiency by using HAProxy.
April 2, 2022
Increased Velocity, Reduced Risk: Non-Disruptive Network Upgrades at Scale
Josh Ricklis, Vice President, Network Engineering
Networks are expected to be available 24/7, even when undergoing maintenance. Explore how we keep our network devices patched without causing noticeable client impact.
March 24, 2022
Applications Now Open for Black Codher Bootcamp
Lydia Whicher, Associate, EMEA Engineering
Goldman Sachs Engineering teams in Birmingham (UK) have partnered with Niyo Network and Coding Black Females to select 10 aspiring Black women coders based in the West Midlands, to take part in a bootcamp sponsored by Goldman Sachs.
March 16, 2022
Building Multi-Region Resiliency with Amazon RDS and Amazon Aurora
John Gorry, Vice President, Transaction Banking Cloud Engineering; Naga Sai Aakarshit Batchu, Vice President, Transaction Banking Cloud Engineering; Ankit Kathuria, Vice President, Transaction Banking Security Engineering
Explore how Transaction Banking Engineering expanded native AWS-provided resiliency options by designing and building a mechanism for failing over their relational databases in a scalable and secure way between AWS regions, along with the database credentials which control application access.
March 11, 2022
Meet the Green Software Foundation Organizational Leads: Sarah Hsu of Goldman Sachs
Green Software Foundation
Sarah Hsu is an Associate at Goldman Sachs working on Site Reliability Engineering. She is also chairing the project on revising and extending the Principles of Green Software Engineering to reflect the evolution the field has seen since the original principles were created in 2019.
March 3, 2022
Startup Foundations: Exploring AWS Options for Serverless APIs
Mark Perkin, Vice President and Architect, GS Accelerate
X25 seeks to provide a digital-first, cloud-based solution to transform the financial product landscape. Explore how the X25 team laid the foundation of their startup by exploring an AWS serverless solution for their client-facing APIs.
February 23, 2022
Monorepos from a Risk Perspective
Karthik Ram, Vice President and Executive Director, Cloud Security and Architecture
The use of monorepos can help companies save time on management, testing, and debugging. Learn how to reap the benefits while still keeping your codebase secure.
February 16, 2022
Driving Developer Productivity via Automated Dependency Tracking
Ankhuri Dubey, Managing Director, CI/CD Engineering
With the proliferation of reusable libraries, one of the challenges that has come about is how to properly manage library dependencies. Learn how the Continuous Integration/Continuous Delivery (CI/CD) team launched support for dependency tracking.
February 9, 2022
Enabling Highly Available Trino Clusters at Goldman Sachs
Ramesh Bhanan, Vice President; Siddhant Chadha, Associate; Sumit Halder, Vice President; and Suman Baliganahalli Narayan Murthy, Vice President - Business Platform Engineering
Discover how the Business Platform Engineering team solved the challenge of making varied datasets from different sources available in a central location for analytics purposes.
January 31, 2022
The Importance of Software Craft and Quality to Goldman Sachs
Hoshil Sejpal, Vice President and Tech Fellow, Storage Engineering
The ability to release changes to software rapidly and reliably is critical to the function of software systems at Goldman Sachs. Learn how a focus on Software Craft ensures a high degree of quality, control, and assurance.
January 12, 2022
Infrastructure and the Command Chain Pattern
Eugeniu Zaharia, Vice President, Storage Engineering
Explore how the Storage Engineering team reduced the amount of time spent on support and eliminated the need for manual intervention in their production environment.
December 16, 2021
Harnessing the Power of Machine Learning to Improve Data Lake Client Happiness
Jaimita Bansal, Vice President, Data Lake Engineering
The Data Lake platform scales to run hundreds of thousands of ingestions and more than a million exports per day.
December 16, 2021
Goldman Sachs Wins Two Awards at Banking Tech Awards 2021
Zach Stevens - Transaction Banking; Eric Esterkin and Sarah Jankowski - Data Engineering
Awards included Best Fintech Partnership and Best Use of Cloud.
December 15, 2021
Six Degrees of Symphony - A Conversation with Ffion Acland about Open Source in Financial Markets
Christopher Satch, Vice President, Developer Experience
Listen to Ffion Acland, Global Co-Manager of Data Models and Governance, as she describes why open source is vital for financial services innovation.
December 8, 2021
Delivering Dynamic and Engaging Machine Learning-Driven Investment Research Experiences Using Amazon SageMaker
Fan Liu, Pravallika Etoori, Henna Dialani, and Alex Woolley - GIR Engineering
Learn how our Global Investment Research team used machine learning to improve the client experience.
December 2, 2021
Mobile CICD with EC2 macOS
Rohan Deshpande - Open Source and Mobile Platform; Anand Biligiri - Mobile Platform
How a streamlined, centralized, and automated process was created.
December 2, 2021
Observability at Scale
Graeme Bennett, Managing Director, Site Reliability Engineering
In 2020, the Site Reliability Engineering (SRE) team began a significant initiative to build a new generation of centralized observability platforms to further accelerate adoption of SRE practices across Goldman Sachs.
December 1, 2021
How to Think Like a Programmer (Part One)
Denis Olasehinde Akinmolasire, Vice President, Data Engineering
This series will aim to encourage readers to think about how to program applications that scale to industry standards, while learning helpful software engineering techniques along the way.
November 30, 2021
Open Source Strategy Forum NYC Recap
Rosalie Bartlett, Developer Advocacy Lead
The event brought together individuals from the financial services, technology, and open source communities. Several presentations were shared by Goldman Sachs.
November 27, 2021
Q and A with Pierre De Belen on our Open Source Platform, Legend
Pierre is the head architect of Goldman Sachs’ open source data management platform, Legend, which was recognized by the 2020 American Financial Technology Awards as Best Collaboration Initiative.
November 27, 2021
Case Study - Managing Page URLs in a Docs-as-Code Based Site Using a Rich UI
Neil Boddy, Vice President, Developer Relations
Learnings from creating a scalable solution for a Docs-as-Code site.
November 22, 2021
Our Open Source Program Office and the TODO Group
Rob Underwood, Bella Wiseman - Open Source Program Office
Why we created an Open Source Program Office (OSPO) and why we joined the TODO Group.
November 19, 2021
Engineering Principles of the GS Design System
Miriam Teitelbaum, Vice President, Developer Experience
The Design System defines the visual styles, interaction patterns, and user experience guidelines to build modern websites and products for desktop and mobile experiences.
November 19, 2021
Measuring Customer Happiness Across 1,000s of Independent Databases
Brett Carrington, Vice President, Middleware Engineering
How Goldman Sachs measures availability across thousands of databases.
November 19, 2021
Builds Today, Breaks Tomorrow: The Mystery of the Disappearing NPM Dependency
Bella Wiseman, Lead Engineer, Open Source Program Office
How a minor change in an open source library had unexpected impact to multiple apps.
November 19, 2021
Announcing CatchIT - Source Code Secret Scanner
Nima Darivandpour, Gunwant Singh, Aniruddha Bhattacharjee - Tech Risk Advisory
CatchIT was recently open sourced and contributed to FINOS (Fintech Open Source Foundation).
October 14, 2021
The Open Source Security Foundation
Atte Lahtiranta, CTO, Goldman Sachs
Announcing Goldman Sachs' commitment to the Open Source Security Foundation.
September 23, 2021
Hello World (from co-CIO Marco Argenti)
Marco Argenti, Co-CIO, Goldman Sachs
Announcing the new Goldman Sachs Developer Blog.
September 23, 2021
Building Platforms for Data Engineering
Neema Raphael, Pierre De Belen, Eric Esterkin, Sarah Jankowski, Beeke-Marie Nelke - Data Engineering
Legend is an open source data platform created by Goldman Sachs.
Solutions
Curated DataData Management Security MasterData AnalyticsPlotTool ProPortfolio AnalyticsGS QuantTransaction BankingGS DAP®
Docs
Data ServicesPricing & Risk ServicesIndex ServicesHedging ServicesPortfolio ServicesContent ServicesRIA CustodyCommission ManagerTransaction BankingGS SelectGS QuantPlotTool Pro
Learn More
BlogOpen SourceData Partnersdatonomy™CareersAbout Goldman Sachs
Goldman Sachs DeveloperPrivacy and CookiesGS Terms & ConditionsRegulatory DisclosuresSecurity
GS DAP® is owned and operated by Goldman Sachs. This site is for informational purposes only and does not constitute an offer to provide, or the solicitation of an offer to provide access to or use of GS DAP®. Any subsequent commitment by Goldman Sachs to provide access to and / or use of GS DAP® would be subject to various conditions, including, amongst others, (i) satisfactory determination and legal review of the structure of any potential product or activity, (ii) receipt of all internal and external approvals (including potentially regulatory approvals); (iii) execution of any relevant documentation in a form satisfactory to Goldman Sachs; and (iv) completion of any relevant system / technology / platform build or adaptation required or desired to support the structure of any potential product or activity. All GS DAP® features may not be available in certain jurisdictions. Not all features of GS DAP® will apply to all use cases. Use of terms (e.g., "account") on GS DAP® are for convenience only and does not imply any regulatory or legal status by such term.
Certain solutions and Institutional Services described herein are provided via our Marquee platform. The Marquee platform is for institutional and professional clients only. This site is for informational purposes only and does not constitute an offer to provide the Marquee platform services described, nor an offer to sell, or the solicitation of an offer to buy, any security. Some of the services and products described herein may not be available in certain jurisdictions or to certain types of clients. Please contact your Goldman Sachs sales representative with any questions. Any data or market information presented on the site is solely for illustrative purposes. There is no representation that any transaction can or could have been effected on such terms or at such prices. Please see https://www.goldmansachs.com/disclaimer/sec-div-disclaimers-for-electronic-comms.html for additional information.
Transaction Banking services are offered by Goldman Sachs Bank USA (“GS Bank”). GS Bank is a New York State chartered bank, a member of the Federal Reserve System and a Member FDIC.
Mosaic is a service mark of Goldman Sachs & Co. LLC. This service is made available in the United States by Goldman Sachs & Co. LLC and outside of the United States by Goldman Sachs International, or its local affiliates in accordance with applicable law and regulations. Goldman Sachs International and Goldman Sachs & Co. LLC are the distributors of the Goldman Sachs Funds. Depending upon the jurisdiction in which you are located, transactions in non-Goldman Sachs money market funds are affected by either Goldman Sachs & Co. LLC, a member of FINRA, SIPC and NYSE, or Goldman Sachs International. For additional information contact your Goldman Sachs representative. Goldman Sachs & Co. LLC, Goldman Sachs International, Goldman Sachs Liquidity Solutions, Goldman Sachs Asset Management, L.P., and the Goldman Sachs funds available through Goldman Sachs Liquidity Solutions and other affiliated entities, are under the common control of the Goldman Sachs Group, Inc.
© 2024 Goldman Sachs. All rights reserved.