This blog post is a recap of a talk I gave at the Open Source Strategy Forum in November 2021. You can watch the video replay here.
The Mystery
Several teams are reporting sudden unexpected JavaScript build failures. The first clue: all these teams use GatsbyJS, an open source JavaScript framework. The second clue: an error message that states that smartwrap-1.2.5 is unavailable for download due to a GPL 2.0 license.
Despite all these clues, mysteries linger: What is smartwrap, and why are we depending on it? Why did the build suddenly fail now, when it succeeded an hour ago? How could we fix this before today's release? What is GPL, and why are the artifacts licensed under this license not permitted for use?
First - a note on GPL that is a simplification, but will suffice for this blog post. (If you have a real-life legal question, seek answers from a lawyer, not a blog post. :) ) Broadly speaking, there are two categories of open source licenses: permissive and copyleft. Permissive open source licenses, like Apache 2.0 and MIT, allow use of the open source software with very few obligations. By contrast, copyleft licenses, such as GPL, require "derivative" software to distribute its source code to its users (e.g. open source it). Many companies do not permit using GPL dependencies in internally developed software due to the logistical complexity of ensuring that all the derivative source code is promptly open sourced under a GPL compatible license.
Now, on to the build failures! Below is the dependency graph we found when debugging the issue earlier this year. This deeply nested GPL dependency was identified during the build, and prevented the dependent products from being built.
While the dependency graph looks straightforward, teams were nevertheless mystified. They were not aware that they depended on smartwrap, and had never encountered a build failure for it before. No code had changed since the last build, and the open source and SDLC teams confirmed that the GPL policy had not changed either. So what had happened?
To solve the mystery, we first need some background on versioning in npm. In npm, dependencies are specified using semantic versioning in a package.json file. In semantic versioning, the version is specified as x.y.z, with x being the major version, y the minor version, and z the patch version. In the npm world, it is a common practice to allow ranges of dependencies. For example, ^1.0.0 means "1.0.0 or any later minor version" (e.g. 1.1.0, 1.2.0... or 1.13.9). This is so common that the ^ x.y.z is actually the default dependency added to your package.json when you run npm install --save package.
While this is great for keeping your dependencies up-to-date, if handled incorrectly, it can lead to builds that are not reproducible over time. You run the risk of pulling in new versions of dependencies without warning and at any time. Even if you choose to use exact versions in your package.json, you are still impacted by this behavior, as your dependencies almost certainly use version ranges, and the exact version of your transitive dependencies will be ambiguous.
As we discovered during our investigation, a classic example is... Gatsby! Specifically, when we looked at the package.json for gatsby-recipes, we found this:
"dependencies": {
"@babel/core": "^7.12.3",
"@babel/generator": "^7.12.5",
"@babel/helper-plugin-utils": "^7.10.4",
...
"@babel/types": "^7.12.6",
"@graphql-tools/schema": "^7.0.0", <-----
"@graphql-tools/utils": "^7.0.2",
"@hapi/hoek": "8.x.x",
"@hapi/joi": "^15.1.1",
"better-queue": "^3.8.10",
...
In the above incident, we dug down, and found that graphql-tools/schema had released a patch version which added a dependency on a new package value-or-promise. Since graphql-tools/schema and value-or-promise were both licensed under the permissive MIT license, adding this dependency appeared at first glance to be a trivial change. However, deeply nested within the dependency tree of value-or-promise were several GPL open source projects, which are blocked for use within the company, including, smartwrap. Thus, in the course of a couple of hours, several teams had their builds starting to fail, though not a single line of their code had changed.
Mystery solved! Just yesterday, Gatsby did not rely on smartwrap. When the build failures started happening, Gatsby did rely on smartwrap. No change occurred in Gatsby or smartwrap - the only change was that graphql-tools/schema added a new dependency on value-or-promise, which pulled in smartwrap as a transitive dependency. And thus concludes our mystery. The end.
Just kidding! You are most likely not a detective. And while solving mysteries is fun, you likely still have some more questions that will now keep you up at night as a software engineer: How can I stop this from happening to my team? How can I make sure it doesn’t happen right before the big release tomorrow? How did you fix your build? And if you're an open source maintainer: how can I make sure changes to my open source project don’t have unintended impact?
Don't Be Taken By Surprise
Let's start with prevention. While this dependency would come in scope eventually on version upgrades, most teams would not want it to randomly fail their build at an unplanned time when they may be under deadline for a release. Nor would they want the build failing on the server, but succeeding on their local machine. To eliminate the element of surprise, you will want to follow some best practices in your JavaScript projects.
Since npm5, npm by default creates a package-lock.json on the first run of a build in an environment. This contains the exact version of each dependency in your tree. Subsequent builds will download and use the exact versions of the dependencies in package-lock.json, even if a newer dependency version becomes available. Thus, builds will be reproducible, as long as the package-lock.json is not modified. Conversely, if there is no package-lock.json in the project directory, npm will download or reuse any version of each dependency that is compatible with the version range; the exact version used can change depending on when you run the build, and what packages are already installed (Please note: this greatly oversimplifies the algorithm that npm uses to determine the version of a package to use.).
However, generating the lock file is not enough. To ensure reproducible builds in your build pipeline, you must commit your package-lock.json to source control. While many auto-generated files should not be committed to source control, package-lock.json is an exception. The clear consensus and best practice in the node community is to commit this file. Once you do, your builds will be reproducible, and you will be in control of when to update your dependencies. This guidance applies to other lock files as well, like pnpm-lock.yaml, yarn.lock and npm's older shrinkwrap.json.
Checking in your lock file will only work if it doesn't get overridden during your build. In order to ensure this won't happen, make sure that your build pipeline runs npm ci rather than npm install. This provides a number of additional benefits, including faster builds. If you follow these best practices, every change to your dependency graph will have a corresponding change in your lock file, making dependency changes significantly easier to detect and debug.
Quick - Fix the Build!
In the short-term, you can remove your dependency on the package with the license that you are unable to use by pinning to the last known good version of the intermediate dependency (in this case, version 7.1.3 of graphql-tools/schema, the version before value-or-promise was introduced). This is not a long-term solution, as you will be permanently pinned to an old version of a package, which is problematic for security and many other reasons. However, it can save the day in the short-term and is a useful technique to know about.
If you use vanilla npm, there are tools that will edit package-lock.json before each install, providing a quick way to override dependency versions. If you use a monorepo management tool, you may have more options. For example, in Rush, one can override dependency versions globally by following these instructions. The benefits of monorepos shine through here, since you can override versions in one central location, rather than doing it for each package (which would take a while, and delay that release you're trying to get out).
Fix the Underlying Issue
In many cases, the maintainer does not have full insight into the impact that a small pull request can have on their users. Reaching out to the maintainer to collaborate on solving your issue is always an ideal solution. In this case, an issue was raised in value-or-promise, noting that the intermediate dependency changesets/cli (see below) was incorrectly marked as a "dependency" when it should have been a "devDependency". The maintainer fixed the issue within 2 days, and now value-or-promise (and Gatsby) no longer pull in any GPL transitive dependencies. In the graph below from https://www.npmjs.com/package/smartwrap, you'll note the spike, and subsequent decline, in usage of smartwrap, as the dependency was introduced, and then removed.
Maintaining Open Source Packages
If you maintain an open source package that is licensed under a permissive license (e.g. Apache or MIT), and you want it to be widely used by commercial enterprises, here are few things to be aware of. First, adding a new dependency can be a breaking change, particularly if it pulls in a package that has a less permissive license (e.g. GPL). To ensure you are not pulling in licenses that are less permissive, you need to traverse the entire dependency graph. Luckily, deps.dev from Google can help you analyze the dependency graph. This is something you may want to consider automating as well, so that new dependencies are scanned for each new pull request.
That's a Wrap!
If you use npm, your dependency graph is always changing...and that's a good thing! Using some of the best practices above can help you keep your dependencies up-to-date in a controlled manner, and help you navigate unexpected changes to your dependency graph, so that you can keep delivering value to your users....and building awesome software!
See https://www.gs.com/disclaimer/global_email for important risk disclosures, conflicts of interest, and other terms and conditions relating to this blog and your reliance on information contained in it.
Solutions
Curated Data Security MasterData AnalyticsPlotTool ProPortfolio AnalyticsGS QuantTransaction BankingGS DAP®Liquidity Investing¹ Real-time data can be impacted by planned system maintenance, connectivity or availability issues stemming from related third-party service providers, or other intermittent or unplanned technology issues.
Transaction Banking services are offered by Goldman Sachs Bank USA ("GS Bank") and its affiliates. GS Bank is a New York State chartered bank, a member of the Federal Reserve System and a Member FDIC. For additional information, please see Bank Regulatory Information.
² Source: Goldman Sachs Asset Management, as of March 31, 2025.
Mosaic is a service mark of Goldman Sachs & Co. LLC. This service is made available in the United States by Goldman Sachs & Co. LLC and outside of the United States by Goldman Sachs International, or its local affiliates in accordance with applicable law and regulations. Goldman Sachs International and Goldman Sachs & Co. LLC are the distributors of the Goldman Sachs Funds. Depending upon the jurisdiction in which you are located, transactions in non-Goldman Sachs money market funds are affected by either Goldman Sachs & Co. LLC, a member of FINRA, SIPC and NYSE, or Goldman Sachs International. For additional information contact your Goldman Sachs representative. Goldman Sachs & Co. LLC, Goldman Sachs International, Goldman Sachs Liquidity Solutions, Goldman Sachs Asset Management, L.P., and the Goldman Sachs funds available through Goldman Sachs Liquidity Solutions and other affiliated entities, are under the common control of the Goldman Sachs Group, Inc.
Goldman Sachs & Co. LLC is a registered U.S. broker-dealer and futures commission merchant, and is subject to regulatory capital requirements including those imposed by the SEC, the U.S. Commodity Futures Trading Commission (CFTC), the Chicago Mercantile Exchange, the Financial Industry Regulatory Authority, Inc. and the National Futures Association.
FOR INSTITUTIONAL USE ONLY - NOT FOR USE AND/OR DISTRIBUTION TO RETAIL AND THE GENERAL PUBLIC.
This material is for informational purposes only. It is not an offer or solicitation to buy or sell any securities.
THIS MATERIAL DOES NOT CONSTITUTE AN OFFER OR SOLICITATION IN ANY JURISDICTION WHERE OR TO ANY PERSON TO WHOM IT WOULD BE UNAUTHORIZED OR UNLAWFUL TO DO SO. Prospective investors should inform themselves as to any applicable legal requirements and taxation and exchange control regulations in the countries of their citizenship, residence or domicile which might be relevant. This material is provided for informational purposes only and should not be construed as investment advice or an offer or solicitation to buy or sell securities. This material is not intended to be used as a general guide to investing, or as a source of any specific investment recommendations, and makes no implied or express recommendations concerning the manner in which any client's account should or would be handled, as appropriate investment strategies depend upon the client's investment objectives.
United Kingdom: In the United Kingdom, this material is a financial promotion and has been approved by Goldman Sachs Asset Management International, which is authorized and regulated in the United Kingdom by the Financial Conduct Authority.
European Economic Area (EEA): This marketing communication is disseminated by Goldman Sachs Asset Management B.V., including through its branches ("GSAM BV"). GSAM BV is authorised and regulated by the Dutch Authority for the Financial Markets (Autoriteit Financiële Markten, Vijzelgracht 50, 1017 HS Amsterdam, The Netherlands) as an alternative investment fund manager ("AIFM") as well as a manager of undertakings for collective investment in transferable securities ("UCITS"). Under its licence as an AIFM, the Manager is authorized to provide the investment services of (i) reception and transmission of orders in financial instruments; (ii) portfolio management; and (iii) investment advice. Under its licence as a manager of UCITS, the Manager is authorized to provide the investment services of (i) portfolio management; and (ii) investment advice.
Information about investor rights and collective redress mechanisms are available on www.gsam.com/responsible-investing (section Policies & Governance). Capital is at risk. Any claims arising out of or in connection with the terms and conditions of this disclaimer are governed by Dutch law.
To the extent it relates to custody activities, this financial promotion is disseminated by Goldman Sachs Bank Europe SE ("GSBE"), including through its authorised branches. GSBE is a credit institution incorporated in Germany and, within the Single Supervisory Mechanism established between those Member States of the European Union whose official currency is the Euro, subject to direct prudential supervision by the European Central Bank (Sonnemannstrasse 20, 60314 Frankfurt am Main, Germany) and in other respects supervised by German Federal Financial Supervisory Authority (Bundesanstalt für Finanzdienstleistungsaufsicht, BaFin) (Graurheindorfer Straße 108, 53117 Bonn, Germany; website: www.bafin.de) and Deutsche Bundesbank (Hauptverwaltung Frankfurt, Taunusanlage 5, 60329 Frankfurt am Main, Germany).
Switzerland: For Qualified Investor use only - Not for distribution to general public. This is marketing material. This document is provided to you by Goldman Sachs Bank AG, Zürich. Any future contractual relationships will be entered into with affiliates of Goldman Sachs Bank AG, which are domiciled outside of Switzerland. We would like to remind you that foreign (Non-Swiss) legal and regulatory systems may not provide the same level of protection in relation to client confidentiality and data protection as offered to you by Swiss law.
Asia excluding Japan: Please note that neither Goldman Sachs Asset Management (Hong Kong) Limited ("GSAMHK") or Goldman Sachs Asset Management (Singapore) Pte. Ltd. (Company Number: 201329851H ) ("GSAMS") nor any other entities involved in the Goldman Sachs Asset Management business that provide this material and information maintain any licenses, authorizations or registrations in Asia (other than Japan), except that it conducts businesses (subject to applicable local regulations) in and from the following jurisdictions: Hong Kong, Singapore, India and China. This material has been issued for use in or from Hong Kong by Goldman Sachs Asset Management (Hong Kong) Limited and in or from Singapore by Goldman Sachs Asset Management (Singapore) Pte. Ltd. (Company Number: 201329851H).
Australia: This material is distributed by Goldman Sachs Asset Management Australia Pty Ltd ABN 41 006 099 681, AFSL 228948 (‘GSAMA’) and is intended for viewing only by wholesale clients for the purposes of section 761G of the Corporations Act 2001 (Cth). This document may not be distributed to retail clients in Australia (as that term is defined in the Corporations Act 2001 (Cth)) or to the general public. This document may not be reproduced or distributed to any person without the prior consent of GSAMA. To the extent that this document contains any statement which may be considered to be financial product advice in Australia under the Corporations Act 2001 (Cth), that advice is intended to be given to the intended recipient of this document only, being a wholesale client for the purposes of the Corporations Act 2001 (Cth). Any advice provided in this document is provided by either of the following entities. They are exempt from the requirement to hold an Australian financial services licence under the Corporations Act of Australia and therefore do not hold any Australian Financial Services Licences, and are regulated under their respective laws applicable to their jurisdictions, which differ from Australian laws. Any financial services given to any person by these entities by distributing this document in Australia are provided to such persons pursuant to the respective ASIC Class Orders and ASIC Instrument mentioned below.
- Goldman Sachs Asset Management, LP (GSAMLP), Goldman Sachs & Co. LLC (GSCo), pursuant ASIC Class Order 03/1100; regulated by the US Securities and Exchange Commission under US laws.
- Goldman Sachs Asset Management International (GSAMI), Goldman Sachs International (GSI), pursuant to ASIC Class Order 03/1099; regulated by the Financial Conduct Authority; GSI is also authorized by the Prudential Regulation Authority, and both entities are under UK laws.
- Goldman Sachs Asset Management (Singapore) Pte. Ltd. (GSAMS), pursuant to ASIC Class Order 03/1102; regulated by the Monetary Authority of Singapore under Singaporean laws
- Goldman Sachs Asset Management (Hong Kong) Limited (GSAMHK), pursuant to ASIC Class Order 03/1103 and Goldman Sachs (Asia) LLC (GSALLC), pursuant to ASIC Instrument 04/0250; regulated by the Securities and Futures Commission of Hong Kong under Hong Kong laws
No offer to acquire any interest in a fund or a financial product is being made to you in this document. If the interests or financial products do become available in the future, the offer may be arranged by GSAMA in accordance with section 911A(2)(b) of the Corporations Act. GSAMA holds Australian Financial Services Licence No. 228948. Any offer will only be made in circumstances where disclosure is not required under Part 6D.2 of the Corporations Act or a product disclosure statement is not required to be given under Part 7.9 of the Corporations Act (as relevant).
FOR DISTRIBUTION ONLY TO FINANCIAL INSTITUTIONS, FINANCIAL SERVICES LICENSEES AND THEIR ADVISERS. NOT FOR VIEWING BY RETAIL CLIENTS OR MEMBERS OF THE GENERAL PUBLIC
Canada: This presentation has been communicated in Canada by GSAM LP, which is registered as a portfolio manager under securities legislation in all provinces of Canada and as a commodity trading manager under the commodity futures legislation of Ontario and as a derivatives adviser under the derivatives legislation of Quebec. GSAM LP is not registered to provide investment advisory or portfolio management services in respect of exchange-traded futures or options contracts in Manitoba and is not offering to provide such investment advisory or portfolio management services in Manitoba by delivery of this material.
Japan: This material has been issued or approved in Japan for the use of professional investors defined in Article 2 paragraph (31) of the Financial Instruments and Exchange Law ("FIEL"). Also, any description regarding investment strategies on or funds as collective investment scheme under Article 2 paragraph (2) item 5 or item 6 of FIEL has been approved only for Qualified Institutional Investors defined in Article 10 of Cabinet Office Ordinance of Definitions under Article 2 of FIEL.
Interest Rate Benchmark Transition Risks: This transaction may require payments or calculations to be made by reference to a benchmark rate ("Benchmark"), which will likely soon stop being published and be replaced by an alternative rate, or will be subject to substantial reform. These changes could have unpredictable and material consequences to the value, price, cost and/or performance of this transaction in the future and create material economic mismatches if you are using this transaction for hedging or similar purposes. Goldman Sachs may also have rights to exercise discretion to determine a replacement rate for the Benchmark for this transaction, including any price or other adjustments to account for differences between the replacement rate and the Benchmark, and the replacement rate and any adjustments we select may be inconsistent with, or contrary to, your interests or positions. Other material risks related to Benchmark reform can be found at https://www.gs.com/interest-rate-benchmark-transition-notice. Goldman Sachs cannot provide any assurances as to the materialization, consequences, or likely costs or expenses associated with any of the changes or risks arising from Benchmark reform, though they may be material. You are encouraged to seek independent legal, financial, tax, accounting, regulatory, or other appropriate advice on how changes to the Benchmark could impact this transaction.
Confidentiality: No part of this material may, without GSAM's prior written consent, be (i) copied, photocopied or duplicated in any form, by any means, or (ii) distributed to any person that is not an employee, officer, director, or authorized agent of the recipient.
GSAM Services Private Limited (formerly Goldman Sachs Asset Management (India) Private Limited) acts as the Investment Advisor, providing non-binding non-discretionary investment advice to dedicated offshore mandates, involving Indian and overseas securities, managed by GSAM entities based outside India. Members of the India team do not participate in the investment decision making process.