Codecov, a leading supplier of computer language testing services, announced an infiltration of their network by a malicious actor. As a result, thousands of copies of a script capable of exfiltrating Linux environment variables has been distributed to software vendors worldwide. To deal with the Codecov hack, the company says they have engaged a 3rd party security consultant and informed United States federal law enforcement as part of their remediation efforts.
The Codecov hack is a software supply chain attack similar to SolarWinds, but potentially wider in scope because the Codecov attack is at an earlier step in the supply chain than SolarWinds.
Codecov Hack News is Breaking
Codecov issued this statement today explaining how the malicious script works, and how to remove the script. The actors are going after environment variables, which are used as an ephemeral storage mechanism for devops scripting. A credential manager, for example, could retrieve keys needed to log onto AWS and then store them in an environment variable. Later on, a continuous integration pipeline utility will access that variable to log onto AWS and perform a function.
Other writers have additional takes:
- Federal investigators looking into breach at software code testing company Codecov – The Verge
- US investigators probing breach at code testing company Codecov – Reuters
- Backdoored developer tool that stole credentials escaped notice for 3 months – Ars Technica
- U.S. Federal Investigators Are Reportedly Looking Into Codecov Security Breach, Undetected for Months – Gizmodo
- Codecov breach triggers fears of another SolarWinds-scale attack – TechRadar
Environment Variables at Risk
This hack goes at a key vulnerability in devops, which is poor credential management. The Codecov hack should encourage devops managers to look more closely at products like Vault from Hashicorp for functions like key regeneration. Check out my earlier post, Okta Goes For Developer Security with Starter Developer Edition, for details on another vendor’s solution to credential management.
For impacted Codecov customers, a key remediation step will be to replace all of the credentials involved in running a CI/CD pipeline. Using a credential management system may help managers to replace all of those keys with a script.
But, We use Salesforce!
This is another warning shot to all organizations that use any kind of developer workstation in their software development pipeline.
It doesn’t seem practical, or desirable, to contain a development pipeline completely within Salesforce. With the Codecov hack, devops managers now need to look at the array of tools available in their organization, and evaluate the tool’s supply chain risk.
Software Supply Chain Risk
Software supply chain risk has moved into the center stage of concerns for IT leaders, and there does not seem to be any easy answers on the horizon. For now, the devops software and services industry needs to step up and start proving they are using security techniques to protect against software supply chain risk.
Vern’s Salesforce Devops Posts
- Use the Salesforce Devops Segmentation Model for IT Success
- Salesforce Devops in Early 2021
- The Ways of the Salesforce Devops: Build or Buy?
- SFDX-CLI Paves The Way for Open-Source Salesforce Devops
About Vernon Keenan
Vernon Keenan (LinkedIn) works as a senior information technology industry consultant based in Oakland, California.
He earned his B.Sc. in Biomedical Engineering at Northwestern University where he programmed a PDP-8 with punched paper tape.
In his 34-year-long career he has been a teacher, SPSS programmer, database administrator, clinical researcher, technology journalist, product marketing manager, market researcher, management consultant, and industry analyst. Most recently he is a telecom operator, cloud architect, Go devops engineer and Salesforce Developer/Architect.
For inquiries about Salesforce strategy briefings or solution architect work please contact Vern directly at +1-510-679-1900 or firstname.lastname@example.org.