Well-Architected DevOps with MuseLab D2X
Salesforce’s Well-Architected Framework is a great guide to building better solutions, but there’s a big gap between the high-level theory and the low-level tooling Salesforce provides to implement the theory. This leaves Salesforce customers and partners frustrated and confused. MuseLab’s new open-source D2X framework was built to make it easy to build Well-Architected DevOps using the full capabilities of GitHub. Our new free web application, D2X Launchpad, lets anyone create a new Salesforce project on GitHub fully configured for D2X in seconds.
Table of contents
MuseLab D2X: Turning Theory Into Practice
MuseLab D2X is an open-source collection of tools and configurations designed specifically for Salesforce development projects on GitHub. It’s a culmination of MuseLab’s hands-on experience helping some of the most complex ISVs on the platform implement CumulusCI, Salesforce CLI, GitHub Actions, and GitHub Codespaces. In fact, D2X is the foundation for all of our consulting engagements. We are incredibly proud to share this latest innovation with the Salesforce community.
D2X is more than just a tool — it’s a philosophy. It is about aligning with the Salesforce Well-Architected principle of being Adaptable, both Resilient and Composable. D2X offers:
- D2X Docker Image: Preinstalled with CumulusCI and SF CLI, ensuring that every development environment and every CI/CD build uses a consistent set of up-to-date tools.
- GitHub Actions Workflows: Reusable workflows that implement key components of the MuseLab ISV D2X North Star process for Composable architecture.
- Dev Container Configuration: Create development environments with tooling and orgs preconnected in the browser with GitHub Codespaces or any IDE with a Dev Containers extension such as Visual Studio Code.
- D2X Template: Create new project GitHub repositories preconfigured for D2X.
D2X is built around CumulusCI, the open-source framework created by Salesforce.org for orchestrating Salesforce DevOps workflows. CumulusCI allows developers to automate processes like data seeding, org configuration, and dependency management to create complete product experience recipes. This ensures that individual components of a Salesforce solution can be developed, tested, and deployed in isolation, yet can still be integrated cohesively.
Additionally, we created D2X Launchpad, a free web application that makes it easy to start your next Salesforce development project on GitHub using D2X to align with Well-Architected DevOps.
You can see a full demo of D2X launchpad at: Introducing D2X
Back to the Theory: Salesforce Well-Architected Framework
If you haven’t yet read the Well-Architected Framework, you really should. It’s the culmination of years of working with some of the most experienced Salesforce architects to learn what works and what doesn’t. Much of the discussion of Well-Architected has focused on your solutions themselves. What’s missing is more discussion about how you run multiple solution projects at scale while staying aligned with the Adaptable chapter’s goals of Resilient and Composable. Two specific sections relevant here are Application Lifecycle Management and Packageability.
Applying Application Lifecycle Management (ALM)
Application Lifecycle Management emphasizes the importance of source-based development over traditional org-based development.
Why is this shift crucial? Source-based development allows for a more streamlined, efficient, and transparent development process. It ensures that the codebase remains the single source of truth for your project, enabling teams to track changes, collaborate more effectively, and deploy with confidence. In contrast, when you attempt to use Salesforce org as the ultimate source of truth, even if some of it is represented in version control, it is impossible to maintain consistency and ensure that all changes are accurately tracked and deployed.
Shifting to source-based development allows teams to adopt robust, modern DevOps practices, emphasizing automation, collaboration, and continuous integration/continuous deployment (CI/CD). These practices help ensure that their Salesforce solutions are not only robust and scalable but also easy to maintain and evolve.
Unpacking Packageability
The Packageability section of the Well-Architected Framework explores the importance of creating Salesforce solutions that can be easily deployed to any kind of org.
A well-packaged solution is like a set of building blocks. Each block (or module) is independent, yet when combined, they form a cohesive whole. This modular approach ensures that solutions can be easily customized, scaled, and deployed across different orgs without hiccups.
For DevOps teams, packageability offers flexibility. It means that solutions can be developed, tested, and deployed in smaller chunks, reducing dependencies and risks. Moreover, it ensures that solutions remain adaptable, catering to the ever-evolving needs of businesses.
The Product Delivery Model is a Middle Ground
The entire Salesforce ecosystem is familiar with the Org Development Model. Adoption of the Package Development Model is moving beyond its traditional ISV roots into SI partners and enterprise Salesforce customers. These two models are nearly polar opposites.
In the Org Development Model, developers are responsible for the entire production org’s configuration all the time, in every environment. In the Package Development Model, developers are only responsible for the contents of a package. Yet there are few ISV packages that are usable simply by installing them. Turning a package into a real, usable product experience in an org requires a mix of package development to build packages and org development to configure the package in the org. In other words, a Product is more than a Package.
The Product Delivery Model is the missing middle ground between these polar opposites by defining a product as an “automation recipe” stored in version control. This recipe is used throughout the product lifecycle to deliver comprehensive product experiences to both new and existing orgs.
The most significant advantage of the Product Delivery Model is its ability to create a complete product experience from any branch in source control that can be easily tested, demoed, or delivered to users. By having the entire recipe for product delivery in version control, it becomes the genuine source of truth for the product.
This shifts the collaboration focus from a few persistent orgs that may have some version controlled code mixed up with the org’s other metadata and data and centers it squarely on the version control system, ensuring consistency, scalability, and adaptability.
The Product Delivery Model also provides the foundation for efficient Application Lifecycle Management because it ensures that everything needed to deliver a complete configuration into any org is under source control.
Runtime: the Missing Piece of the Salesforce DevOps Puzzle
The Salesforce community has created many tools to automate parts of an org’s configuration and data. Many projects have automation using these tools in their project repositories. But this automation isn’t truly portable. It’s a set of configurations or scripts that require a place to run that has all the right tooling versions, files, and environment variables set up. Creating new runtime environments is still a time consuming, mostly-manual process (think about the time it takes a new developer to set up their computer).
Your continuous integration builds should test only your product functionality. Do you really want to test installing CumulusCI from pip or Salesforce CLI from npm? Then why let these steps introduce points of failure, generate unnecessary noise in logs, or waste build time? It’s far better to have a runtime environment that’s pre-configured with all the necessary tools, so your CI builds are efficient, consistent, and focused solely on testing the actual functionality of your application.
What’s missing is a dedicated runtime for Salesforce DevOps. This runtime would be an environment that is fully pre-configured for Salesforce automation tasks. It would come equipped with the correct versions of tools like CumulusCI and Salesforce CLI, eliminating the need for manual installs or updates. Environment variables, often a pain point when setting up CI/CD pipelines, would be standardized, ensuring that automation scripts run consistently across every stage of the development lifecycle.
By providing one common runtime across all projects, teams can achieve true portability of their automation. No longer would developers need to spend time setting up or troubleshooting environment-specific issues. Instead, they can focus on what truly matters: building and deploying high-quality Salesforce applications.
Also, since D2X is BSD3-licensed open source software, anyone can run their own copy to generate images with custom build tooling dependencies.
The Need for Reusable CI/CD Workflows
Another common anti-pattern we see across the Salesforce ecosystem is a failure to leverage reusable workflows for CI/CD builds. The result is a proliferation of redundant logic in build scripts across multiple repositories. This was the initial reason we created D2X.
In one MuseLab project with an ISV partner, we were working on a product that consists of 15 managed 1GP and 2GP packages with extension package hierarchies at least 5 layers deep (base, core, feature, app, add-on). Each package was developed in its own GitHub repository.
Finding the Path
When we first started setting up the build scripts, I realized the path we were on would lead to adding hundreds of lines of GitHub Actions yaml definition files containing logic instead of simply mapping secrets to reusable workflows stored outside the repository. Despite your best intentions, hundreds or thousands of lines of build logic duplicated across 15 repositories will never get refactored. Worse, those build scripts weren’t using a common runtime. This meant that each build was inadvertently testing the ability to install all the tooling at the start of each build, adding time and stability risk to each build.
Our client needed a common foundation for DevOps across all their repositories. A few weeks of development later, they became the first users of D2X! Each build (feature, beta, release) now requires only 14 lines of yaml configuration which contains no logic, only mapping of secrets to reusable workflows. There’s nothing in each repository to maintain and there’s a single central place to make changes to the common D2X runtime and reusable CI/CD workflows across all repositories.
Better yet, D2X has unlocked new abilities that our client could previously only dream of like being able to develop in parallel across all packages and construct fully usable product environments in scratch orgs on-demand from any branch in the repository without ever needing anything from their DevOps team. Or the ability to develop at any layer in the package hierarchy and have fully configured scratch orgs for the layer you’re developing and the layers below it.
Is this approach for ISVs only?
ISVs only make up a small portion of the Salesforce developer ecosystem. But they have been at the forefront of Packageability for over a decade. Salesforce customers and SI partners can learn from their experience and challenges as they venture into the Well-Architected world of Adaptable development with Application Lifecycle Management and Packageability.
While D2X was born from the experience and requirements of highly complex ISV product architectures, it was designed from the ground up to be simple and cost efficient to start projects and to make it easy to maintain them as they scale.
Conclusion and Stay Tuned!
That’s the why and the what of D2X. Easy, batteries-included, well-architected Salesforce projects that leverage the power of GitHub.
But wait, there’s more! Stay tuned until next week, when we’ll dive deeper into D2X and walk you through the process of starting and running a project with D2X and CumulusCI. Or, get started today with the resources below!
