Tagged: nuget

Streamlining Dependency Management: Lessons from 2015 to Today

In this throwback Tuesday post, we revamp at a dusty draft post from 2015.

In 2015, I faced a challenging problem. I had to manage dependencies across a suite of interconnected applications. It was crucial to ensure efficient, safe builds and deployments. Our system included 8 web applications, 24 web services, and 8 Windows services. This made a total of 40 pipelines for building, deploying, and testing. At the time, this felt manageable in terms of automation, but shared dependencies introduced complexity. It was critical that all applications used the same versions of internal dependencies. This was especially important because they interacted with a shared database and dependencies can change the interaction.

Back then, we used zip files for our package format and were migrating to NuGet to streamline dependency management. NuGet was built for exactly this kind of challenge. However, we needed a system to build shared dependencies once. It was necessary to ensure version consistency across all applications. The system also needed to handle local, and server builds seamlessly.

Here’s how I approached the problem in 2015 and how I’d tackle it today, leveraging more modern tools and practices.


The 2015 Solution: NuGet as a Dependency Manager

Problem Statement

We had to ensure:

  1. Shared dependencies were built once and consistently used by all applications.
  2. Dependency versions were automatically synchronized across all projects (both local and server builds).
  3. External dependencies are handled individually per application.

The core challenge was enforcing consistent dependency versions across 40 applications without excessive manual updates or creating a maintenance nightmare.

2015 Approach

  1. Migrating to NuGet for Internal Packages
    We began by treating internal dependencies as NuGet packages. Each shared dependency (e.g., ProjB, ProjC, ProjD) was packaged with a version number and stored in a NuGet repository. When a dependency changed, we built it and updated the corresponding NuGet package version.
  2. Version Synchronization
    To ensure that dependent applications used the same versions of internal packages:
    • We used nuspec files to define package dependencies.
    • NuGet commands like nuget update were incorporated into our build process. For example, if ProjD was updated, nuget update ProjD was run in projects that depended on it.
  3. Automating Local and Server Builds
    We integrated NuGet restore functionality into both local and server builds. On the server, we used Cruise Control as our CI server. We added a build target that handled dependency restoration before the build process began. Locally, Visual Studio handled this process, ensuring consistency across environments.
  4. Challenges Encountered
    • Updating dependencies manually with nuget update was error-prone and repetitive, especially for 40 applications.
    • Adding new dependencies required careful tracking to ensure all projects referenced the latest versions.
    • Changes to internal dependencies triggered cascading updates across multiple pipelines, which increased build times.
    • We won’t talk about circular dependencies.

Despite these challenges, the system worked, providing a reliable way to manage dependency versions across applications.


The Modern Solution: Solving This in 2025

Fast forward to today, and the landscape of dependency management has evolved. Tools like NuGet remain invaluable. However, modern CI/CD pipelines have transformed how we approach these challenges. Advanced dependency management techniques and containerization have also contributed to this transformation.

1. Use Modern CI/CD Tools for Dependency Management

  • Pipeline Orchestration: Platforms like GitHub Actions, Azure DevOps, or GitLab CI/CD let us build dependencies once. We can reuse artifacts across multiple pipelines. Shared dependencies can be stored in artifact repositories (e.g., Azure Artifacts, GitHub Packages) and injected dynamically into downstream pipelines.
  • Dependency Locking: Tools like NuGet’s lock file (packages.lock.json) ensure version consistency by locking dependencies to specific versions.

2. Automate Version Synchronization

  • Semantic Versioning: Internal dependencies should follow semantic versioning (e.g., 1.2.3) to track compatibility.
  • Automatic Dependency Updates: Use tools like Dependabot or Renovate to update internal dependencies across all projects. These tools can automate pull requests whenever a new version of an internal package is published.

3. Embrace Containerization

  • By containerizing applications and services, shared dependencies can be bundled into base container images. These images act as a consistent environment for all applications, reducing the need to manage dependency versions separately.

4. Leverage Centralized Package Management

  • Modern package managers like NuGet now include improved version constraints and dependency management. For example:
    • Use a shared Directory.Packages.props file to define and enforce consistent dependency versions across all projects in a repository.
    • Define private NuGet feeds for internal dependencies and configure all applications to pull from the same feed.

5. Monitor and Enforce Consistency

  • Dependency Auditing: Tools like WhiteSource or SonarQube can analyze dependency usage to ensure all projects adhere to the same versions.
  • Build Once, Deploy Everywhere: By decoupling build and deployment, you can reuse prebuilt NuGet packages in local and server builds. This ensures consistency without rebuilding dependencies unnecessarily.

Case Study: Revisiting ProjA, ProjB, ProjC, and ProjD

Let’s revisit the original example that help me figure this out in 2015 but using today’s tools.

  1. When ProjD changes:
    • A CI/CD pipeline builds the new version of ProjD and publishes it as a NuGet package to the internal feed.
    • Dependency lock files in ProjB and ProjC ensure they use the updated version.
  2. Applications automatically update:
    • Dependabot identifies the new version of ProjD and creates pull requests to update ProjB and ProjC.
    • After merging, ProjA inherits the changes through ProjB.
  3. Consistency is enforced:
    • Centralized package configuration (Directory.Packages.props) ensures that local and server builds use the same dependency versions.

The Results

By modernizing our approach:

  • Efficiency: Dependencies are built once and reused, reducing redundant builds.
  • Consistency: Dependency versions are enforced across all projects, minimizing integration issues.
  • Scalability: The system can scale to hundreds of applications without introducing maintenance overhead.

Conclusion

In 2015, we solved the problem using NuGet and MSBuild magic to enforce dependency consistency. Today, with modern tools and practices, the process is faster, more reliable, and scalable. Dependency management is no longer a bottleneck; it’s an enabler of agility and operational excellence.

Are you ready to future-proof your dependency management? Let’s talk about optimizing your build and deployment pipelines today.

Deploying NuGet Packages Instead of Zips

I was on a project to improve an application deployment process that used zip files for packaging the applications. Zips are good. They allow you to package and compress files into one bit sized file, but there is so much more to be had with a dedicated package solution.  Maven, gem, wheel, npm, cpan, rpmdebnuget, chocolatey, yum… the list goes on and with so many options to provide an improved package for deployment its hard to justify using plain old zips.

Since this was a .Net project I focused on NuGet. NuGet is itself a zip file, but a zip on steroids. Zip provides the compression and NuGet adds additional meta data and functionality.

  1. Standard package meta data and file layout.
  2. Versioning ala SemVer.org.
  3. Package manager to control install, upgrade, and uninstall.
  4. Dependency management.
  5. Having a package manage file deployment means you have a repeatable process as opposed to manual where one missed file can kill you. Also, when I deploy the same package multiple times the system is in the same state after each deployment, idempotent.

Enough of the sales pitch. Actually, one problem that I had with using NuGet alone was no easy way to validate the package through checksum. So, in addition to NuGet, using a dedicated artifact repository solution like Artifactory gives an added layer of comfort. A good paper, although biased, on Artifactory can be found here.

Happy Packaging!

Setup a NuGet Server

Setting up a NuGet Server is so easy that everyone should do it. Why? If you are beholden to corporate policies that restrict the applications and references your projects can have, you can still benefit from the awesomeness of NuGet by hosting corporate approved packages. If you have a critical build process, you may not want to depend on the reliability of third party servers. Oh, I can keep going, but I won’t. The point is with 5 easy steps (depending on how you may break it down), you can have a NuGet server up and serving packages.

  1. Create an Empty Web Application (I’m using Visual Studio)
  2. Use NuGet to add reference in the Web Application to “NuGet.Server”
  3. Add the nupkg files that you want to host to the Packages folder
  4. Deploy the Web Application
  5. Add the URL of the Web Application to your local NuGet package manager.

Thanks to docs.nuget.org and Adam James Naylor for opening my eyes to how simple this is:

http://docs.nuget.org/docs/creating-packages/hosting-your-own-nuget-feeds

http://www.adamjamesnaylor.com/2013/04/26/Setting-Up-A-Private-NuGet-Server.aspx

Formalized Versioning

Phil Haack did a post about SemVer 2.0, a formal specification for public API versioning. I think it’s a novel idea as it puts a name and formal structure to what a lot of dev shops I know have been doing. It also allows packaging tools to standardize, like the work Phil talks about with NuGet.

I have added “integrating SemVer and NuGet into CI” to my very long list of things to do for my main side project. That list just keeps on growing.