Tagged: msbuild

Confirming MSBuild Project Dependency Build Behavior

So we have one giant Visual Studio solution that builds every application project for our product. It is our goal to one day be able to build and deploy each application independently. Today, we have a need to build one project for x86 and all others as x64. This requirement also provides a reason to explore per application build and deploy. The x64 projects and x86 project share some of the same dependencies and provides a good test for per application build. The purpose of this exploration is to determine the best way to automate the separate platform builds and lay groundwork for per application build. These are just my notes and not meant to provide much meat.

First, I will do some setup to provide a test solution and projects to experiment with. I created 4 projects in a new solution. Each project is a C# class library with only the default files in them.

  • ProjA
  • ProjB
  • ProjC
  • ProjD

Add project dependencies

  • ProjA > ProjB
  • ProjB > ProjC, ProjD
  • ProjC > ProjD

Set the platform target (project properties build tab) for each project like so

  • ProjA x64
  • ProjB x64
  • ProjC x64
  • ProjD x86

Behavior when a Dependent Project is Built

Do a Debug build of the solution and inspect the bin folders.

  • BuildTest\ProjA\bin\Debug
    • ProjA.dll               10:03 AM             4KB        B634F390-949F-4809-B937-66069C5F058E              v4.0.30319 / x64
    • ProjA.pdb             10:03 AM             8KB
    • ProjB.dll                10:03 AM             4KB        B0C5B475-576D-44D2-BD41-135BDA69225E          v4.0.30319 / x64
    • ProjB.pdb             10:03 AM             8KB
  • BuildTest\ProjB\bin\Debug
    • ProjB.dll               10:03 AM             4KB        B0C5B475-576D-44D2-BD41-135BDA69225E          v4.0.30319 / x64
    • ProjB.pdb            10:03 AM             8KB
    • ProjC.dll               10:03 AM             4KB        DBB9482F-6609-4CA5-AB00-009473E27CDA          v4.0.30319 / x64
    • ProjC.pdb            10:03 AM             8KB
    • ProjD.dll               10:03 AM             4KB        4F0F7877-5046-4A32-8B8E-FAD8E2660CE6            v4.0.30319 / x86
    • ProjD.pdb            10:03 AM             8KB
  • BuildTest\ProjC\bin\Debug
    • ProjC.dll               10:03 AM             4KB        DBB9482F-6609-4CA5-AB00-009473E27CDA          v4.0.30319 / x64
    • ProjC.pdb            10:03 AM             8KB
    • ProjD.dll               10:03 AM             4KB        4F0F7877-5046-4A32-8B8E-FAD8E2660CE6            v4.0.30319 / x86
    • ProjD.pdb            10:03 AM             8KB
  • BuildTest\ProjD\bin\Debug
    • ProjD.dll              10:03 AM             4KB        4F0F7877-5046-4A32-8B8E-FAD8E2660CE6            v4.0.30319 / x86
    • ProjD.pdb           10:03 AM             8KB

Do a Debug rebuild of ProjA

  • ProjA all DLLs have new date modified.
  • ProjB all DLLs have new date modified.
  • ProjC all DLLs have new date modified.
  • ProjD all DLLs have new date modified.

Do a Debug build of ProjA

  • ProjA no DLLs have new date modified.
  • ProjB no DLLs have new date modified.
  • ProjC no DLLs have new date modified.
  • ProjD no DLLs have new date modified.

Change Class1.cs in ProjA and do a Debug build of ProjA

  • ProjA: ProjA.dll has new data modified, ProjB.dll does not.
  • ProjB no DLLs have new date modified.
  • ProjC no DLLs have new date modified.
  • ProjD no DLLs have new date modified.

Change Class1.cs in ProjB and do a Debug Build of ProjA

  • ProjA all DLLs have new date modified.
  • ProjB: ProjB.dll has new data modified, ProjC.dll and ProjD do not.
  • ProjC no DLLs have new date modified.
  • ProjD no DLLs have new date modified.

We change Class1.cs in ProjC and do a Debug Build of ProjA

  • ProjA all DLLs have new date modified.
  • ProjB: ProjB.dll and ProjC.dll have new data modified, and ProjD does not.
  • ProjC ProjC.dll has new date modified, and ProjD does not.
  • ProjD no DLLs have new date modified.

We change Class1.cs in ProjD and do a Debug Build of ProjA

  • ProjA all DLLs have new date modified.
  • ProjB all DLLs have new date modified.
  • ProjC all DLLs have new date modified.
  • ProjD all DLLs have new date modified.

Conclusion

  • If a dependency has changes it will be built when the dependent project is built.

Behavior When Project with Dependencies is Built

Next, I want to verify the behavior when a project that has dependencies is built.

Clean the solution and do a debug build of the solution.

Do a Debug build of ProjD

  • ProjA no DLLs have new date modified.
  • ProjB no DLLs have new date modified.
  • ProjC no DLLs have new date modified.
  • ProjD no DLLs have new date modified.

We change Class1.cs in ProjD and do a Debug build of ProjD

  • ProjA no DLLs have new date modified.
  • ProjB no DLLs have new date modified.
  • ProjC no DLLs have new date modified.
  • ProjD all DLLs have new date modified.

We change Class1.cs in ProjC and do a Debug Build of ProjD

  • ProjA no DLLs have new date modified.
  • ProjB no DLLs have new date modified.
  • ProjC no DLLs have new date modified.
  • ProjD no DLLs have new date modified.

Conclusion

  • If a project with dependencts is built, any projects that depend on it will not be built.

Behavior When Bin is Cleaned

I manually deleted DLLs in ProjD and built ProjA and the DLLs with same date modified reappeared. Maybe they were fetched from obj folder.

I do a clean on ProjD (this cleans obj) and build ProjA and new DLLs are added to ProjD.

Conclusion

  • Obj folder acts like a cache for builds.

Behavior when External Dependencies are Part of Build

Add two new projects to solution

  • ExtA x64 > ExtB
  • ExtB x86

Updated these projects so they output to the solution Output/Debug folder.

Added references to the ExtA and ExtB output DLLs

  • ProjA > ExtA
  • ProjB > ExtB

I did a solution rebuild and I noticed something that may also be a problem in other tests. When building ProjC, ProjD, and ExtA we get an error:

warning MSB3270: There was a mismatch between the processor architecture of the project being built “AMD64” and the processor architecture of the reference. This mismatch may cause runtime failures. Please consider changing the targeted processor architecture of your project through the Configuration Manager so as to align the processor architectures between your project and references, or take a dependency on references with a processor architecture that matches the targeted processor architecture of your project.

Also, ProjA and ProjB are complaining about reference resolution:

warning MSB3245: Could not resolve this reference. Could not locate the assembly “ExtA”. Check to make sure the assembly exists on disk.

In Visual Studio I update the dependency for ProjA and ProjB to include the Ext projects. This fixes the MSB3245 error.

Conclusion

  • We need to build all dependencies with the same platform target as the dependent.
  • We need to build external references before building any dependents of the external references (e.g. get NuGet dependencies).
  • When a solution contains a project that is dependent on another project, but does not have a project reference, update the dependency to force the dependency to build first.

Separating Platform Builds

Add new Platforms for x64 and x86. Update configuration so each project can do a x86 and x64 build. Have Ext project output to x86 and x64 folders for release and debug builds.

Add new projects for ExtC and ExtD and have respective Proj reference their release output. ProjC should ref ExtC x64 release and ProjD should ref ExtD x86 release.

Issue Changing Platform on Newly Add Solution Projects

So, I am unable to change platform target for ExtD/C as x86 and x64 do not appear in drop down and I can’t add them because UI says they are already created. I manually add them to project file.

 

<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Debug|x64'">
    <DebugSymbols>true</DebugSymbols>
    <OutputPath>bin\x64\Debug\</OutputPath>
    <DefineConstants>DEBUG;TRACE</DefineConstants>
    <DebugType>full</DebugType>
    <PlatformTarget>x64</PlatformTarget>
    <ErrorReport>prompt</ErrorReport>
    <CodeAnalysisRuleSet>MinimumRecommendedRules.ruleset</CodeAnalysisRuleSet>
  </PropertyGroup>
  <PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release|x64'">
    <OutputPath>bin\x64\Release\</OutputPath>
    <DefineConstants>TRACE</DefineConstants>
    <Optimize>true</Optimize>
    <DebugType>pdbonly</DebugType>
    <PlatformTarget>x64</PlatformTarget>
    <ErrorReport>prompt</ErrorReport>
    <CodeAnalysisRuleSet>MinimumRecommendedRules.ruleset</CodeAnalysisRuleSet>
  </PropertyGroup>
  <PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Debug|x86'">
    <DebugSymbols>true</DebugSymbols>
    <OutputPath>bin\x86\Debug\</OutputPath>
    <DefineConstants>DEBUG;TRACE</DefineConstants>
    <DebugType>full</DebugType>
    <PlatformTarget>x86</PlatformTarget>
    <ErrorReport>prompt</ErrorReport>
    <CodeAnalysisRuleSet>MinimumRecommendedRules.ruleset</CodeAnalysisRuleSet>
  </PropertyGroup>
  <PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release|x86'">
    <OutputPath>bin\x86\Release\</OutputPath>
    <DefineConstants>TRACE</DefineConstants>
    <Optimize>true</Optimize>
    <DebugType>pdbonly</DebugType>
    <PlatformTarget>x86</PlatformTarget>
    <ErrorReport>prompt</ErrorReport>
    <CodeAnalysisRuleSet>MinimumRecommendedRules.ruleset</CodeAnalysisRuleSet>
  </PropertyGroup>

No I can update the output path for ExtC/D and update Configuration Manager to proper platform.

Since ProjD is exclusively an x86 projects I removed them from the build for x64. I have ExtD building both x86 and x64. I updated dependencies so ExtC/D build before ProjC/D.

Final Conclusion

This was a bit much to verify what may be common knowledge on the inter-webs, but I wanted to see for myself. There is more that I want to experiment with like NuGet, build performance and optimization, but this gave me enough to move forward with an initial revamp of our automated build. I am going to proceed with a separate pipeline for the an x86 build of the entire solution and a separate deploy for the x86 application. I really believe that going forward that NuGet can become a key tool in standardizing and optimizing per application build and deployment.

Summary

  • If a dependency has changes it will be built when the dependent project is built. Otherwise it will not be rebuilt. This is all a feature of MSBuild. When we move to per application build we will have to build separate solutions for each application and build in isolated pipelines. To prevent conflicts with other applications, we should build dependencies that are shared across applications in a separate pipeline.
  • If a project with dependents is built, any projects that depend on it will not be built. If we
  • Obj folder acts like a cache for builds. We can extend this concept to a common DLL repo where all builds send their DLLs, but we would need a confident way of versioning the DLLs so that we always use the most recent or specific version… sounds like I am proposing NuGet (smile).
  • We need to build all dependencies with the same platform target as the dependent. We may build the main solution as x64 and build other projects that need x86 separately. I believe this would be the most efficient since the current x86 projects will not change often.
  • We need to build external references before building any dependents of the external references (e.g. get NuGet dependencies). We do this now with NuGet, we fetch packages first, but when we move to per application build and deploy this will automatically be handled by Go.cd’s fan in feature. We will have application builds have pipeline dependencies on any other application builds it needs. This will cause the apps to always use the most recent successful build of an application. We can make this stronger by having application depend on test pipelines to insure the builds have been properly tested before integration.
  • When a solution contains a project that is dependent on another project, but does not have a project reference, update the dependency to force the dependency to build first.

NOTE

This is actually a re-post of a post I did on our internal team blog. One comment there was we should also do an Any CPU build then we can have one NuGet will all versions.

Build Once, Deploy Everywhere

We were faced with an interesting problem. We want to build once and deploy the build to multiple environments for testing, staging and ultimately consumption in production. Well in addition to build once, we also want to allow remote debugging. We need to build in debug mode to get pdb generated and have other configurations that will allow debugging. Yet, we don’t want to deploy a debug build to staging or production. What do we do?

The thought right now is to do a debug build and create to packages, one for debug and one for release. To do this we would have to strip out the pdb and turn on debugging in the release package. So, we still have one build, but we have two flavors of packages.

I am not yet sure if this is viable, hence the reason I am blogging this out. First I need to fully understand the difference between a debug and release build in MSBuild. I have an idea, but I need to verify my assumptions.

Difference Between Debug and Release Build

What I have found is the main difference between debug and release building are:

  1. Debug build generates pdb files.
  2. Release build instructs the compiler to use JIT optimizations.

PDB files are symbol databases that allow a debugger to map machine code to source code (actually MSIL) so you can set break points in source code and the debugger can halt execution of the machine code. This is probably a terrible explanation, but you should get the gist.

JIT optimizations are things the compiler does to speed up the execution of your code. It may reorganize loops to make them run faster and other little magic tricks that happen under the cover that we usually never have to worry about.

Scott Hanselman has an interesting post on this, http://www.hanselman.com/blog/DebugVsReleaseTheBestOfBothWorlds.aspx. This posts suggest that you could do a release build and configure the runtime with an ini file that would determine if JIT optimizations are performed or tracking information is generated.

http://msdn.microsoft.com/en-us/library/9dd8z24x(v=vs.110).aspx this post explains more about the ini.

Now What?

After doing this research I learned a lot about building .Net code, but I also realized that I am taking this a little to far. My primary goal is that we build our application once and use that build in multiple environments to get it tested and deployed to production. When we need to do a remote debug we are using researching an issue and there is no reason that we couldn’t flip a switch on one particular build so that it builds in debug mode, deploy it to a test environment, debug it, make some fixes after finding the cause of the issue we are debugging, then flip the switch back to release and build again, this time allowing the build to go all the way to production.

Issues

The problem here is that we need to make sure that we do not allow debug builds to make it into production. My initial thought is to mark debug builds with a version that is tagged with DEBUG. Then I can have logic in the production deploy that checks for the DEBUG tag and fail the deploy if it is present. We can do the same for pdb files and web.config. Specifically, check for inclusion of pdb (we shouldn’t have pdb file in production). We can also have logic that checks for debug=true and other configurations that we don’t want leaking into production.

We would have to alter our deployment pipeline to add a job that will do these checks based on the environment being deployed. We would have to also look at maybe putting debug builds in a different artifact repository to keep them segregated from release candidates. This would also cause another change to the deployment pipeline where we check the release candidate or debug repository based on some setting.

Conclusion

This would be a lot of changing to our pipeline, but I believe it is worth it in the long run. It also prevents us from leaking manual processes into how we build and deploy the app.