Category Archives: Build

NuGet Developer WorkFlow

During the proof of concept of migrating our internal dependency resolution from SVN Externals to NuGet I forgot an important stakeholder; Developers.  🙂

When the build updates the packages; how does a developer ensure their environment is up to date?

Edit: There is now a NuGet PowerTools Package that will add a PackageConsole command to restore all packages.
I still like the TortoiseHook and still wish I could specify the packages location, but we’re getting closer to ‘there’.

Check In Packages

The obvious answer is to check in the packages. After all that’s how the SVN Externals solution worked and it ensures the repeatability tenant of continuous builds. Sometimes however it’s not an option for a number of reasons; company policy, ruthless team leads or dictating architects, take your pick.

Check In Packages to a “temp” repository.

The way our SVN externals solution worked was to keep a separate repository entirely for “packages”. This repository could be blown away when it got bloated and replaced by a new version. Because it was used for multiple solutions/Company projects, it also enabled remote workers to keep their own copy of the repository locally (using DNS or host file entries).

This method can still be used with NuGet and means that an SVN update updates everything.

Pre-Build Step

A pre-build step can be added for each project to run NuGet Install. I don’t really like this as it is an unnecessary build step. This task is an “Update” task not a build task and the speed of my build is sacrosanct to me.

Manually with Batch File

After updating the local copy, the Developer can run a batch file manually to update packages.

for /r %cd% %%X in (*.csproj) do (nuget install %%~pX\packages.config -OutputDirectory packages)

This batch file command runs nuget install for every packages.config in a solution.

Automate the Batch File with Tortoise Hooks

Obviously an automated solution is preferred to adding a secondary step to the developer’s workflow. If your VCS client supports hooks such as TortoiseSVN client hooks  you can check-in the above batch file and get developers to hook it in to their VCS Update.

Configure NuGet to use shared packages repository

The true Holy Grail for people who don’t want to check-in packages is the ability to tell NuGet where packages live. It could then be configured to use a shared network drive to find the packages. This would be the same repository used by the build so by default it is always up to date. Unfortunately this is not yet a feature of NuGet. It is being discussed as a new feature, so if its something you are keen on go vote for it at http://nuget.codeplex.com/workitem/215

Tagged ,

Diamonds Aren’t Forever – The Build

This example will use TeamCity as the build platform as well as the Alpha Nuget Trigger and Update/Install Packages Build step found here.
It will also use my preferred Package Update Strategy defined here.
The build targets are specified in the build directory of the solution (here)
The build targets are linked into the solutions via the Named MsBuild Hook.

The Build

The automated build for each component has 4 main build steps

  1. Update and Install packages
    This is done by adding the Alpha NuGet Build Step and selecting the options relevant to your update strategy. I make liberal use of Build Parameters here.
  2. Increment Version Number
    This is done via the TeamCity.targets.
    It takes the TeamCity build number, makes it the revision number in the AssemblyInfo. The pack command later uses this as the package version.
  3. Build
  4. Check In
    This is not implemented in the example and will depend on your VCS. However updating packages will change the packages.config and the project files of your solution. These will need to be checked in. If you also check in packages, these will need to be added as well.
  5. Create, Lock Versions, Publish Package
    Creating and Publishing a package are trivial with NuGet. Package.Targets contains targets that Exec NuGet to do create a package. I use a Shared Drive for my repository so publishing is just a matter of specifying the output directory of pack to the repository UNC path. However there is also a publish command in NuGet if you want to push to the official feed.The interesting part of this step is my need to ensure all dependencies read from my project file will be locked to the version used to build the current version. This involves changing all dependency elements in the nuspec file inside the package. The target unzips the package, runs the LockDepedencyVersions Target found here and then re-zips the package.

Triggering

The last piece of the TeamCity puzzle is triggers; we want a build to be triggered if there is a new dependent package. At the moment it is a little clumsy as you must create a trigger per package you wish to monitor. I only monitor internal dependencies and then only direct internal dependencies.

So for example the UI project whilst it uses Common, Common is a dependency of Service A and Service B. So I only need to add triggers for those two packages. Because when Common changes, Service A and B will be triggered and when they publish their new versions the UI build will be triggered.

Tagged , , ,

Diamonds Aren’t Forever – Choosing a package update strategy

There are two distinct yet related parts in the automatic creation of a package that need to be considered.

  1. Updating of its dependencies for building

    Should we always update to the latest versions of our dependencies before building?
    Or only up to the latest minor version? Or Major version?
    Should we specify ranges for each individual package that we should update to?

    Nuget provides two options here, one is too specify (by hand ATM) the allowedVersions attribute in the packages.config (following the Version Specification). The update command then only updates to versions that are within the specified range.

    At the moment, the Nuget pack command doesn’t copy these across into the package. So this needs to be automated manually.
    (This should be fixed in the next version)

    The other option is to use the –safe option of Update. From the docs “Looks for updates with the highest version available within the same major and minor version as the installed package.”.

    In other words, if you have Elmah 1.0.0.0 installed, and 1.0.1.0, 1.0.2.0, and 1.1.0.0 are in the feed:

    Command Description
    Update-Package Elmah Updates Elmah to 1.1.0.0
    Update-Package Elmah -Safe Updates Elmah to 1.0.2.0

     

  2. Specifying allowable dependency versions during packaging?

    Should we restrict consumers to only using the exact version of our dependencies that we built with?
    Or should we allow any new version to be included?

    At the moment NuGet Pack adds dependencies from your project file with no version restrictions. So any consumers of your package can upgrade to newer versions of your dependencies. Any restriction you require will need to be done via custom build step to customise the dependencies or manually maintain a nuspec template that pack will use.

    I have created an MsBuild target called LockVersionDependencies . Which will lock dependency versions to the exact version used to build the package.

These are interrelated issues;  regardless whether your package is happy to upgrade to the latest of every dependency, those dependencies may restrict the allowable versions of other dependencies to an exact version or a restricted range.

My Preference

I prefer to allow all dependencies to update to the latest version during the build. If it’s successful I lock the dependencies to the exact version I built with when I create the package; so downstream builds will only use a combination of versions I know have built and passed the tests.

If a dependency updates and breaks the build, I would then fix the issue, or specify a version restriction for it so it doesn’t upgrade in future builds.

Tagged , , ,

Diamonds Aren’t Forever – Example Projects

As per this diagram I have created a set of basic .Net projects that represent the problem and put it in a bitbucket repository.

The starting point is Tag, Version 1.

The Sample

The sample consists of 4 separate visual studio solutions, one for each component; the build directory that contains the MsBuild targets; the local Nuget Repository and some Content files.

Installation

  1. Clone Tag – https://bitbucket.org/AdamMills/diamondsarentforever/src/Version1
  2. Add the Local Repository to Nuget Settings or Update Repository Path in Build/All.Targets
  3. Run PrepareEnvrionment.bat

The Local Repository

So the build doesn’t regularly download megabytes of packages (I don’t like storing them in VCS) I use a local repository. There are scripts on the interweb to mirror the full NuGet repository, but that seems too much for this little project. There is a nugget extension that can be used to copy individual packages, see here (note if using XP, you’ll need to manually add the LocalAppData Environment Variable)

With this I just maintain and checkin a batch file that downloads each package I need, so I can move or restore the local repository as needed.

I imagine that in the future Nuget will allow you to specify a cache, and then all team members can point to a common cache. Until then….

Tagged , , ,

Diamonds Aren’t Forever

As above, a Diamond Dependency is when the dependency graph of a package forms a diamond shape.  It makes upgrading components towards the bottom of the graph more difficult as dependencies have to picked so all package and dependency versions match or are compatible.

For a more concrete example, taking the diagram above, say there is a new nifty feature in Common we want to use in the UI. ServiceA 2.0 is built with Common 2.0, however there is no ServiceB 2.0, so to upgrade to Common 2.0 in the UI we must either upgrade ServiceB, abandon all hope, or determine if Common 2.0 can be brought in side by side with Common 1.0. The latter solution can cause a lot of issues if types in Common are shared between communication with UI and ServiceB.

The general solution is then to “level” the dependencies i.e. upgrade everything to lowest common denominator. In the above solution; the lowest common denominator is Common 1.0, so we must use Service A and Service B 1.0, as well as using Common 1.0 in UI. When Service B 2.0 is released we can upgrade everything to 2.0

For a better explanation of the issue, see here. http://www.well-typed.com/blog/9

Previously, for me, the solution to packaging and levelling in .Net was to use another platforms packaging tools such as Maven or use SVN Externals to bring in dll’s and manually update the externals property, working out the levelling myself. With the introduction of Nuget the .Net world now has a package manager which can do the levelling for you, and with a little bit of work lets you automate and integrate the build process of numerous packages; regardless of diamond dependencies.

This is a series of posts; broken into the following:

Tagged , , ,

Customising Visual Studios Build

Previously, the way I would hook into a projects/solutions build was to edit the .csproj file to import a build file. It always annoyed me and isn’t immediately clear to people (or me after I come back to a project) how the targets are being injected.

I recently discovered that you can follow certain naming conventions to automagically get your targets imported into the build. Now, I know some people think automagic is even less transparent than explicitly defining something, however the naming is quite straightforward and obvious.

Add a targets file named; [after|before].<SolutionName>.sln.targets to the same directory as your solution file.

The after file is added after the project files and the before file is added before the project files.

This only works for solutions.
As an example

<Target Name="RunPostBuildScripts" AfterTargets="Build">
   <!-- Do Stuff Here -->
</Target>

More Info Here
Tagged

Setting TeamCity variables from build

I stumbled upon a useful feature of TeamCity today. The ability to set TeamCity variables from your build. This can be useful for instance if you create a version number in your build which you wish to use as the build number or label for VCS inside TeamCity.

All you need to do is output a message in a particular format. Here is an example of setting the build number in MSBUILD to the value of a property.

<Message Text="##teamcity[buildNumber '$(AssemblyVersionString)']" />

After this has run in your build, the value of the TeamCity variable %system.build.number% will now be assigned the value of  AssemblyVersionString.

Tagged ,

CI Factory

Continuous Integration Factory (CI Factory) has just popped up onto my radar and looks fantastic.

Being currently on hiatus and looking to get into the London contracting scene I have been talking to expats about how much pain to expect. Build process comes up time and time again. I’ve heard of builds being manual and taking whole weekends!! So this looks to be a handy tool to have under your belt for getting CI setup quickly.

Tagged ,

Is it just me?

… or is XML a really stupid choice for Build Scripts?
Im sick of writing NANT scripts in XML.

I guess it’s just me as there is almost nothing around that doesn’t use XML, at least in the .NET world.

Tagged ,