Category Archives: MSBuild

Direct integration of ILMerge with VB.net Projects

0
Posted by admin on October 27, 2011 – 3:03 pm
Filed under .NET, Error Handling, MSBuild, Troubleshooting

I’ve been working on a few utility type applications lately (more on them later, once they’re firmed up), but one thing I’ve found incredibly useful for certain projects is ILMerge.

If you’re not familiar with it, ILMerge is a utility that essentially can combine two or more .net assemblies into a single assembly, and reducing an application’s footprint is something I’m particularly passionate about!

In the past, I’ve always created postbuild steps that included executing ILMerge as just a command line post build process, but doing it this way has always been problematic. To start, debugging in the IDE tends to be compromised at best, and not possible at worst. Plus it just never felt clean.

In researching the problem, I ran across three blog posts that have provided more than enough info to get me past the Post Build process and actually integrate ILMerge directly into the build process itself, and one very intriguing alternative.

First, Scott Hanselman wrote about his experiences using ILMerge to merge a VB.net assembly into an otherwise C# project back in 2007. A great article and a fantastic first step.

Then, I came across Daniel Fortunov’s  article about integrating ILMerge into the VS build process. His example is also very good and a little cleaner, I’d say than Scott’s approach, but both articles are definitely worth checking out if you find yourself needing this kind of thing.

Lastly, Jeffrey Richter wrote a terrifically short but incredibly eye-popping article about a technique to load .net assemblies dynamically from binary resources baked into a single assembly at compile time. Very simple and clean. A great technique to have in your toolbox if the need ever arises.

For my part, I’ve used the ILMerge technique to merge Mono.Cecil.dll (a C# library from the mono project for directly altering .net assemblies) into a VB.net utility that can repackage portions of an assembly’s PDB file directly into the assembly’s EXE or DLL file, to supply line numbers during the rendering of an exception’s stack trace without actually have to provide a PDB (and all the additional metadata about your assembly) to your client. A fantastic debugging technique and one that I’ve been working with (off and on) for several years now. I’ll write about it in more detail later.

Normal Ol’ DLLs from VB.net

7
Posted by admin on October 25, 2010 – 10:31 pm
Filed under .NET, Arcade, Code Garage, MSBuild, VB Feng Shui

Every once in a while, I find a need to do something a bit off the wall. Recently, I had another one of those situations.

I’ve spent a lot of time working with some of the old arcade emulators that are floating around (the most famous of which is MAME, or Multi Arcade Machine Emulator).

Mame itself is pretty utilitarian, so there are a number of front ends  that are essentially menuing systems to provide a user with an easy to browse interface for selecting games to play and among the more popular front ends is MaLa.

image

MaLa Main screen (using one of many available skins) showing list of games, and a screenshot of the selected game

One nice aspect of MaLa is that it supports plugins, and there are a number of them out there, to control LED lights, play speech, etc.

I had had a few ideas about possible MaLa plugins for awhile, but the MaLa plugin architecture centers around creating a standard Win32 DLL with old fashioned C styled Entrypoints, and, well, I kinda like working in VB.net these days.

Gone Hunting

Eventually, curiousity got the better of me, and I started looking for ways to expose standard DLL entry points from a .net assembly. I ended up finded Sevin’s CodeProject entry called ExportDLL that allowed just that. Essentially, it works by:

  1. You add a reference in your project to a DLL he created, that only contains a single Attribute for marking the functions you want to export.
  2. Create the functions you want to export as shared functions in a MODULE
  3. You mark those functions with the Attribute
  4. You compile your DLL
  5. You then run ExportDLL against your freshly compiled DLL
  6. ExportDLL then decompiles your DLL into IL, tweaks it, and recompiles the IL code back into a DLL

It sounds complicated but it’s really not.

I set it all up and had things working in about 30 minutes.

Gone South

Unfortunately, all was not quite right. MaLa requires 2 entry points (among a host of them) defined with a single integer argument passed on the stack. Pretty simple stuff. So I coded up:

    <ExportDLL("MaLaOrientationSwitch", CallingConvention.Cdecl)> _
    Public Shared Sub EntryPoint_MaLaOrientationSwitch(ByVal Orientation As Integer)

But when I ran the DLL within MaLa, it crashed immediately after calling this function, even with NO CODE in the function itself.

What this meant is that something about the export process was trashing the stack. I spent a solid day hunting for clues as to what might be failing. I did find that eliminating the argument from the exposed entrypoint allowed MaLa to work properly AND call my entrypoint, but, being unable to get the passed Orientation value, the call was basically useless.

Gone Around Again

In digging through it all, I happened to notice a comment on the CodeProject page for Sevin’s article pointing to a similar library by Robert Giesecke. I’m not sure if the two were developed independently or not, but Robert’s is certainly a more polished set of deliverables. He even went so far as to put together a C# project template that makes it ridiculously easy to kick off a project using his technique.

It turns out, not only is Robert’s approach cleaner, it actually properly exports the MaLaOrientationSwitch function above with no problems. MaLa can call it, pass in the argument and all is good.

Gone Fishing

One big difference between the two techniques is the Robert actually defines an MSBuild targets file to patch his DLL directy into the Visual Studio build process. Very cool! But, his build step happens AFTER the PostBuildEvent target, and it was in that target that I’d setup some commands to copy the DLL into a file called *.MPLUGIN, which is what MaLa specifically looks for. Hooking that process into the build itself makes debugging things quite natural, but, Mr. Giesecke’s target wasn’t allowing for that.

Here’s Robert’s targets file:

<Project
  xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <UsingTask TaskName="RGiesecke.DllExport.MSBuild.DllExportTask"
             AssemblyFile="RGiesecke.DllExport.MSBuild.dll"/>
  <Target Name="AfterBuild"
          DependsOnTargets="GetFrameworkPaths"
          >
      <DllExportTask Platform="$(Platform)"
                   PlatformTarget="$(PlatformTarget)"
                   CpuType="$(CpuType)"
                   EmitDebugSymbols="$(DebugSymbols)"
                   DllExportAttributeAssemblyName="$(DllExportAttributeAssemblyName)"
                   DllExportAttributeFullName="$(DllExportAttributeFullName)"
                   Timeout="$(DllExportTimeout)"
                   KeyContainer="$(KeyContainerName)$(AssemblyKeyContainerName)"
                   KeyFile="$(KeyOriginatorFile)"
                   ProjectDirectory="$(MSBuildProjectDirectory)"
                   InputFileName="$(TargetPath)"
                   FrameworkPath="$(TargetedFrameworkDir);$(TargetFrameworkDirectory)"
                   LibToolPath="$(DevEnvDir)\..\..\VC\bin"
                   LibToolDllPath="$(DevEnvDir)"
                   SdkPath="$(FrameworkSDKDir)"/>
  </Target>
</Project>

I’d worked with MSBuild scripts before, so I knew what it was capable of, I just couldn’t remember the exact syntax. A few google searches jogged my memory, and I ended up here at a great post describing exactly how you can precisely inject your own targets before or after certain other predefined targets.

I modified Robert’s targets file and came up with this:

<Project
    xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    <UsingTask TaskName="RGiesecke.DllExport.MSBuild.DllExportTask"
             AssemblyFile="RGiesecke.DllExport.MSBuild.dll"/>

  <!-- Add to the PostBuildEventDependsOn group to force the ExportDLLPoints
       target to run BEFORE any post build steps (cause it really should) -->
  <PropertyGroup>
    <PostBuildEventDependsOn>
      $(PostBuildEventDependsOn);
      ExportDLLPoints
    </PostBuildEventDependsOn>
  </PropertyGroup>

  
  <Target Name="ExportDLLPoints"
          DependsOnTargets="GetFrameworkPaths"
          >
    <DllExportTask Platform="$(Platform)"
                   PlatformTarget="$(PlatformTarget)"
                   CpuType="$(CpuType)"
                   EmitDebugSymbols="$(DebugSymbols)"
                   DllExportAttributeAssemblyName="$(DllExportAttributeAssemblyName)"
                   DllExportAttributeFullName="$(DllExportAttributeFullName)"
                   Timeout="$(DllExportTimeout)"
                   KeyContainer="$(KeyContainerName)$(AssemblyKeyContainerName)"
                   KeyFile="$(KeyOriginatorFile)"
                   ProjectDirectory="$(MSBuildProjectDirectory)"
                   InputFileName="$(TargetPath)"
                   FrameworkPath="$(TargetedFrameworkDir);$(TargetFrameworkDirectory)"
                   LibToolPath="$(DevEnvDir)\..\..\VC\bin"
                   LibToolDllPath="$(DevEnvDir)"
                   SdkPath="$(FrameworkSDKDir)"/>
  </Target>
</Project>

Now, I can perform the compile, and execute my postbuild event to copy the DLL over to the MaLa Plugins folder and give it the requisite MPLUGIN name, all completely automatically.

And, the icing on the cake is that I can build a MaLa plugin completely in VB.net, with no C or C# forwarding wrapper layer and with a fantastic XCOPY-able single DLL application footprint (save for the .net runtime, of course<g>).

It’s a wonderful thing.

Getting Rid of the Bogus Warnings in MSBuild Projects

0
Posted by admin on October 15, 2009 – 3:07 am
Filed under MSBuild, Troubleshooting

If you’ve ever loaded up an MSBuild project in VS, you have likely seen some warning messages in the Error List window, essentially saying that either the first item in a custom ItemGroup is invalid or the first Property in a custom PropertyGroup is invalid.

The problem is that the MSBuild designers wanted to provide a schema to you’d get some nice intellisense functions when editing a proj file, but, they had to allow for custom properties and items because, well, that’s what MSBuild is all about, after all!

So, those custom elements by definition, can’t be included in the generic schema and, bingo, validation warnings.

The warnings don’t actually harm anything and MSBuild effectively ignores them, so one solution is to do just that. Ignore all those new warnings in your Errors window. That’s even what MS recommends <sigh>

For me, though, ANYTHING in that errors window is a red flag.

Getting that Nice, Clean Errors Window

After digging around, it turns out there’s essentially 2 options to get rid of those warnings.

  1. Edit the Microsoft.Build.xsd file and include ALL the custom Property and Item names you might be using
  2. Or tell VS to not validate the proj file against any schema.

I can’t imagine bothering to try to keep that xsd file up to date constantly as I change proj files, so that option was out.

Fortunately, telling VS not to validate the project file is relatively easy, and pretty easy to undo as well.

First, open the proj file in Visual Studio (if it’s an actual VBPROJ or CSPROJ file, you’ll need to unload the project first, then right click on it again and select Edit Project).

Click in the project editing window, then in the Properties box, Click on the Ellipse button to change the schemas property.

image

You should see this (though, the red Xs will be green checks):

image

When you get the dialog showing all the schemas against which the file is validated, select each one and choose “Do not use this schema”

image

Click ok and repeat for any other PROJ files you need to handle this way.

When done, you should no longer see any validation warnings in the error window.

Unfortunately, this also means that the proj file won’t be pre-validated at all anymore and you won’t get any intellisense anymore. But, really, once you’ve got your proj file setup, you shouldn’t be needing to edit it much.

And if you really need to, just reverse the above process to turn the schema validation back on.

Creating a Dummy Build Project In VS2008

5
Posted by admin on August 3, 2009 – 2:45 am
Filed under MSBuild

In the process of automating my build process for a project in 2008 (and VB6), I needed to have several additional steps happen during the build.

If the steps were directly related to a specific project within the overall solution, I could just alter the VBPROJ or CSPROJ file directly to include the required targets, no big deal. But in this case, I needed these steps to happen AFTER the overall build (at least, the build of the normal .NET solution).

I’m also using TFS and its build system to (eventually) automate the entire build process from a standalone server. But I’d like for the build process to be executable from any old workstation (assuming the required prerequisite files and apps are installed) for debugging and troubleshooting purposes. Anybody that would say that automating the build process isn’t an entire development project in it’s own right is out of their mind!

Furthermore, I wanted to impact the overall project build process as little as possible, so I really didn’t want to go traipsing around in the TFSBuild.proj file, or altering any of the TFS config at all if possible.

I knew that VBPROJ and CSPROJ files are nothing more than MSBUILD xml config files, which can be easily extended. But I wanted/needed something that would execute AFTER all the projects in my solution were built (and possibly before, but the same principle applies).

Then it struck me. What about a dummy project? One that could participate in the build process just as every other project in the solution, but which didn’t ACTUALLY build any specific .net assembly at all.

First step, add a new project. I picked an “empty VB project” for this.

image

But when I did this, I got an error

image

Rats, VS was insisting that the project have a sub Main, and I don’t really want anything compiling in this project. But a little poking around in the VBPROJ file turned up this IMPORTS line

   <Import Project="$(MSBuildBinPath)\Microsoft.VisualBasic.targets" />

Remove it, and the Sub Main error goes away. Those VisualBasic.targets are what controls that validation.

If you try and build the project at this point, though, you’ll get an error that no “Build” target exists in the file. That’s because the default Build target is defined by that VisualBasic.Targets line, and we just removed it.

So, I added my own Build target near the end of the file, with some testing tasks in it just to see what happens.

  <Target Name="Build">
      <Message Text="The Test build was called" />
      <Exec Command="Pause" />
  </Target>
  <!-- To modify your build process, add your task inside one of the targets below and uncomment it. 
       Other similar extension points exist, see Microsoft.Common.targets.
  <Target Name="BeforeBuild">
  </Target>
  <Target Name="AfterBuild">
  </Target>

Interestingly, the Message text does NOT display in the VS output window, but the EXEC task does (and the build is NOT paused by the PAUSE command).

image

Also, I’ve left them in for reference above, but when you remove the import of the Microsoft.VisualBasic.Targets file, the BeforeBuild and AfterBuild targets are effectively moot. They won’t do anything anymore, so you can remove them.

Editing the Project in the IDE

One problem that you’ll run into immediately with all this is that the Visual Studio IDE wants to handle those VBPROJ files special. And if you have the VBPROJ file opened in the IDE, the ONLY way you can change it is via the VS GUI. But the VS GUI doesn’t give you access to all the nooks and crannies of a VBPROJ file that you need to mess with for this to work.

One way to get around this problem is to unload the project (Right click on the Project in the Solution Explorer and select unload. Then Right click on the project again and you should see a “Edit your project name” menu item. Select it and your project should load up in VS just like any other XML file.

Then, when you’re ready to test again, right click the project and select reload.

The downside is that’s an awful lot of unloading and reloading. Quite simply, it’s a pain.

Luckily, there’s another way.

What if I put all my “real” targets in a plain ol’ PROJ file, and added it to this dummy project. After a few tests, it was clear that if you do this, you can easily open this “sub-PROJ” project file directly from within the VS IDE, no unloads or reloads necessary. But, I’d need a way to start up my sub project from within the actual dummy VBPROJ file.

Turns out, MSBuild has several actions that essentially let you “fire off” other targets as if they were “subroutines”.

The first I checked out was CallTarget. Unfortunately, it can only call targets within the current project file, and what I really wanted was to fire a target in a different project file.

Then I read up on the MSBuild task. It does allow you to specify both a Project to build (which could be a SLN solution file, but that’s another topic), and a Target within that project. AHA! The answer.

So I setup a simple test. First, my altered dummy VBPROJ file

<?xml version="1.0" encoding="utf-8"?>
<!-- This is a dummy build project for Post Build steps -->
<Project DefaultTargets="Build" ToolsVersion="3.5" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
    <Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
    <ProductVersion>9.0.30729</ProductVersion>
    <SchemaVersion>2.0</SchemaVersion>
    <ProjectGuid>{4D124D21-B683-40F2-89C3-658AA1B51DE9}</ProjectGuid>
    <FileAlignment>512</FileAlignment>
    <MyType>Empty</MyType>
    <TargetFrameworkVersion>v3.5</TargetFrameworkVersion>
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
  </PropertyGroup>
  <ItemGroup>
  </ItemGroup>
  <ItemGroup>
    <ProjectsToBuild Include="PostBuild.proj" />
  </ItemGroup>
  <ItemGroup>
    <Folder Include="My Project\" />
  </ItemGroup>
  <Target Name="Build">
    <MSBuild Projects="@(ProjectsToBuild)"/>
  </Target>
</Project>

The key element is the ProjectsToBuild item and the MSBuild task of the Build target.

That ProjectsToBuild item should cause the PostBuild.proj file to show up in the VS Solution explorer as just another file you can open like normal and easily edit.

Then, in the PostBuild.proj file,

<?xml version="1.0" encoding="utf-8" ?>
<Project ToolsVersion="3.5" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" >
   <!-- Must Set MSBuildEmitSolution envvar to 1 to cause msbuild to convert a SLN to a proj file -->
   <!-- Define various properties to use during the build here -->
   <PropertyGroup>
   </PropertyGroup>

   <Target Name="Build">
      <Exec Command="Echo Did the build" />
   </Target> 
</Project>

Now, if you build the project from VS and look at the Output window, you show see the “Echo Did the build” item.

Success!

Problems in Paradise

Unfortunately, all is not roses with this approach. As a test, I changed “Did the build” to “Did the build2” in my PostBuild.proj file, then rebuilt the project.

The output didn’t change. At all. Apparently, VS caches the projects in some fashion and only rereads them in peculiar circumstances. I say peculiar because even UNLOADING the project and RELOADING it did not force the PostBuild.proj changes to be read in. Only unloading all of Visual Studio and reloading it worked. This is specifically not what Microsoft alludes to as being correct behavior here, excerpted:

Editing Loaded Project Files

Visual Studio caches the content of project files and files imported by project files. If you edit a loaded project file, Visual Studio will automatically prompt you to reload the project so that the changes take effect. However if you edit a file imported by a loaded project, there will be no reload prompt and you must unload and reload the project manually to make the changes take effect.

But, no matter what I tried, the only thing that seemed to force a reload of cached project files was reloading VS.

Then I happened to notice two other build tasks, Vbc and VCBuild. They wrap the VB and VC compile processes, essentially doing the same thing. But looking at their parameters, they are VERY different beasts. the biggest difference is that Vbc appears to take a list of Source files as an argument, whereas VCBuild is almost identical to MSBuild. It takes Projects and Targets parameters. Hmm, could it be?

I replaced “MSBuild” with “VCBuild” and tried again. Surprisingly, even though my PostBuild.proj file has nothing to do with Visual C, VCBuild happily compiled away, rereading the PostBuild.proj file as I’d wanted. Change the PostBuild.proj file from the IDE, save and Build, and the changes are picked up immediately and compiled just as I was looking for.

The altered dummy VBProj file.

<?xml version="1.0" encoding="utf-8"?>
<!-- This is a dummy build project for Post Build steps -->
<Project DefaultTargets="Build" ToolsVersion="3.5" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
    <Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
    <ProductVersion>9.0.30729</ProductVersion>
    <SchemaVersion>2.0</SchemaVersion>
    <ProjectGuid>{4D124D21-B683-40F2-89C3-658AA1B51DE9}</ProjectGuid>
    <FileAlignment>512</FileAlignment>
    <MyType>Empty</MyType>
    <TargetFrameworkVersion>v3.5</TargetFrameworkVersion>
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
  </PropertyGroup>
  <ItemGroup>
  </ItemGroup>
  <ItemGroup>
    <ProjectsToBuild Include="PostBuild.proj" />
  </ItemGroup>
  <ItemGroup>
    <Folder Include="My Project\" />
  </ItemGroup>
  <Target Name="Build">
    <VCBuild Projects="@(ProjectsToBuild)"/>
  </Target>
</Project>

But, there’s a problem here too. If you’ve configured the output verbosity level for MSBuild to be something other than the default, VCBuild doesn’t appear to honor the change, and ends up using a fairly minimal output level. That can make debugging far more painful than it should be. None-the-less, it does work.

One more shot

Given that the verbosity level doesn’t follow through, I decided to make one last attempt. Instead of invoking the MSBuild task, why not invoke the MSBuild.EXE itself via an Exec task? Since VS has already invoked MSBuild to perform the outer project build, you can use the MSBuildBinPath property to know where to invoke MSBuild from, so the dummy VBProj file ends up looking like this.

<?xml version="1.0" encoding="utf-8"?>
<!-- This is a dummy build project for Post Build steps -->
<Project DefaultTargets="Build" ToolsVersion="3.5" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
    <Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
    <ProductVersion>9.0.30729</ProductVersion>
    <SchemaVersion>2.0</SchemaVersion>
    <ProjectGuid>{4D124D21-B683-40F2-89C3-658AA1B51DE9}</ProjectGuid>
    <FileAlignment>512</FileAlignment>
    <MyType>Empty</MyType>
    <TargetFrameworkVersion>v3.5</TargetFrameworkVersion>
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
  </PropertyGroup>
  <ItemGroup>
  </ItemGroup>
  <ItemGroup>
    <ProjectsToBuild Include="PostBuild.proj" />
  </ItemGroup>
  <ItemGroup>
    <Folder Include="My Project\" />
  </ItemGroup>
  <Target Name="Build">
    <Exec Command='$(MSBuildBinPath)\MSBuild.exe /v:diag "@(ProjectsToBuild)"'/>
  </Target>
</Project>

I’ve added the /v:diag switch to force diagnostic level verbosity, but that is easily changeable.

Lo and behold, I get the full output from BOTH MSBuild invocations in the Visual Studio Output window.

But What About Rebuild

At this point, performing a Build from VS will work exactly as expected, but if you try a Rebuild, you’ll get an error about your project file missing a Rebuild target.

In this case, a rebuild is the same as a build, so I’ll just make them dependent tasks.

<Target Name="Build">
    <Exec Command='$(MSBuildBinPath)\MSBuild.exe /v:diag "@(ProjectsToBuild)"'/>
</Target>

<Target Name="Rebuild" DependsOnTargets="Build" />

Notice the new Rebuild target above, and that it “Depends On” the Build target. That’s the only change necessary.

Conclusion

At this point, I have a project that I can add to any solution, which itself contains an MSBuild proj file which automatically builds when its host project needs to be built. And this would apply to builds within the IDE as well as builds from TFSBuild or any other automated build system.

Using some conditions and properties, it would be fairly easy to control when the project compiled (for instance, ONLY during a TFSBuild build, or only in the IDE when in RELEASE configuration.

And best of all, it’s all still completely editable from within the Visual Studio IDE!

Controlling MSBuild with Environment Variables

218
Posted by admin on July 31, 2009 – 4:38 am
Filed under MSBuild

It’s relatively trivial to control MSBuild from within Visual Studio by simply selecting different configurations (typically there are two, DEBUG and RELEASE, though you can create as many as you need).

However, for large projects, switching configurations is not a speedy process, and maintaining multiple configurations can be a chore.

I’ve been working toward automating the build process at my company and one element that I really wanted to implement was selective control of the build process.

For instance, in some cases, I might want to build locally, from within VS, and only build the app itself, but in other cases, I might want to build the Windows portion of the app AND the web portion, and in still other cases, I might want to build the entire app AND compile the InstallShield files to build an MSI installer file.

Obviously, there would come a point where the combinations would get unwieldy using configurations.

I was looking for a solution when I realized that MSBuild maps ALL environment variables by default as “properties” that are easily queried and used in conditions within the build script.

Then, I just needed a way to easily set those environment variables.

I’ve long known that the SET batch command only sets environment variables for that particular DOS session, so that wouldn’t work for this purpose. But in researching this, I discovered the SETX command, exactly like SET, only it sets global environment variables instead of those in the current session. This means that if you invoke, say:

SETX MyVar “This is a test”

The variable MyVar will not be created in the current DOS session, but, close it, and open an new DOS session, and it will be set there!

The Concept

So, to automate the selection, all I should have to do is:

  1. Define the environment vars and their uses
  2. Create batch files that set those variables
  3. Create one more batch file to clear all the variables
  4. Add the batch files to my VS project somewhere

Then, when I want to set the variables one way or another, just run the batch file from within VS, then start a build.

The Batch Files

All that’s required in the batch files is a simple SETX for each variable you need to set.

REM Enabled Building Install
SETX BuildInstall "true"

You might also want to include a bat file that clears all the variables (to “reset” things, if you will), but otherwise, this is all that’s necessary.

To make these BAT files easy to use I created a “Folder” in my solution, dropped the BAT files into that folder and then defined an “Open With” action to open the bat files with CMD.EXE.

Unfortunately, it wasn’t quite that simple. If you right click on a BAT file that you’ve added to the solution, you should see an “Open With” option on the menu. Click it and you’ll get the “Open With” dialog.

Click Add on that dialog and you’ll get something that looks like this

image

The problem is, you can’t specify command line parameters in with the program name. It’s just the name.

To get around that, I created a “RUNBAT.BAT” file, and put it somewhere on my PATH. It’s contents are simple:

REM Just run a passed in bat file 
cmd.exe /c "%1"

Essentially, it’s just a wrapper around CMD.EXE that invokes whatever BAT file name is passed in on the command line.

But now, you can point that “Program Name” field in the “Add Program” dialog to your RUNBAT.BAT file, and presto! You can open that BAT file with just a right click, “Open With”

Even better, select your RUNBAT program entry and click the “Set as Default” button, and now, when you double click the BAT file from within Solution Explorer, the BAT file will automatically be run.

One Final Trick

Unfortunately, that won’t quite be enough to make things work. The thing is, Visual Studio is smart, sometimes too smart for it’s own good. In this case, it’s all about caching. VS caches all the environment vars at the time that it loads, and when you invoke a build, it supplies those cached env vars, and not the currently active set. That would mean you’d have to exit and reload VS each time you wanted to change any env vars to control the build. Definitely not a workable solution!

But there is a way around this.

The Microsoft MSBuild Extensions Pack contains an “EnvironmentVariable” task that explicitly retrieves or sets environment vars from the active env var buffer, and NOT from VS’s cache.

Use it from your MSBuild script like this:

<Target Name="PerformBuild">
     <MSBuild.ExtensionPack.Computer.EnvironmentVariable TaskAction="Get" Variable="ForceBuild" Target="User">
         <Output PropertyName="ForceBuild" TaskParameter="Value"/>
     </MSBuild.ExtensionPack.Computer.EnvironmentVariable>
....

You use the Get action, name the variable, and be sure to use the “User” target (this retrieves User level environment variables, which is where the SETX command will save them). Then, it stores the variable’s value in the property named by the “PropertyName” argument.

You now have the value of that variable in a property that you can easily use in conditions on tasks, or other properties, or whatever. In this example, I check if the “ForceBuild” property that was just retrieved above, is blank, if it is, then the ForceBuild property value is set to “false”.

<PropertyGroup>
    <ForceBuild Condition="'$(ForceBuild)' == ''">false</ForceBuild>

Accessing the environment variables in this fashion, your build script will see the values of those variables as they’ve been set by your BAT files, even if those BAT files were executed from within Visual Studio.

It’s pretty simple, clean, and easily maintained from within Visual Studio. All qualities I really like!

If you know of a simpler way, I’d love to hear about it!

MSBuild Properties Don’t Always Evaluate Properly

4
Posted by admin on July 26, 2009 – 3:19 am
Filed under MSBuild, Troubleshooting

image In my travails with MSBuild lately (the latest 3.5 version), I recently ran into some rather puzzling behavior. Essentially, it boils down to this.

If you set a property value via a PropertyGroup element within a target, and then use CallTarget to call another target, the property values you set won’t be visible to conditions on that called target.

If, on the other hand, you set those same property values from a PropertyGroup within a different target that your main target refers to via the DependsOnTargets attribute, then those properties will be visible to conditions on the called target.

Yeah, you read that right <g>

Here’s a test MSProj project file that illustrates the problem.

<?xml version="1.0" encoding="utf-8" ?>
<Project ToolsVersion="3.5" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" >
    <!-- Define various properties to use during the build here -->
    <PropertyGroup>
      <TestValue>false</TestValue>
    </PropertyGroup>

    
    <!-- A DependsOnTargets target -->
    <Target Name="DetermineTestValue">
        <PropertyGroup>
            <TestValue>true</TestValue>
        </PropertyGroup>
        <Message Text="TestValue=$(TestValue)" />
    </Target>

    
    <!-- The test condition fails here and this target is skipped -->
    <Target Name="DoTheBuildSkips" 
            Condition="'$(TestValue)' == 'true'">
      <Message Text="Did the Build (skipped)" />
    </Target>

    
    <!-- The test condition DOES NOT fail here and this target is executed -->
    <Target Name="DoTheBuildWorks" 
            Condition="'$(TestValue)' == 'true'">
      <Message Text="Did the Build (works)" />
    </Target>


    <!-- Root Target just Executes two different test targets -->
    <Target Name="Build" DependsOnTargets="PropertyIsNotSeenProperly;PropertyWorks" />
    
    <!-- First test target, set the TestValue to true and use calltarget, this fails -->
    <Target Name="PropertyIsNotSeenProperly">
        <PropertyGroup>
            <TestValue>true</TestValue>
        </PropertyGroup>
        <!-- the TestValue is true at this point but false when evaluated from DoTheBuildSkips -->
        <Message Text="TestValue (place1)=$(TestValue)" />
        <CallTarget Targets="DoTheBuildSkips" />
    </Target>    

    <!-- Second test target, set the TestValue to true from the DependsOnTarget
         and the "DoTheBuildWorks" target is executed because the condition is true -->
    <Target Name="PropertyWorks" DependsOnTargets="DetermineTestValue">
        <!-- the TestValue is set to true from within DetermineTestValue and is ALSO true
             when evaluated from DoTheBuildWorks -->
        <CallTarget Targets="DoTheBuildWorks" />
    </Target>    

</Project>

Just drop it into a folder and run MSBuild. Be sure to use the /v:diag command line switch so you see the full output.

I piped the results to a file and (with some bits trimmed out) this is the result:

Microsoft (R) Build Engine Version 3.5.30729.1
[Microsoft .NET Framework, Version 2.0.50727.3074]
Copyright (C) Microsoft Corporation 2007. All rights reserved.

Build started 7/23/2009 4:25:13 PM.
Project "O:\Dev\Darin\Articles\MSBuild Property Failure\Test.proj" on node 0 (default targets).
Initial Properties:
ALLUSERSPROFILE = C:\ProgramData
...Many env vars cut out here
TestValue = false

Building with tools version "3.5".
Target "PropertyIsNotSeenProperly: (TargetId:4)" in file "O:\Dev\Darin\Articles\MSBuild Property Failure\Test.proj" from project "O:\Dev\Darin\Articles\MSBuild Property Failure\Test.proj":
Using "Message" task from assembly "Microsoft.Build.Tasks.v3.5, Version=3.5.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a".
Task "Message" (TaskId:0)
  TestValue (place1)=true (TaskId:0)
Done executing task "Message". (TaskId:0)
Using "CallTarget" task from assembly "Microsoft.Build.Tasks.v3.5, Version=3.5.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a".
Task "CallTarget" (TaskId:1)
Target "DoTheBuildSkips" skipped, due to false condition; ('$(TestValue)' == 'true') was evaluated as ('false' == 'true').
Done executing task "CallTarget". (TaskId:1)
Done building target "PropertyIsNotSeenProperly" in project "Test.proj".: (TargetId:4)
Target "DetermineTestValue: (TargetId:0)" in file "O:\Dev\Darin\Articles\MSBuild Property Failure\Test.proj" from project "O:\Dev\Darin\Articles\MSBuild Property Failure\Test.proj":
Task "Message" (TaskId:2)
  TestValue=true (TaskId:2)
Done executing task "Message". (TaskId:2)
Done building target "DetermineTestValue" in project "Test.proj".: (TargetId:0)
Target "PropertyWorks: (TargetId:5)" in file "O:\Dev\Darin\Articles\MSBuild Property Failure\Test.proj" from project "O:\Dev\Darin\Articles\MSBuild Property Failure\Test.proj":
Task "CallTarget" (TaskId:3)
Target "DoTheBuildWorks: (TargetId:2)" in file "O:\Dev\Darin\Articles\MSBuild Property Failure\Test.proj" from project "O:\Dev\Darin\Articles\MSBuild Property Failure\Test.proj":
Task "Message" (TaskId:4)
  Did the Build (works) (TaskId:4)
Done executing task "Message". (TaskId:4)
Done building target "DoTheBuildWorks" in project "Test.proj".: (TargetId:2)
Done executing task "CallTarget". (TaskId:3)
Done building target "PropertyWorks" in project "Test.proj".: (TargetId:5)
Done Building Project "O:\Dev\Darin\Articles\MSBuild Property Failure\Test.proj" (default targets).

Project Performance Summary:
      107 ms  O:\Dev\Darin\Articles\MSBuild Property Failure\Test.proj   1 calls

Target Performance Summary:
        0 ms  Build                                      1 calls
        5 ms  DoTheBuildWorks                            1 calls
        5 ms  DetermineTestValue                         1 calls
       11 ms  PropertyWorks                              1 calls
       21 ms  PropertyIsNotSeenProperly                  1 calls

Task Performance Summary:
       12 ms  Message                                    3 calls
       13 ms  CallTarget                                 2 calls

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:00.11

(I’ve highlighted the key lines)

This file essentially has a Build target that is just used to kick off two other targets, that are the actual tests.

The first, PropertyIsNotSeenProperly illustrates the problem. The second, PropertyWorks, works just fine.

Notice that the TestValue property starts out false, but in both cases, it does appear to get set to true.

However, at the point that the condition for the DoTheBuildSkips target is evaluated, the value of TestValue must be false, which should be impossible.

I’ve read that the CallTarget is a bit of a deprecated method anyway, so maybe this is all just a problem of old code that hasn’t been properly updated along with the rest of MSBuild.

But, I’ve found that using CallTarget on occasion can simplify some aspects of debugging larger MSBuild scripts, so that’s why I’ve used it. Further, for me anyway, it makes the script a lot easier to read when the steps that it entails are laid out in a list, one CallTarget after another, instead of one target being dependent on another, which depends on another, etc, etc.

Something to be aware of.

UnBatching in MSBuild With Team Foundation Server

0
Posted by admin on July 23, 2009 – 2:39 am
Filed under MSBuild, VB Feng Shui

I’ve been working on getting MSBuild setup and configured to handle some Continuous Integration builds for our company. One task that came up was needing to get a large batch of files from our TFS server and pull them down to the appropriate directories on a local machine.

But here’s the catch. I didn’t want to just pull everything  from those folders down. Some folders contain very large files that didn’t really need to come down at all, because the intent was that the build process would be building them.

So, it would have been relatively easy to use the MSBuild Extensions pack to get all the files recursively in my project.

    <Target Name="GetInstallFileSet">
        <MSBuild.ExtensionPack.VisualStudio.TfsSource TaskAction="Get" ItemPath="..\InstallFiles" WorkingDirectory="$(MSBuildProjectDirectory)" Recursive ="true" Force ="true" />
    </Target>

The WorkingDirectory parameter indicates where TFS will base all relative file specs from. The ItemPath indicates the folder location (relative to where the MSBuild proj file is located that this Target is in) that TFS should retrieve. The TaskAction of GET just retrieves all files in that folder, and the Recursive parameter tells TFS to get all files in that ItemPath and all it’s subfolders.

Pretty simple, but if there were large files anywhere in the path, TFS will obligingly retrieve them as well, which could suck up a lot of time and bandwidth.

So, first, how to specify only those files that I really want to retrieve? Easy, just use an ItemGroup.

    <Target Name="GetInstallFileSet">
        <ItemGroup>
            <InstallFileSet Include="..\..\Install;
                                     ..\..\Install\System;
                                     ..\..\Install\OtherFiles;
                                     ..\..\Install\DiskSet">
            </InstallFileSet>
        </ItemGroup>
    </Target>

The InstallFileSet ItemGroup will end up with these specific folder names (all relative to the path to the proj file the target is defined in). Unfortunately, I don’t see any straightforward way to “leave out” specific files, because using any wildcard specs when defining this ItemGroup would be based on files that already exist on the local workstation, and hence we run the risk of NOT getting newly added files that exist in TFS but not locally.

But, if those files happen to live in specific subfolders, we CAN leave out those subfolders from the list in the INCLUDE attribute of the Item definition above.

Ah, but what about recursion? In the above case, I specifically DO NOT want to recurse down from the first path in the list (..\..\Install), but I DO want to recurse on all other paths.

That’s where the ‘metadata’ aspect of MSBuild comes into place. Modify the ItemGroup slightly.

    <Target Name="GetInstallFileSet">
        <ItemGroup>
            <InstallFileSet Include="..\..\Install>
                <Recurse>false</Recurse>
            </InstallFileSet>
            <InstallFileSet Include="..\..\Install\System;
                                     ..\..\Install\OtherFiles;
                                     ..\..\Install\DiskSet">
                <Recurse>true</Recurse>
            </InstallFileSet>
        </ItemGroup>
    </Target>

Notice that I’ve split the InstallFileSet item into two pieces, and added the Recurse attribute to each piece.

Now, all the items are still in the single InstallFileSet ItemGroup, but one has a Recurse property of false, the others have it set to true.

Using the ItemGroup

All I have to do now is indicate how to use the ItemGroup that I’ve defined

...
        <MSBuild.ExtensionPack.VisualStudio.TfsSource TaskAction="Get" ItemPath="@(InstallFileSet)" WorkingDirectory="$(MSBuildProjectDirectory)" Recursive ="%(InstallFileSet.Recurse)" Force ="false" All="true" />
    </Target>

Adding this one TaskAction=”Get” line will no perform the Get from TFS on all those Items.

So I ran the build, and… Fail.

TFS gave me a “parsing” error on the ItemName argument. Apparently, it doesn’t like being passed a long list of semicolon delimited path names.

Gah. This is where the UnBatching comes in.

Batching Explained

As a build engine, MSBuild will generally try to “batch” multiple items together into one processing “call”, so that as few invocations of that call as possible are made, under the assumption that fewer invocations will be faster.

Unfortunately, in some cases, you really don’t want the processing batched. In this case, batching would cause multiple paths to be supplied to one invocation of TF.exe (the TFS client app), which, as I mentioned, doesn’t know how to deal with that.

The key to understanding batching is MSBuild will batch by default, but will separate out batches based on two things

  1. Any metadata attributes associated with the batch items
  2. How you reference the items in the batch.

1) is easy. In the above example, I’ve got two values of the metadata tag “Recurse”, so MSBuild by default will execute the GET task twice, once, with the Item(s) with Recurse=true, and once for those with Recurse=false.

But 2) is harder to grok. As it is now, I’m using ItemPath=”@(InstallFileSet)” to reference the itemgroup. That allows MSBuild to batch the items, but then it splits the batch by the Recurse attribute.

However, if I reference the items by the built in identity metadata tag, MSBuild will consider each item in the group has having unique metadata associated with it, so it will execute the GET task once for each item, do that by using %(InstallFileSet.identity) instead, as in:

...
        <MSBuild.ExtensionPack.VisualStudio.TfsSource TaskAction="Get" ItemPath="%(InstallFileSet.identity)" WorkingDirectory="$(MSBuildProjectDirectory)" Recursive ="%(InstallFileSet.Recurse)" Force ="false" All="true" />
    </Target>

If you build now, you’ll notice that the TF.EXE process is launched once for each path in the InstallFileSet ItemGroup, and that the Recurse attribute is applied appropriately depending on which item is being processed.

Conclusion

MSBuild is definitely powerful. In some ways, it’s simpler to deal with than the MAKE/NMAKE systems of old, and the source proj files are definitely more flexible in how they can be used.

But, many of the more advanced functions (that you’ll end up needing quite quickly) can leave you scratching your head. And the available documentation and examples often don’t help much.

The key is patience and experimentation.

Automating an InstallShield 2009 Build with MSBuild

0
Posted by admin on July 13, 2009 – 10:23 pm
Filed under MSBuild

I’ve been working for some time at automating the InstallShield build process from within an MSBuild script. After a successful attempt using an Exec task along with ISCmdBld.exe, I decided that I really wanted the InstallShield logging included in the MSBuild logging process.

I came across the Microsoft.Sdc tasks project on CodePlex and initially thought it’d work great, but, after a few tries, I discovered it was hardcoded to a very old version of InstallShield.

Then I happened to notice the “InstallShield” folder under the \Program Files\MSBuild folder. In that folder was a 2009 folder and in there were two files:

  • InstallShield.targets
  • InstallShield.Tasks.dll

Surely, this was what I’d been needing all along!

First step, google “InstallShield.Tasks.DLL”.

I ended up here, with a page detailing all the options for the InstallShield MSBuild task. This is for IS 2009, but there are other pages for IS 2008, etc. Looks like most of the parameters are the same though.

Next, hook up the tasks to my MSBuild project file.

<Project ToolsVersion="3.5" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="swiNumbering">
   <!-- Define various properties to use during the build here -->
   <PropertyGroup>
      <!-- Test for the InstallShield Extensions -->
      <InstallShieldTasksPath>$(MSBuildExtensionsPath)\InstallShield\2009\InstallShield.targets</InstallShieldTasksPath>
   </PropertyGroup>
   
   <Import Project="$(InstallShieldTasksPath)"/>

This makes the InstallShield task accessible to MSBuild.

Then, just add a target using that task:

      <InstallShield.Tasks.InstallShield
           InstallShieldPath="C:\Program Files\InstallShield\2009 StandaloneBuild\System"
           Project="@(InstallToBuild)"
           ProductConfiguration="Package"
           ReleaseConfiguration="MSI" />

Of course, your values for those parameters will likely be different. The @(InstallToBuild) property points directly to the ISM file you want to build. In my first attempt, I thought I could leave out the InstallShieldPath. Surely the task would look that up from the registry. No dice. You have to have it and it needs to point to your InstallShield System folder or you’ll get an error.

Presto! MSBuild now sees the output of the ISCmdBld process (as evidenced by the nice colored output in the DOS Window when I run MSBuild).

One step closer to complete “Continuous Integration”.

Problems with Microsoft.Sdc.Tasks.Tools.InstallShield

0
Posted by admin on July 11, 2009 – 5:02 am
Filed under .NET, MSBuild

I’m trying to automate the building of an MSI file using InstallShield’s “Standalone Build Manager” via MSBuild, and one nicety I was hoping to implement was integrating the output of the InstallShield build log into the results of MSBuild.

Originally, I’d used an MSBuild EXEC task to fire up ISCmdBld.exe to perform the build, which works just fine, but which writes the log file to a more or less arbitrary location within the InstallShield project folder structure.

After a bit of Googling, I noticed that the Microsoft.SDC.Tasks MSBuild Extension Library had an InstallShield task as a part of it. Great! Just use that to perform the compilation, and I’m all set!

Unfortunately, that was not quite the case.

Once I’d downloaded the files, extracted them to the proper locations, and modified the Microsoft.Sdc.Common.Tasks file to point to the right place, I was still getting an error when I attempted a build:

Message              = Object reference not set to an instance of an object.

After some investigation, including browsing the source on the codeplex site, I figured out at least part of the problem:

        protected override void InternalExecute()
        {
            #region Execute code

            Directory.CreateDirectory(buildpath);

            RegistryKey installShieldKey = Registry.LocalMachine.OpenSubKey(@"SOFTWARE\InstallShield\Developer\7.0");

The registry subkey referenced in the task points to an InstallShield 7 key, and I’m using IS 2009. <Sigh>

At this point, I’m back to using the EXEC task directly. Not optimal, but it does work.

Controlling MSBuild Verbosity Within the Output Window In Visual Studio

0
Posted by admin on July 11, 2009 – 4:59 am
Filed under .NET, MSBuild

I’ve build working a lot recently with MSBuild, specifically in relation to automating the build process for our companies products. In my case, our product is mostly .NET, with some VB6 projects still hanging around, and of course some database bits and InstallShield and DemoShield projects to wrap things up for a disc image. Pretty typical commercial product build process.

At any rate, I’ve been experimenting with an MSBuild project type that I could embed in my solution to control all this, not only from the IDE, but also automatically get hooked in with the TFSBuild process, when I really started needing a way to the the diagnostic logging level of the build process when MSBuild is run from within Visual Studio to perform the build.

Normally, VS runs MSBuild with a minimal logging level, so you really don’t see that much in the Visual Studio Output window. I wanted to alter that.

I’d been looking pretty fruitlessly until I came across Sara Ford’s blog here. Turns out, there’s an option in Visual Studio’s Options Window to do just this! It’s not Project specific, unfortunately, so if you set it to diagnostic and then compile a large project, the Output Window will be huge.

But still, nice to know it’s there.