Why Install Multiple MSIs

One of the most common questions I get from people in our organization is "Why do we install multiple MSIs?" These people want to understand why our lab builds many different MSIs instead of creating one MSI that installs all the software, or more accurately, one MSI for each product release.

There are many benefits to doing this and many downsides, all of which can be argued, but there is only one really improtant reason that made us change our install design to use multiple MSI files: compatibility between our releases.

Our company makes hardware products, each of which comes with a software CD that includes drivers and some applications that let the customer use the product. To give a better user experience and to simplify development and testing, our lab started sharing the applications between all of our product CDs. If a user purchases two of our products, they get one copy of the application that works with both products. This makes sense, right?

So, this means that we're going to be sharing binaries that get installed between product CDs. There were a couple options for doing this:

1. Create a single install CD that supports all the products
2. Have a single MSI for each install CD that shares components with MSIs on other CDs
3. Create an MSI for each "application" that gets shared and an MSI that is specific to each CD to install CD specific pieces.

Option 1 doesn't work for our lab, because there are too many products on release schedules that are too different. Also, it wouldn't be a great user experience for any user who buys one of our products to have to install drivers and support for 100 other products at the same time.

The problem with option 2 is basically that Windows Installer does a really good job of reference counting files, but not reference counting "features" or "applications." Let me explain... Let's pretend one of the applications that we're installing is texteditor.exe. Let's say we have two products, product A and product B. Let's say we'll create one MSI to go on each of the products' CDs: A.msi and B.msi. A.msi and B.msi will then each have a component to install texteditor.exe, that will have the same component ID and will install the file to the same location. Most likely there would be a texteditor merge module that has that component. Installing and uninstalling the two MSIs on the same machine works fine, leaving working software, because the texteditor.exe component is reference counted by the two MSIs.

Well, everything is working in this example, so what's the problem? Let's pretend that version 2 of texteditor.exe includes more functionality and now depends on textformat.dll. The texteditor merge module adds a new component to install the new dll and product C's MSI gets built that uses the new version of texteditor. Now when you install C.msi and A.msi, texteditor.exe is version 2 and works fine, because textformat.dll is on the system. But that dll is reference counted only once, by C.msi. If the user chooses to uninstall C.msi (and leave A.msi), texteditor.exe stops working even though product A is still installed. There are a couple workarounds for this problem: (a) make texteditor.exe gracefully degrade to its previous functionality if the dll isn't present, (b) install texteditor.exe v2 side-by-side with v1, or (c) force the customer to uninstall and reinstall all products. (c) is a crappy user experience, and with the number of products we sell and incidence of overlap, it just doesn't make sense. (b) was not a good option, because we had the desire to give the user only 1 instance of each application, because it wouldn't be clear to the user which version of the app to use with each product. We wanted the user to have only one suite of applications that would function with all installed products. (a) seems like a reasonable idea, but the applications we were installing were much more complicated than a simple text editor, and we made a technology change where we went from C++ to .NET applications, which changed almost all of the files that got installed -- except for the name of the exe that had shortcuts in the start menu. So, those exes would need to contain a whole boatload of logic (and code) to determine what functionality it could make available to the user and it would have to carry around all the old code.

So, instead of dealing with all these problems with our option 2, it made more sense to go with option 3 -- make each application a standalone MSI that would get upgraded independently of the rest of the software solution. Then each product CD would include a ProductA.msi and a TextEditor.msi and an MSI for everything else that was shared.

One of the other big benefits of this design was that each "application" or "feature" could be developed and tested independently. Then each product CD could pick up a fully tested MSI. Also each team creating an application could build their MSI on a separate schedule and wouldn't have to depend on products A, B, C, etc. to build MSIs so they could test their applications.

So, that's the big reason why we chose to deploy our software in a set of several MSIs.

But, the big problem that this design created was how to get all the MSIs installed and uninstalled. Which is a much longer discussion...

Fun with MsiEmbeddedChainer

One of the features of the recently released Windows Installer 4.5 is the ability to add an "embedded chainer" to your MSI. Basically this is an executable that you write that gets put into your MSI that gets called during the install. The executable can then call MsiInstallProduct to install additional MSIs.

I just wanted to put together a list of the things I learned about the embedded chainer (and embedded UI) so other people don't have to go through the same discovery process to learn the same things. The things I learned are:

1. To show only one progress bar while installing chained MSIs, you must use embedded UI as well.


2. To show only one elevation prompt, the chained MSIs must be digitally signed and the MSI must include the MsiPackageCertificate table (and thus a MsiDigitalCertificate table).


3. To uninstall properly, the chained MSIs must not embed their cab files. If there are embedded cabs, when the MSIs get cached locally, the embedded cabs are removed from the MSI's _Streams table which invalidates the digital signature, and therefor the embedded chainer will fail to uninstall the MSIs on Vista (assuming the MSIs require elevation to install and uninstall).


4. Embedded UI runs instead of the InstallUISequence, so results of standard actions like AppSearch and costing cannot be used in your embedded UI dll.


5. An embedded chainer runs after the main MSI's install execute sequence and therefore does not have access to the database (it does not get passed a handle to the install). In addition, calling MsiOpenProduct in an attempt to read the main MSI's database results in the embedded UI being closed. So, if you want to read anything out of the MSI's database, you need to create a custom action.
   

Here's my detailed description:

If you've ever needed to install multiple MSIs as part of your product's deployment, you might see how this is useful. Now, instead of having to write an executable that starts up, shows some UI, then installs multiple MSIs while showing progress, you can just write an exe that installs multiple MSIs. Windows Installer will install your main MSI, then launch your exe. It will also maintain a single transaction for all of the MSI installs, so if one fails, then it gracefully cleans up all of the MSIs that it has installed so far. Pretty cool, huh?

For years, our lab has maintained a set of executables that runs some UI, does some basic install preparation (system checking), and then runs a set of MSIs, then creates an Add/Remove Programs entry to call our uninstaller that will uninstall that same set of MSIs. So, I decided to try out this embedded chainer to see how it might help us reduce the amount of code we need to write and maintain.

Our desired product install experience is to give the user the experience of installing one software package while actually installing multiple MSI packages. So, I set out to create an MSI that would show UI, install some stuff, then its embedded chainer would install some more MSIs. The chainer would install those MSIs silently and show the install progress on the main MSI's progress screen. At the end, the main MSI would show a finish screen (assuming everything installs correctly).

Here's where I came across the first problem... I wanted to just use the UI from the main MSI, and show the progress from the chained MSIs on that progress bar. I didn't want any basic UI (or full UI) from the chained MSIs to show up on top of my main MSI's UI while the chained MSIs install. So my chainer calls MsiSetInternalUI (INSTALLUILEVEL_NONE...), then MsiInstallProduct. However, when I do this, the main MSI's progress bar just sits there, doing nothing while the chained MSIs install. I tried several things to get this to work, but in the end asked a question on Microsoft's connect site asking for help. The answer I recieved was that I must use embedded UI to be able to get a single progress bar for all the MSI installs.

So, next I needed to create an embedded UI dll and add the MsiEmbeddedUI table to my MSI. While I was probably going to use an embedded UI dll in our install to create a nice looking UI anyways, I was annoyed that I was required to use embedded UI to get the right user experience.

So, I threw together an embedded UI that didn't do much other than show a progress bar. I also changed my embedded chainer so that when it calls MsiJoinTransaction, it passes MSITRANSACTION_JOIN_EXISTING_EMBEDDEDUI so that the embedded UI doesn't close. Then I tested it. So far so good.

My next step was making my chainer reusable. So, I decided to create a custom table that described what MSIs the chainer needed to chain. Makes sense, right? I put basic information in there -- like the relative path to where to find the MSI and some sort of condition on when to install and uninstall the MSI (component install states, for example). While running, I wanted the chainer to open up the database, read this information, then go about calling MsiInstallProduct on all the chained MSIs. Since the chainer doesn't get an install handle passed to it (it gets a transaction handle), I figured I would use MsiOpenProduct passing the product code of the main MSI that just got installed. My chainer command line would take the product code of the MSI, use that to open the database, then read the custom table. This works beautifully, however, as soon as I start calling MsiCloseHandle once I'm done reading the database, my embedded UI gets called back with ShutdownEmbeddedUI. Again, I took a [virtual] trip to MS's connect site to post a question. The response -- a defect, but they won't have time to fix it. Since I posted the question after the release of WI 4.5, I wasn't surprised by this answer :) So anyways, now I needed another way to read that data. I ended up creating a dll custom action that runs during the InstallExecuteSequence and reads my custom table and sets up the command line for my chainer exe. It actually works out better in the end, I think, because it has the opportunity to use MsiEvaluateCondition, check component states, and a variety of other useful things.

Well, at this point, I felt pretty good, because things were working well (on my XP test machine). Then I decided to move my testing over to Vista. Here came my next set of problems. Basically my chainer kept failing. Logs showed that my chained MSIs were failing with error codes meaning that elevation was required. With some experimentation, I found that if I let the chained MSIs show UI, I would get prompted for elevation each time a chained MSI was installed. Without the chained MSIs showing UI, their installs simply fail. I then set about to figure out how to get one elevation prompt and not have the MSI installs fail when run silently. A teammate pointed me to this information about the MsiPackageCertificate table. Well, I felt pretty dumb about not finding that myself, but we were happy that our elevation prompt problems were solved by signing our MSIs and adding the MsiPackageCertificate table. Well, what happened next was another stupid mistake on my part... Now that I successfully got my chainer to install all the MSIs, uninstall failed, because of the same problem. It took me another message on MS's connect site and a response from someone to realize my mistake of embedding cab files in my chained MSIs. When the MSIs get cached on the system during install, embedded cabs are removed from the _Streams table, which invalidates the digital signature. So, even though I had a MsiPackageCertificate table in my main MSI, the chainer failed to uninstall the chained MSIs, because the signature of the chained MSIs was invalid. Some quick build changes to leave the cab files external, and we were back in business.

My next step was actually doing something useful in my embedded UI. Our install needs to show the same sort of UI that everyone shows during install -- EULA, choose where to install, choose the features to install. Here's my problem with embedded UI. My hope when reading about embedded UI and the embedded chainer was that I would be finally able to install multiple MSIs, but still use Windows Installer standard actions to do some of the work that we have been writing code to do. Basically I wanted to be able to get a list of the features to install and use MsiGetFeatureCost to determine how much space each feature will take, so that I can show that on the UI screen where the users chooses what to install. I also wanted to be able to use AppSearch or MsiGetComponentState to determine whether chained MSIs were already installed so that I could show that information on the feature selection screen as well. But unfortunately, the embedded UI runs instead of the InstallUISequence, so I have to write my own code that would do the same thing. I actually experimented with returning INSTALLUILEVEL_FULL from my InitializeEmbeddedUI function and letting the InstallUISequence run. But if I do that, then I don't get another opportunity to reduce the UI, so my chained MSIs all run with full UI as well, so I end up with multiple progress bars, which I was trying to avoid. I also attempted to call MsiDoAction from my embedded UI to run some of those actions (AppSearch, CostInitialize, CostFinalize, etc), but all those calls all failed as well (as I expected they would). So now I get to go write some code to calculate feature costs and determine what is already installed.

That's what I've learned so far. I'm still not quite sure if using an embedded UI and an embedded chainer is "better" than creating a setup.exe that shows its UI, creates a progress bar, calls MsiSetExeternalUIRecord, and calls MsiInstallProduct several times. But I'll keep using it until I run into some real problems :)

My First Post

This is my first post. I created this post to share some of the information I have learned over the years as an install developer. I've used Windows Installer for years -- hand editing .msi files, using InstallShield to create standalone .msi files, writing a large install engine that uses the Windows Installer API to install a whole set of .msi files, and even writing a pretty comprehensive tool that checks for errors in our software lab's .msi files.

Since I've been playing with WI4.5 recently and haven't seen any good information on the internet, I thought it might be useful to have some more information out there.

Hopefully someone finds this useful. I'll probably find it useful to simply write these things down.