semi/signal

A long time ago I wrote some code to play CD audio tracks for a game engine I was writing. CD audio (via mixed mode CDs) was popular for video game music at the time because it allowed for high quality music without having to deal with large un-compressed audio files (e.g. uncompressed WAV) or relatively expensive decompression algorithms with compressed files (e.g. MP3).

I cleaned up the code a bit and decided to post it here. It’s surprising simple code as it uses the very high-level Media Control Interface (MCI).

Compiling and Linking

You must link with winmm.lib

cdaudio.h

cdaudio.cpp

main.cpp (example showing how to use cdaudio functions to play track)

I’ve always considered the Windows Registry to be a pretty awful piece of technology and a terrible way to store system and application settings. That said, if you’re doing Windows development, you’re probably going to have to touch it at some point. I was digging through some old C++ code, and came across some code to get the current version of Flash installed on the system (I can’t remember why I need to do this at all), which I think serves a good example of how to query the registry.

The code required some modification as the name name of the key and value has since changed (FlashPlayerPlugin was previously FlashPlayer, Version was previous CurrentVersion).

A while back I wrote about 2 applications, The KMPlayer and JCreator not behaving well when attempting to run both concurrently (see post).

Now, I have an incredibly weird situation on my system. KMPlayer and JCreator don’t play nice together. If they’re both open, some JCreator panels and menus are suddenly blank and don’t refresh and the side tabs panel is transparent, showing thru to the desktop. As for KMPlayer, I can’t open anything, clicking play (which plays the last file opened when nothing else has been loaded) does nothing, and certain items are mysteriously missing from the context menu. This hasn’t been a big deal for me, and I still use both JCreator and KMPlayer, but it would be nice if they worked together. Also, I have to wonder, what is the common component causing the conflict here, what would a media player and a java IDE both be using or trying to access concurrently? (assuming there is a conflict for a common component, which I suspect might be the issue here)

My suspicion was wrong, it was not a conflict between the applications or a common component, it was the system running out of User objects. In Winforms (and I suspect most other GUI toolkits as well) any GUI control or window will consume at least 1 user object (more complex controls, with multiple sub-components will consume more User objects), and when the system or process hits the limit (65,536 for the user session, 200 – 18,000 per-process; default is 10,000 on Windows XP), creation of new User objects will fail, even if the system has enough memory to support whatever it is that’s being created. On the .NET Framework, you’ll notice this if you get an exception that looks similar to the following,

System.ComponentModel.Win32Exception: Error creating window handle. at System.Windows.Forms.NativeWindow.CreateHandle(CreateParams cp) at System.Windows.Forms.Control.CreateHandle() at System.Windows.Forms.Control.CreateControl(Boolean fIgnoreVisible) at System.Windows.Forms.Control.CreateControl(Boolean fIgnoreVisible) at System.Windows.Forms.Control.CreateControl()

I’m still puzzled as to why there simply isn’t a limit based on available memory, but I haven’t been able to find a whole lot written on User objects in general much less details on why they exist.

I was looking for a fast way to get the number of folders in a directory. I didn’t need anything else, just the number of folders – this was for a folder tree, and to determine whether or not to display an expansion arrow that would pull and list the subfolders when clicked. The brute-force (read: slow) way to accomplish this is just to get an array of folders and get its length. I thought such information would exist within the directory metadata, but it turns out it doesn’t, and the brute-force way is the only way.

Quicker (quickest?) way to get number of files in a directory with over 200,000 files

How do I find out how many files are in a directory?

The stackoverflow questions refer to files, but files and folders are seen/enumerated the same way by Windows (FindFirstFile, FindNextFile), so the same limitation exists.

This post on The Old New Thing explains the reason for the lack of such metadata.

The file system doesn’t care how many files there are in the directory. It also doesn’t care how many bytes of disk space are consumed by the files in the directory (and its subdirectories). Since it doesn’t care, it doesn’t bother maintaining that information, and consequently it avoids all the annoying problems that come with attempting to maintain the information.

Another issue most people ignored was security. … If a user does not have permission to see the files in a particular directory, you’d better not include the sizes of those files in the “recursive directory size” value when that user goes asking for it. That would be an information disclosure security vulnerability.

Yet another cost many people failed to take into account is just the amount of disk I/O, particular writes, that would be required. Generating additional write I/O is a bad idea in general…

Completely understandable… but it still sucks on the application developer’s end. One solution I toyed around with is using the folder info to make a cache, instead of just getting the count and tossing the array to the garbage collector. This means, everytime a folder is expanded, the subfolders within the subfolders of the directory being expanded would be pulled and the necessary UI widgets would be created. However, this isn’t always ideal, especially as you go deeper into a folder tree, as you can end us wasting time pulling and creating UI widgets for a lot of folders which will never be seen by the user.

I finally nailed down an annoying little bug tonight. In a certain app, I’ve been calling CreateProcess() to periodically spawn a process, do some work, and shut down. Unfortunately after running several hours, the app would fail with an exception saying: insufficient quota to complete the requested service. After a fair bit of monitoring the app’s activity, I notice that the handle count of the main process, slowly and surely, kept going up. After a bit of trial-and-error, disabling modules systematically, I finally noticed that this was occurring when I spawned off the child process I mentioned.

Reading the MSDN docs for CreateProcess(), I finally got to the root of the issue; the handles returned in the PROCESS_INFORMATION struct must be closed via. CloseHandle() or the handles are kept open, even though the child process has terminated.

If the function succeeds, be sure to call the CloseHandle function to close the hProcess and hThread handles when you are finished with them. Otherwise, when the child process exits, the system cannot clean up the process structures for the child process because the parent process still has open handles to the child process.

This oversight was probably due to the fact that I was working in C# (I was P/Invoking this stuff) and lulled into a false sense of safety, thinking the garbage collector would take care of stuff like this, but from working with files, various streams, sockets, etc. I realized that C# doesn’t really close handles automatically. Now that has me thinking, why not? Wouldn’t handle management be very similar to memory management?

So, a little story. I was working on some code that periodically spawns off a child process to do its thing, then read the results in the output stream from the parent process. Now, this periodic spawning was being done asynchronously in the parent process via. a thread. All was well and good, but there was also another thread in the parent process which created a file with a temporary name, downloaded and wrote a bunch of bytes into it, then renamed the file to a proper name. Unfortunately, I began to notice that the rename operation was failing as the parent process couldn’t get access rights to the file that needed to be renamed due to a sharing violation. I checked, double checked, and triple checked that I was closing the file, and everything looked perfect. After a bit of headbanging, I figured I should probably check to verify that another process is not holding the file handle. I then ran a wonderful utility called Handle to see exactly what processes had the file handle and, to my surprise, it was a child process that was spawned. Now, this was weird as hell as the child process did nothing with this file or directory or any file i/o whatsoever. After a bit more headbanging, I made the unpleasant discovery of child process inheritance. This forum thread, discussing a similar issue with a .NET’s TcpListener, actually pointed me to the issue.

Now most of this was in C#. One tricky aspect of this issue is that the file i/o mentioned was done in a bit of native code using fopen/fclose. I never encountered the issue with similar code using a C# FileStream. My assumption is that the file handles from the C# functions were explicitly not inheritable, while those created by fopen were. The underlying Win32 CreateFile API function does provide for this feature, but it’s not exposed via. fopen.

If this parameter is NULL, the handle returned by CreateFile cannot be inherited by any child processes the application may create and the file or device associated with the returned handle gets a default security descriptor.

CreateFile ignores the lpSecurityDescriptor member when opening an existing file or device, but continues to use the bInheritHandle member.

The bInheritHandle member of the structure specifies whether the returned handle can be inherited.

Now it was time for the really tricky part, fixing this. I didn’t want to change the native code (which in retrospect may have been an appropriate course of action and much easier to do), so instead I tried to see if I could prevent the process from inheriting the handle. The Win32 CreateProcess function has a bInheritHandles argument that can be set to false to prevent child processes from inheriting handles. Unfortunately, I was using the C# Process class and it provides no means to set such a flag. I eventually P/Invoked the CreateProcess function (this blog entry helped a great deal), but faced disappointment as I discovered that I can’t redirect standard output from the child process without bInheritHandles being set to true. I eventually altered the child process’ code to write its output to a file (which was actually better behavior for the app) and finally closed the door on this issue.

Playing around with WebKit a bit and stumbled across the headaches of working with a COM DLL. WebKit uses COM interfaces, so an application using a WebKit DLL must register the DLL. COM DLL registration is system-wide (i.e. all calls will be directed to a single DLL file), and this creates a problem if an application wants to use a DLL of a different version or a custom one (assuming renaming all the interfaces is not a viable option).

A simple solution to this seems to be to register and unregister the DLL programmatically. By doing so, any existing (or non-existent) registration is ignored and the exact DLL file the application needs is registered and used. The registration is still system-wide, but if other applications follow the same policy there doesn’t seem to be a problem. For applications that don’t follow this policy, it seems their calls will go to the last DLL registered.

(This is just from my own investigation and I really don’t know a whole lot about COM stuff. If anything mentioned is incorrect, please correct me.)

How to register and unregister COM DLLs can be found here. This is a bit convoluted. I actually just followed the steps listed in the second post of this thread on the MSDN forums to write my code.

To register:

To UnRegister:

1. ShellExecuteEx and lpFile

The code above is correct and it works, however if the lpFile member of the SHELLEXECUTEINFO struct is the full path to the executable file the ShellExecuteEx call fails. It seem you must specify the working directory for the child process (lpDirectory) and the filename only for the lpFile member.

2. SHFileOperation and pFrom

In the above code, SHFileOp.pFrom (killFolder variable) is a double-null-terminated string to a folder I want to delete (I want to also recursively delete everything inside the folder, that’s why I’m using SHFileOperation). However, aside from the peculiar double null-termination, the path can’t end with the path separator char or the SHFileOperation call fails.

3. WaitForSingleObject

After calling ShellExecuteEx or CreateProcess you can do the following to wait for the child process to terminate:

4. fclose

Don’t forget to close your file handles. I spent a few hours today trying to figure out why I couldn’t delete an exe, thinking it was an issue with the ShellExecuteEx function. Turns out I opened the file and forgot to close it.

btw, Process Explorer is very helpful and a great tool for programmers; it’s wonderful for detecting things like unclosed file handles.

I’ve been working on an installer/uninstaller for firesync 2. The installer is fairly straightforward, although it is quite a bit of work to set registry keys and make shortcuts, but overall it’s all fairly standard stuff. The one unique aspect of the firesync installer is that it is multi-part and actually consists of 2 executables. The first executable, the setup.exe file that will be distributed, is a native code binary + a zip archive. This program checks that the .NET Framework 2.0 is installed and extracts the zip archive to a folder in the user’s temporary directory. The zip archive contains the firesync executable and data files, but also contains another setup executable setupx.exe (a .NET executable). This executable is the real setup, as it shows/prompts the user for setup information (installation directory, whether or not to make a desktop icon, etc.), copies over the necessary files, makes program shortcuts, and writes registry entries so that firesync appears in the Add/Remove Programs list.

The reason for the multi-part installation is that it is much easier to design an interface using the WinForms designer in Visual C# compared to MFC, wxWidgets, etc. However, a .NET executable alone won’t suffice because users without the framework will just be given a cryptic error message (I think you get dialog about mscoree.dll not being found) and they’re not actually told that they need to install the framework.

(fyi, the native code executable is being done with wxDevC++)

—

Now making an uninstaller should be simple, your just deleting files and removing a couple of registry entries (in general, I avoid the registry and for firesync I’m only using it for the entry in Add/Remove Programs). Also, I think it’s rare that a user will uninstall the .NET Framework then try to uninstall firesync, so it’s pretty safe to make the uninstaller a .NET executable. However, there’s one big issue (and it’s a major pain in the ass) that pops up when it comes to making an uninstaller, how do you delete the uninstaller executable itself? I was lucky enough to stumble across this old Q&A from Microsoft Systems Journal. It’s a very inter… well to be honest it’s a very boring read (how interesting can deleting a file be?), but it’s important stuff. Ultimately, I went with the simplest method, and just used a batch file (although I haven’t played around with setting thread priorities as mentioned in the article; I’m not crazy about doing it and I’m not sure if it’s even worth it).

Finally, some important issues I ran across that was not mentioned in the article:

• The directory your attempting to remove may be in use by the OS or another application. Another loop in the batch file should take care of this.
• You can’t have the batch file in the directory your trying to delete (you’ll never be able to delete it as it’ll be in use by cmd.exe, you’ll have to manually kill cmd.exe to delete the folder and free the CPU). This is simple to solve, I just put the batch file in the user’s temp directory.

Seems like I’m talking too much about firesync in this blog, but anyway, another rare, but serious bug popped up in firesync on this special day. However, unlike the previous bug (see previous blog entry), this one doesn’t seem to be unique to firesync; it’s the daylight savings time bug (note: this article is informative, but some parts are not written too clearly and can be difficult to understand).

The bug popped up when I noticed firesync seemed to be thinking that all of my files were out-of-sync. I looked at the file modification times of a few files on the two PCs and noticed that modification, creation, and access times were off by 1 hour. The reason for this, as I discovered, is that the filesystem of one system was FAT32 while the other was NTFS. FAT32 time stamps are equal to the local time (so, inherently, DST information is contained within the time stamp). NTFS time stamps are based on UTC time and the local time is calculated as an offset from the UTC time. The problem (i.e. the daylight savings bug) results from the fact that Microsoft chose a somewhat odd way of handling conversion from UTC-to-local time. Instead of storing DST information as part of a time stamp (which would enable a correct conversion), NTFS systems will add 1 hour to the UTC time (during the UTC-to-local time conversion process) if daylight savings is currently in effect!

So, the solution seems to be to do all time stamp comparisons based on UTC time. Ahh, more work for firesync 2.