Archive for March, 2009

.Net FtpWebRequest and 503 (bad sequence of commands) errors

I’ve been working on some FTP code in C# with the .Net framework’s FTP classes and frustrated that I was occasionally getting 503 (bad sequence of commands) errors. Turns out these errors will pop up if the FtpWebRequest.KeepAlive property is set to true, which it is by default. Unfortunately, this doesn’t really address why this is occurring in the first place. I’m doing a lot of multithreaded stuff, so perhaps that’s the culprit (i.e. 2+ threads using the same connection, layering their calls on top of each other). However, I’ve found some forum posts (such as this one) indicating others are having the same issue doing seemingly serial, non-asynchronous stuff.

DPC latency issues

Last month the motherboard in my main desktop died. I replaced it with this board (GA-MA78G-DS3HP) from Gigabyte (I had some reservations, as the previous board was from Gigabyte as well, but the price and feature-set won me over). The installation went smoothly and I was surprised that Windows booted up (probably b/c of similarities due to both boards coming from the same manufacturer; fyi, I later discovered there is a way to install a new motherboard w/o a windows reinstall). I was pretty happy until I started hearing seemingly random stuttering in the audio. It was very short, but noticeable, especially in games.

I spend hours trying to figure this out. I was thinking it was the onboard sound, but it was Realtek audio, which I’ve never had issues with, and this new board has one of their best chipsets. I finally stumbled across some forum posts mentioning DPC latency and the DPC Latency Checker. I had no clue what DPC latency was, but it’s summed up nicely on DPC Latency Checker page,

If any kernel-mode device driver in your Windows system is implemented improperly and causes excessive latencies of Deferred Procedure Calls (DPCs) then probably drop-outs will occur when you use real-time audio or video streaming applications.

This is a wonderful utility, and, sure enough, after a few tests I noticed very long red bars, indicating I indeed had a DPC latency issue. Now, there are issues with older Gigabyte boards and DPC latency issues, discussed in this Anandtech thread, but the board I purchased was newer that those afflicted. What was also curious was how the issue was occurring. I was not getting steady red bars, but spikes, and after many hours of pulling my hair out, I finally realized that that the spikes were coming from the keyboard and they were fairly random in occurrence (although more prevalent within Direct3d or OpenGL games and apps).

Trying to think of solutions and trying to avoid a BIOS update, I tried a repair install of Windows; perhaps something went wrong b/c I was using a new motherboard w/o a clean installation. The repair install had no effect. I thought about doing a fresh install, but the amount of time for reinstallation of programs and drivers put me off the idea – this machine is my main development rig and I need it at 100% as much as possible.

It then occurred to me that maybe this was related to input in general, so why only the keyboard effected? I have a mouse and gamepad that didn’t cause any DPC latency spikes. I realized it was a PS/2 keyboard and all other input devices were on USB ports. Luckily I had a USB keyboard on my Mac that I could test with and, like magic, the latency spikes disappeared.

My long term solution? I don’t know. I’ll probably just get another USB keyboard. There’s a BIOS update available which I’ll probably try at a later date – BIOS updates scare me. It’s possible a fresh install of Windows may solve this as well, but I’m doubtful. I was surprised that the board got great reviews on Newegg and no one there or anywhere else on the web seems to have encountered similar issues with this board.

Let your app talk about itself

Fragment Sync has an automated update system called autobot. The system is used both for development and public releases (I should mentioned that automated updates to development copies of a networking app is a beautifully amazing thing). Updates are published to the autobot server using a simple publisher app, which puts the update package on the server and calls a server-side script to enter the patch details into a database. One annoyance with the publisher is that I would have to enter details (version, whether or not update is public, etc.) manually. This isn’t too bad, but on more than one occasion I’ve screwed things up by accidentally entering the wrong version or packaging an old executable.

I finally decided that the best way forward was to have the publisher automatically query the executable for information. You can embed this sort of information via. the application manifest, but using the manifest really didn’t cross my mind and I’m not a big fan of manifest files in general. In any case, what I did was have the executable take arguments and output the queried data to the console. This is a bit trickier than it sounds because, for whatever reason, a GUI app can’t send messages to a console without some hackery with the AttachConsole() Win32 function. (I’m dealing with C# and WinForms, but native Win32 GUI apps don’t have a default console either, so this isn’t a WinForms specific thing). The publisher redirected the standard output stream and read in the data that was pushed out.

This seems to have worked out nicely and has pushed me to think about other information an app can output about itself, lending to better communication between related apps.

Handle leaks and CreateProcess

I finally nailed down an annoying little bug tonight. In a certain app, I’ve been calling CreateProcess() to periodically spawn a process, do some work, and shut down. Unfortunately after running several hours, the app would fail with an exception saying: insufficient quota to complete the requested service. After a fair bit of monitoring the app’s activity, I notice that the handle count of the main process, slowly and surely, kept going up. After a bit of trial-and-error, disabling modules systematically, I finally noticed that this was occurring when I spawned off the child process I mentioned.

Reading the MSDN docs for CreateProcess(), I finally got to the root of the issue; the handles returned in the PROCESS_INFORMATION struct must be closed via. CloseHandle() or the handles are kept open, even though the child process has terminated.

If the function succeeds, be sure to call the CloseHandle function to close the hProcess and hThread handles when you are finished with them. Otherwise, when the child process exits, the system cannot clean up the process structures for the child process because the parent process still has open handles to the child process.

This oversight was probably due to the fact that I was working in C# (I was P/Invoking this stuff) and lulled into a false sense of safety, thinking the garbage collector would take care of stuff like this, but from working with files, various streams, sockets, etc. I realized that C# doesn’t really close handles automatically. Now that has me thinking, why not? Wouldn’t handle management be very similar to memory management?

The fragment sync blog

The Fragment Sync blog is up. Up first is a series of posts about the history of Fragment Sync development, from its original incarnation as Firesync to what it is now. It’ll also give me a chance to present some of the design and technical decisions that went into making Fragment Sync and explain what sets it apart from many of the existing sync solutions out there.