So, do you want to try a different cooler to solve your problems? Yes, that does not address the who's at fault issue with the stock cooler, but as well as solving your stability problem, it would demonstrate a different cooler does not cause the problem. Dynatron and Thermaltake have several coolers similar to the Intel design, but with backplate mounting. Coolers that fit socket 775 CPUs are very common, and inexpensive ones can be found. One example here:
Personally I feel that mounting the stock cooler with the clips only half secured (ie the gray, transulcent plastic pins are pushed through and the black push-pins are pushed down to a point where they are firmly positioned, but do not go all the way through requiring that one force them through just to get them to click into their secured position) and having the system run stable where it's unable to operate in a reliable, stable fashion when the cooler is properly secured as per the Intel instructions (and design requirement for securing the clips), is proof enough that the coolers are to blame.
Furthermore, mounting non-supplied coolers would void the CPU's warranty, according to the warranty booklet received with each of these processors; removing the TIM supplied on the heatsinks does the same, from what I understand. I know that not all people are so pedantic about such things that they would immediately disqualify a warranty claim due to such things, but I'd rather not take my chances.
*edit* I also don't believe it should be my responsbility (financially) to have to replace a component of a boxed processor package that is supposed to function properly, that clearly doesn't.
I understand all that, just an idea. Continuing your line of thought regarding honoring warranty conditions, does not engaging the push pins as you have described remain outside of those conditions? Yes, engaging them causes your problems in your opinion (and reality) but mention unengaged push pins to an Intel rep and tell me what you think their reaction will be?
Failure to install the product in the intended fashion could be grounds for a voiding of the warranty assuming that it is the likely cause for a failure of the product to function as a whole; as I'm taking reasonable measures to ensure that the processor itself remains safe during my testing, I sincerely doubt that the processor will fail, and as such, the processor's warranty should remain intact on the grounds of it having suffered no damage. The heatsink/fan assembly remains undamaged during this testing, also.
What would be cause for argument on the part of the person or people handling the warranty return would be the removal of the compound present on the processor's shield, or the mixing in of some other compound into that, by placing a different cooler on the processor; they don't need to care that I "know what I'm doing" so much as that they can turn me around and tell me they can't help me because I 'broke' something of their PIB package on my own.