Intel HD Graphics Blue Screen

[idata]

Hello

I've got an issue with 2 (out of a possible 5 in use so far) - IEI NANO-HM551 boards which use the Intel HD Graphics. I'm running Windows Embedded Standard 2009.

The IEI boards have a Core i5 installed and 2GB of RAM. I have some systems using the LVDS port (48bit - 1920x1080 - 3 are setup like this, 2 are ok whilst the 3rd is not). I also have 2 which are running LVDS 48bit - 1920x1080 and simultaneously running the HDMI port - 1920x1020 with the driver setup to run the desktop in Extended mode. One of these is also experiencing the problem.

The symptom is: our WPF .NET 4.0 application suddenly stops updating the UI.

It's been difficult to trace the problem because it happens at random, usually no less than 19+ hours after the OS and application have been running. In other words, to try and reproduce the problem - either it doesn't happen or will take at least 24 hours. The system running with 2x displays it can happen every few hours or every 24+ hours, unfortunately I don't have the luxury to access this system and use it as a test bed given it's deployed to a remote location. All I can do is remotely reboot it to get the application running again.

I've managed to get the problem to occur once in the lab with one of the systems that is connected via LVDS to a single monitor. I tried to poke around so I could hopefully discover what was going on. At this point I tried to see if I could interact with our WPF application and discovered it wasn't responding to input. In-fact, I couldn't open other applications either, except I could get into Task Manager and at that point terminate our application. Upon terminating our application, this resulted in the system blue-screening and rebooting. It happened so quickly, I couldn't see the exception code.

I was however luckily enough the blue-screen recorded a dump file which I've since analysed. Here's the output:

Microsoft (R) Windows Debugger Version 6.12.0002.633 X86Copyright (c) Microsoft Corporation. All rights reserved.Loading Dump File [c:\Windows\DUMP180c.tmp]Mini Kernel Dump File: Only registers and stack trace are available

WARNING: Inaccessible path: 'c:\windows\i386'

Symbol search path is: srv*c:\symbols*http://msdl.microsoft.com/download/symbols http://msdl.microsoft.com/download/symbolsExecutable search path is: c:\windows\i386Windows XP Kernel Version 2600 (Service Pack 3) MP (4 procs) Free x86 compatibleProduct: WinNt, suite: EmbeddedNTBuilt by: 2600.xpsp.080413-2111Machine Name:Kernel base = 0x804d7000 PsLoadedModuleList = 0x805634c0Debug session time: Sun Dec 4 12:16:31.359 2011 (UTC + 11:00)System Uptime: 0 days 19:22:29.862Loading Kernel Symbols....................................................................................Loading User SymbolsLoading unloaded module list...............******************************************************************************** ** Bugcheck Analysis ** ********************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck 100000EA, {889c25e8, 88c4db68, f78cecbc, 1}

Unable to load image igxpdx32.DLL, Win32 error 0n2

*** WARNING: Unable to verify timestamp for igxpdx32.DLL*** ERROR: Module load completed but symbols could not be loaded for igxpdx32.DLLProbably caused by : igxpdx32.DLL ( igxpdx32+8785 )

Followup: MachineOwner

---------

2: kd> !analyze -v

******************************************************************************** ** Bugcheck Analysis ** ********************************************************************************

THREAD_STUCK_IN_DEVICE_DRIVER_M (100000ea)

The device driver is spinning in an infinite loop, most likely waiting forhardware to become idle. This usually indicates problem with the hardwareitself or with the device driver programming the hardware incorrectly.If the kernel debugger is connected and running when watchdog detects atimeout condition then DbgBreakPoint() will be called instead of KeBugCheckEx()and detailed message including bugcheck arguments will be printed to thedebugger. This way we can identify an offending thread, set breakpoints in it,and hit go to return to the spinning code to debug it further. BecauseKeBugCheckEx() is not called the .bugcheck directive will not return bugcheckinformation in this case. The arguments are already printed out to the kerneldebugger. You can also retrieve them from a global variable via"dd watchdog!g_WdBugCheckData l5" (use dq on NT64).On MP machines it is possible to hit a timeout when the spinning thread isinterrupted by hardware interrupt and ISR or DPC routine is running at the timeof the bugcheck (this is because the timeout's work item can be delivered andhandled on the second CPU and the same time). If this is the case you will haveto look deeper at the offending thread's stack (e.g. using dds) to determinespinning code which caused the timeout to occur.Arguments:Arg1: 889c25e8, Pointer to a stuck thread object. Do .thread then kb on it to find the hung location.Arg2: 88c4db68, Pointer to a DEFERRED_WATCHDOG object.Arg3: f78cecbc, Pointer to offending driver name.Arg4: 00000001, Number of times "intercepted" bugcheck 0xEA was hit (see notes).

Debugging Details:

------------------FAULTING_THREAD: 889c25e8

DEFAULT_BUCKET_ID: GRAPHICS_DRIVER_FAULT

BUGCHECK_STR: 0xEA

PROCESS_NAME: WPFNet4.

LAST_CONTROL_TRANSFER: from 804f0a8c to 804f0f08

STACK_TEXT:

a9d19444 804f0a8c 89b13000 ff000011 00000253 nt!RtlFindClearBits+0x230a9d1945c 8054e1b5 89b13000 00000011 00000253 nt!RtlFindClearBitsAndSet+0x14a9d194a8 80551858 00000001 00010018 00000000 nt!MiAllocatePoolPages+0x4c0a9d19510 bf802b0c 00000001 00010018 43544e49 nt!ExAllocatePoolWithTag+0x109a9d19530 bf85b6c4 00010018 43544e49...

[idata]

Hello Ben,

Thank you for the detailed explaination that you have provided about the issue. Regrettably, we only provide technical support for Desktop Versions of Windows, as embedded versions are not supported in our department.

We do hope that you can obtain assistance on the community from other users with similar configurations.

[idata]

I haven't 100% confirmed this yet, but here's what I have found.

The WPF Application has a number of animations.

In order to get those animations to appear smooth, I was utilising an Option of 'BitmapCache' - which causes the Graphics Card to cache the visuals in GPU memory.

Turns out this Cache isn't being flushed and over time the GPU is running out of memory.

Interestingly, the HD Graphics allows up to 1024MB but as soon as it reaches about 950MB - 960MB, it's all over, the OS needs to be restarted.

On my development machine, it uses an nVidia Graphics set and it does flush the cache (albeit the screen blinks similar to when you change resolution).

I've turned off almost all BitmapCaching and the problem has resolved (testing for past 6 hours and counting - so far so good - only 250MB of GPU memory consumed).

Unfortunatly the animations are choppy, but the app is not consuming up all the GPU memory which means it shouldn't crash.

I owe a huge amount of thanks to Pete Brown from Microsoft for his time in helping me find the problem as well as giving me some pointers in my application.

Now it would be nice if Intel could fix the driver so it flushes the GPU cache.

It should be noted - the behaviour of the graphics driver is irrelivant to the OS installed, the issue is unchanged on a full copy of Windows XP Professional (which I ended up needing to install so I could profile the application on the Intel hardware in Visual Studio because VS wouldn't install on Windows Embedded).

Maybe for the next release of the Intel HD Graphics Driver you could get the GPU Cache to flush?