In part 3 we covered troubleshooting common Provisioning Console issues. In part 4 we now focus on those components operating in the background during provisioning. With a functioning install and console, and when the issue appears to be server-related (In part 1 we covered troubleshooting the locale AMT system) now any issues seen must be evaluated on the server side. This article covers this process in a Problem - Cause - Solution format.
The server components constitute a lot of ‘background' processes that support what is only seen as Altiris Console points. Much of what goes on in the background is invisible to the user save as a change in status. If setup correctly, machines simply provision. It's when they do not provision that a user should understand the server components so that proper troubleshooting can be accomplished. Note that this covers the symptoms of server-component problems. Some of the symptoms do overlap client-side issues, but in this process we are assuming we've confirmed that the client systems is functioning as expected. If you are unsure, see Part 1 of this article series.
The following symptoms are seen on the Server. Please note that some of the symptoms may appear to be both client and server related making it difficult to know where the issue lies. Use Part 1 in conjunction with this article if necessary in troubleshooting these issues.
No update to Intel AMT Systems Node - At times this node can abruptly appear stagnant with no new systems coming in and no provisioning taking place
No Systems Appearing - The Intel AMT Systems node may stay blank even after connecting systems in Setup Mode onto the Network.
FQDN Not Acquired - Once the SCS receives a hello message, it needs to acquire the FQDN, and if this fails the machine will remain in an unprovisioned state
No systems Provisioning - This can occur where systems show up in the system, but none of them provision
Properties Script Failed - This is a common error to be covered separately, though many of the above symptoms end up throwing this particular error
In addition to the symptoms, the following tools were used to troubleshoot the issues to find out which particular issue afflicted the Server:
OOB Trace Loggging
See Part 1 in this article series on how to use these. These will be referenced in the below items.
No update to Intel AMT Systems Node
The typical symptom is an abrupt stop to updates on this node. For example if you have a number of provisioned systems, with systems added as systems are brought up on the network, and abruptly they stop updating or being added, this is indicative of this issue.
AMT Logs - No updates to this log occur.
AMTConfig Service - The AMTConfig service has stopped, crashed, or is in a hung state. This isn't common in version 3.0 of SCS or higher.
Check that the AMTConfig Service is running.
1. Go to Services Manager under Administrative Tools.
2. Check the Service named AMTConfig to make sure it is running.
3. If the service is not running, start it. If the service is running, try restarting it just in case it's in an hung state.
4. Once the service is up and running again (if this is the issue) provisioning should start occurring.
No Systems Appearing
The symptom is that no machines appear in the Intel AMT Systems list when the page is refreshed over a period of time when new systems are expected. The page ties directly into the IntelAMT database to populate the systems, so if the list isn't updating on the page, the list is also not updating in the database.
AMT Logs - I. No entries found
II. No entries found
III. Invalid PID Map error
Wireshark - II. On the client the "Hello" packet is sent, but on the server it never arrives.
The causes vary. See below for known causes for this issue:
I. AMTConfig Service - The AMTConfig service has stopped, crashed, or is in a hung state. This isn't common in version 3.0 of SCS or higher.
II. "Hello" packets - The routing of "hello" packets is not configured correctly, so clients can't reach the Provision Server.
III. PID rejected - The PID provided in the "Hello" packet is not contained as a valid security key in the IntelAMT database. This is only seen in the AMT Log found in the Provisioning Console under Logs, selecting the ‘Log' icon.
See the steps to follow for the above causes.
I. AMTConfig Service
1. See the resolution to the section No update to Intel AMT Systems Node.
II. "Hello" Packets
1. In the Provisioning console go to the DNS Configuration node. Does the ‘Test' button allow Provisionserver to resolve back to the IP of the Notification Server?
2. If yes, go to the segment of the network the client is on and try to ping the name ‘Provisionserver'. Does the IP resolve?
3. If answer to either question above is NO, a CNAME record needs to be created on each DNS Server to route to the IP address of the Notification Server.
III. PID rejected
1. In the Provisioning Console go to the Security Keys node under the Configuration Service Settings. The list of unused PID and PPS combinations are listed.
2. In the IntelAMT database, within the csti_pid_map table all used and unused security keys are listed. The ones with a value ‘True' in the ‘Used' column will not show up in the console.
3. Either import the keys if the OEM placed the AMT systems in TLS-PSK Setup Mode through the import button in the Security Keys page, or manually enter the PID PPS.
FQDN Not Acquired
One or more Intel AMT Systems are registering in Intel SCS, but they never show an FQDN and never move out of the ‘Unprovisioned' status. In the AMT Log often these systems show the error ‘Properties Script Failed' (note that the cause of this error can be many, and this issue is but one of them).
NOTE! If no system is provisioning the issue may not be FQDN related. See No Systems Provisioning in this article for more information.
AMT Logs - Properties Script Failed messages
OOB Trace - Unable to locate FQDN (Fully Qualified Domain Name) entries
Intel SCS calls the Out of Band Provisioning or Properties script oobprov.exe to do a number of things. The first thing it does is obtain an FQDN for the machine needing provisioning. If it fails to obtain an FQDN Provisioning will fail and the computer will remain in an unprovisioned state until oobprov.exe can successfully locate the FQDN.
To find the FQDN, oobprov.exe runs through a number of checks. The suggested method is to have the Altiris Agent installed and have run the OOB Discovery Task (located in the Altiris Console under View > Solutions > Out of Band Management > Configuration > Out of Band Discovery > Out of Band Discovery). This populates the Altiris database so it has both an FQDN in the AeX AC Location data class and the UUID in the Inv_OOB_Capability data class. If this data is not available, another option is to check DNS resolution as a method. In the Altiris Console look under the Resource Synchronization node, within the Intel AMT Systems folder. As shown below, this option enables oobprov.exe to use DNS IP resolution as a method.
NOTE the warning found directly below the checkbox: Warning! Using DNS for IP to FQDN resolution might lead to incorrect profile mapping. Make sure your DHCP server is configured correctly to give update the DNS server for dynamic addresses.
No systems Provisioning
Systems are added regularly to the Intel AMT Systems node, but they never provision. This includes never getting an FQDN (see the above section for more information), though the cause may not be the inability of oobprove.exe to obtain the FQDN.
AMT Logs - Provisioning Script Failed messages
OOB Trace - No references to oobprov.exe
If not an FQDN mapping issue, this issue stems from a timeout value in the IntelAMT database being set to 0. In the IntelAMT database, in the table csti_configuration, under the column Props_script_timeout if the value is 0 IntelSCS will timeout before it even has a chance to call oobprov.exe.
Normally only one row exists in this table. The following SQL query will properly update this value to the default level. The default is 180 and should be set.
SET props_script_timeout = 180
WHERE use_props_script = 'True'
Execute the script within SQL Query Analyzer or SQL Enterprise Studio to update the value.
Properties Script Failed
This message can mean a number of things, including the symptoms described in the preceding two section. This message can continually appear into the AMT logs as provisioning is attempted over and over.
The causes of this issue vary. The basic explanation is that when oobprov.exe is called, if it returns anything other than success, the resulting error message in the AMT logs is ‘Properties Script Failed'.
See the above two sections for the symptoms No Systems Provisioning and FQDN Not Acquired, but for additional information see the following article:
This concludes the troubleshooting section for the Provisioning process. For the most common issues, the resolutions and steps presented in the first four parts of this series will resolve them. I also hope the methodology here helps explain how the background processes are working. In the next parts of this series we'll cover troubleshooting issues with the management components after systems have been successfully provisioned.