Whether you are planning to implement a Vendor TLS Certificate in the future, or you are having trouble applying a certificate you’ve already obtained, this article walks through the best practices.  The details include all the steps to properly install the right items and resolve issues we’ve encountered up to this point.  This article applies to Out of Band Management Solution 7.0, included with Client Management Suite 7.0.  Since certificates introduce tight encryption security, if the right items and steps are not in place or followed, it can break the ability of AMT systems to provision with Remote Configuration.

Introduction

Why is Configuring a vPro capable system important?  Without setup and configuration, the functionality provided by vPro is not accessible within your Symantec Client Management Suite environment.  Out of Band Management Solution allows setup and configuration to occur automatically using Remote Configuration.

Using Remote Configuration to setup and configure your Intel AMT vPro capable computers takes the work out of the process, after some initial setup.  AMT systems that come preconfigured with versions 2.2, 2.6, 3.0+, 4.0+, and 5.0+ will automatically use Remote Configuration to setup and configure with a valid Provisioning Server.  Out of Band Management provides such a server.  The hashes from vendors (AMT 3.0 includes Verisign, GoDaddy, Comodo) are already configured in the firmware, and upon connection to power and the network, will begin to send out requests for configuration.  Thus in this way the managed vPro systems are already prepared to be configured without any intervention by the IT staff.

Usually the issues we see with the Remote Configuration process originate on the server-side process of adding a certificate from the before mentioned vendors.  Obtaining and installing a vendor TLS Remote Configuration certificate needs to be done the correct way so that authentication can succeed.  Once in place, provisioning will roll forward without any further intervention as long as the certificate remains valid.  This article focuses on applying the server-side certificate so that setup and configuration can move forward automatically.

Obtaining a Remote Configuration Certificate

This subject has been covered previously.  I wanted to lightly touch upon this as there is a vital step that should be taken so that if anything goes wrong we can correct it.  First, the following article covers how to properly obtain a certificate:

• http://juice.altiris.com/article/4496/obtaining-and-applying-a-verisign-remote-configuration-certificate

Note that part of obtaining a Remote Configuration is submitting the request from the Server you plan to install the certificate onto.  This process creates the private key for the server-side certificate, and this item will not be available until partway through the application of the crt (or cer) file obtained from the vendor.  The specific step that provides the full key, both private and public, is when the certificate is exported into a PFX format after the initial import, checking the option to export the private key will give you a complete backup of the full certificate in case it is needed in the future.  If something happens, or if the application doesn’t go right, we’ll need both, so it’s essential to export this as soon as possible.

During the steps to install the certificate emphasis will be given on the step where the export should take place.

Certificate Authority (CA)

In order to use Remote Configuration with Out of Band Management the Microsoft Certificate Authority services must be installed on the Notification Server or the OOB Site Server.  Use the following steps to install if it is not installed:

1. Go to Start > Administrative Tools > and click on Add or Remove Programs.
2. In the left-side button bar click the button Add/Remove Windows Components.
3. Check the option labeled Certificate Services.  See this screenshot for details:
   
4. You’ll receive the pop-up:
   After installation Certificate Services, the machine name and domain membership may not be changed due to the binding of the machine name to CA information stored in the Active Directory.  Changing the machine name or domain membership would invalidate the certificates issues from the CA.  Please ensure the proper machine name and domain membership are configured before installing Certificate Services. Do you want to continue?
5. Click Yes to continue once your system has the intended identity.  Click Next.
6. Choose what type of CA to create.  If you are not installing a hierarchy of CAs you can leave the stand-alone root CA option selected.  Click Next.
7. Input the name the CA will be known by.  This must match what is in the hierarchy or by what the Remote Configuration certificate name will be known by.
8. The Distinguished Name is generated automatically in an AD Environment and will be the suffix of the system.
9. Click through the rest of the options, noting where the services data files are stored.
10. You will be prompted to restart IIS.  This is required during the installation.
11. Click Finish to complete the installation.
12. Done!  The NS or Site Server is now prepared to handle certificates in the Remote Configuration process.

Installing the Certificate

The recommended application for a Remote Configuration certificate is to let the certificate dictate where to be installed.  However this process has sometimes resulted with the certificate installed to an incorrect place.  When this occurred we’ve had headaches trying to clean up the system to properly install the certificate.  Why this occurs is unclear.  For reference I’m including the process of adding a certificate automatically here:

1. Save the acquired cer or crt file from the vendor onto the Notification Server or the Site Server for Out of Band Management.
2. Right-click on the file and choose Install Certificate.
3. Click next on the Welcome screen.
4. Leave the radial option on ‘Automatically select the certificate store based on the type of certificate’ and click Next.
5. Click Finish to complete the installation.  You’ll receive a confirmation pop-up that the certificate installed successfully.

While I won’t advise against using this method, the below steps uses the manual installation method to ensure the certificate is installed to the correct place.

I’ve condensed the steps required into the following list.  This process works for all vendors once you’ve obtained a certificate.  Note that these steps are provided to consolidate both recommended steps and documentation into one whole.

1. Go to Start > Run > type mmc > and click OK.
2. In the resulting console click under File and choose Add/Remove Snap-ins…
3. Near the bottom of the resulting window click the Add button.
4. From the list that appears select Certificates and then click the Add button.
5. Leave the radial button selected on ‘My user account’ and click Finish.
6. From the same list select Certificates again and click the Add button.
7. From the resulting window change the radial select to ‘Computer account’ and click Next.
8. Leave the selection at ‘Local computer: (the computer this console is running on) and click Finish.
9. Click the Close button in the window offering you the list of available snap-ins.
10. At the original add/remove snap-in screen verify that you have two entries:
11. 1. Certificates – Current User
    2. Certificates (Local Computer)
12. Click OK.
13. Expand both trees in the left-hand pane within the console.  You should see the full certificate stores as shown in this screenshot:
    
14. Right-click on the Personal folder under the Current User certificate store and highlight ‘All Tasks’ and click on ‘Import’ in the pop-out menu.
15. Click Next on the Welcome page of the Certificate Import Wizard and click the Browse button.
16. Browse to the cer or crt file provided by the vendor, highlight it, and click Open.
17. Click Next, and leave the radial option on ‘Place all certificates in the following store’, which should be set to ‘Personal’.  Click Next.
18. Under the Completing section of the wizard, Click Finish.  You should receive a pop-up indicating the certificate was successfully installed.
19. NOTE!  This is the vital step mentioned previously in the article.  We will now export the certificate with both public and private keys, which will give us the full set and allow us to remove and reapply if necessary.  In the MMC select the newly imported certificate > right-click > and choose All Tasks > Export…
20. Click Next on the Welcome screen.  In the resulting list you should have an active option for ‘Personal Information Exchange – PKCS #12 (.PFX)’.  If this option is not available (grayed out as shown in this screenshot), there is a problem with the certificate and the private key is not accessible:
    
    If this occurs please note the following items:
21. 1. The application of the public key, or cer/crt file, must be done on the server where the key was requested. 
    2. If this is not your Provisioning Server you’ll need to contact the Vendor of the certificate to resolve the discrepancy.
    3. If you did request this certificate from the server you are operating on, you’ll also need to contact the vendor to explain that the private key is not found when exporting the certificate after initial application.
22. Follow the wizard, and ensure you select the option ‘Yes, export the private key’.  When saving the file, it will prompt you to set a password to protect the private key (this is recommended for security reasons).  The export should leave you a PFX file.  Keep this in a safe place, preferably in line with your company’s encryption certificate backup policy.
23. Next we need to import the full key into the Computer store.  Start back in the MMC > under the Local Computer certificate store > right-click on the Personal folder > select All Tasks > Import…
24. Click Next on the Welcome screen and click the Browse button on the subsequent screen.
25. Browse to the newly exported PFX file.  Note that you will need to change the ‘Files of type’ to include the PFX format.  Click Next.
26. The Password screen prompts for the password you set when you exported the key in step #20, as shown in the following screenshot.  Enter the password and click Next.
    
27. Choose or leave the select to ‘Place all certificates in the following store’.  The value should be Personal.  Click Next.
28. Click Finish on the end details page to complete the import.
29. Done!

NOTE: In Out of Band Management 6.x, with Intel SCS 3.x or earlier, a separate utility was required to load certificates into Intel SCS so the Provision Server was aware of them.  This is no longer required as Intel SCS 5.x possesses intelligence to automatically acquire all installed Intel vPro Remote Configuration encryption certificates.

Reinstalling the Certificate

If you need to reinstall the certificate and you have a PFX file, you can do so by opening both certificate stores (User and Local Computer) as outlined in the previous steps.  Browse through the certificate stores and delete any instance of the vendor certificate.   This will remove any associations and allow a clean application of the certificate to occur.  Look for the following:

• The name matching the name of the cer or crt file obtained from the vendor
• The vendor’s certificate (the entry will contain the vendor name).

NOTE: Be careful when removing vendor certificates as they may not be part of the Remote Configuration.  The best example is Verisign, which may have many entries.  If unsure, leave the certificate in place, or export it before deleting it so you can restore it if necessary.

Enabling Remote Configuration

To ensure that Out of Band Management is setup to use Remote Configuration as a valid setup and configuration method, follow these steps:

1. In the Symantec Management Console browse under Home > Remote Management > and click on Out of Band Management.
2. In the left-hand tree browse under Configuration > Configuration Service Settings > and select General.
3. In the resulting page ensure that the option labeled Allow Remote Configuration is checked.  If it is not, check it.  See this screenshot for an example:
   
4. If you needed to check the option, be sure to click Save Changes to register the change.

That should do it for the certificates.  You’ve now completed the steps required to install and enable Remote Configuration in the Out of Band Management Environment.  However you are not done yet!  Certain infrastructure components are required to make this process seamless.  Proceed to the next section for details.

Other Setup Requirements

The following items will be used to automate the setup and configuration process.  Remote Configuration will use these to locate and communicate with the Provisioning Server (Out of Band Management).

ProvisionServer

Each zone within DNS should have a ProvisionServer entry to ensure that Remote Configuration requests are properly routed to the Server.  This will also help properly resolve names during the authentication process.  Use the following steps to add ProvisionServer to DNS:

1. Go to Start > Run > type mmc > and click OK.
2. In the resulting console click under File and choose Add/Remove Snap-ins…
3. Near the bottom of the resulting window click the Add button.
4. From the list that appears select DNS and click Add and click Close.
5. Click OK in the next Window.
6. Browse in the tree to the Forward Lookup Zones.
7. Right-click the entry for the Notification Server computer and choose New Alias.
8. Type ProvisionServer as the Alias name, in this manner:
   ProvisionServer
9. Done! 

Though simple, this is the key to directing the automatic Remote Configuration hello packets from enabled vPro systems to the Notification Server or Site Server.  Without this step no setup and configuration of vPro systems will occur.

To test, log onto a system on the subnet you’re trying to conduct Remote Configuration from.  Run a command prompt and use the following command:

• ping ProvisionServer

We should see the responding IP Address by the IP Address of the Notification Server, or, if you’ve set it up this way, the Intel SCS Server conducting provisioning.  Another test you can try is to run the following command:

• nslookup ProvisionServer

We should get the data on the Notification Server’s Fully Qualified Domain Name (FQDN).

DNS Zones

In a multiple domain structure this is especially important, but all environments need to have the right data in DNS to properly pass and authenticate in a TLS environment.  The DNS Primary Zone should be set to the Domain path contained within the certificate.  For example, if the certificate name is MyNSServer_My1Domain_local, the DNS Primary Zone should be My1Domain.local.  Without this, authentication can fail as the FQDN is used during authentication, and if the name being transmitted across the wire doesn’t match what’s in the certificate, authentication will fail.  Here is another example:

• Certificate: MyNSServer_My1Domain_local.crt
• DNS Primary lookup Zone: My1Domain.local

DHCP Option

Another Network related requirement may be DHCP Option 15.  While I’m not sure why this has proven to be required in some environments and not others, creating this option has resolved failed authentication issues within Remote Configuration.

In DNS, create an entry for Option 15, with the value of the domain path.  This will often be the same as what is located in the DNS Primary Zone.  The following details are an example:

• Certificate: MyNSServer_My1Domain_local.crt
• DNS Primary lookup Zone: My1Domain.local
• DHCP Option 15: My1Domain.local

Conclusion

Following the above procedure should allow remote configuration to occur without problems.  Once in place, the configuration will move forward with automatic setup and configuration for all vPro enabled systems that support Remote Configuration.

For those who have Provisioned Intel AMT Systems, you may wonder what takes place in the background. This article is for you! The process has often been covered at a high level, but here the technical details are provided. Hopefully this helps you understand the inner workings, and provide you information when troubleshooting Provisioning issues. And for those of you who are technically minded, it's also neat to know! This information was compiled working on issues and running through provisioning processes from Symantec Support.

Introduction

Often the Provisioning process for Intel vPro systems has been described as complex. This comes from the fact that the Provisioning process was designed with high security in mind. Since the initial release we have improved success rates by working with Intel to make the process more user friendly without compromising the high level of security. To this end this document will explain the process of Provisioning from a technical level, providing an unfiltered view of the process, also without compromising its security.

Provisioning Flow

The following process assumes that Altiris Out of Band Management and Intel SCS are install, configured, and ready to go. This process follows the flow of Provisioning and what data points, technologies, and methods are used. The level of details is meant to be a resource when working with Provisioning or troubleshooting Provisioning issues, so not all details are available for this process. Note the following points before moving through the process:

• The console items in the Altiris Console under View > Solutions > Out of Band Management > Provisioning are not tied to the Altiris database like most of the rest of the Altiris Console. They connect through a virtual Website (AMTSCS under the Default Website of the SCS Server) to the IntelAMT database.

• Data from two databases (IntelAMT and Altiris) are used during the Provisioning process.

The following articles can assist if you need information on these:

• http://juice.altiris.com/article/1314/altiris-and-intel-reg-vpro-copy-technology-evaluators-guide

• Administrator's and Reference Guides: http://www.altiris.com/support/documentation look under Out of Band Management

1. The server is loaded with a security key or certificate. See the following two items for how these keys are loaded:

1. For a PID PPS, either keys are randomly generated or imported into the IntelAMT database. Specifically they reside in the table csti_pid_map. Once created/imported, they are available for verifying authentication from an incoming provisioning request from AMT.

2. For TLS-PKI (certificate-based Remote Configuration) a certificate is loaded onto the server. See this article for details: http://juice.altiris.com/article/4496/obtaining-and-applying-a-verisign-remote-configuration-certificate.

2. The clients need the matching keys loaded onto them. This is done differently depending on the type:

1. For PID PPS the keys are set by one of the following methods: the OEM sets it, it's entered manually into the Intel ME, or inputted via a one-touch USB flash drive. The PID and PPS are written into the firmware to be used as the authentication credentials when it looks for a provisioning server.

2. For Remote Configuration (TLS-PKI) at the factory predefined hashes are burned into the firmware for the following certificate vendors (more to come in subsequent versions of AMT). This means AMT already has authentication keys to begin the provisioning process direct from the factory.

• VeriSign

• Komodo

• GoDaddy

1. The client machine, once it has it's keys and has been connected to the network and power, uses one of two methods to find the Provisioning Server:

1. The IP address of the server can be manually put into the Intel ME, including what port the SCS listener is configured for (default 9971). When this is done, the AMT client will transmit its Hello message directly to the IP Address and port.

2. The client will transmit its message on port 9971 to the name of ‘ProvisionServer'. If Out of Band Management, Intel SCS, and DNS have been properly setup DNS will route the packet to the Notification Server.

2. The Notification Server is set to listen for AMT Provisioning traffic on port 9971, but can be configured to use a different port if so desired in the Altiris Console under View > Solutions > Out of Band Management > Configuration > Provisioning > Configuration Service Settings > General. The top options labeled: ‘Listen port:".|
   !ListenPort.jpg!

3. When SCS, via the service AMTConfig (process AMTConfigWinService.exe) receives the incoming "hello" packet, it initiates an authentication request with the client to complete the authentication process, the beginning of which was stored in the packet. Once authentication completes successfully, the process moves on.

4. The service, AMTConfig, catches the incoming packet and logs the data in the IntelAMT database, in the table csti_amts. This table contains all the relevant data for this system's identity.
   !csti_amts.jpg!

5. Once the system has been logged into the IntelAMT database, Intel SCS uses the database entries under csti_configuration to initiate what's known as the props script. This script is what will assist in the provisioning process. In Altiris case, it is oobprov.exe, located by default at C:\Program Files\Altiris\OOBSC\oobprov.exe. For an example of how Intel SCS knows about this, see this data snippet from the csti_configuration table:
   !csti_configuration.jpg!

6. On a busy SCS server you can look at Task Manager and see multiple instances of oobprov.exe running. The default settings allow 10 threads to work on provisioning requests at any given time. These threads will interface with the Altiris Database via the Altiris Agent on the local server system. In a standard setup the local system is also the Notification Server.

7. OOBPROV runs a SQL query to fetch the Fully Qualified Domain Name (FQDN) for the system it is to provision. The query is based off the following data points:

1. UUID passed to it via Intel SCS, Source is as follows: Database: IntelAMT, Table: csti_amts, Data Source: "Hello" packet from AMT system, Values used: uuid

2. Database: Altiris, Data-class: OOB Capability, Table: Inv_OOB_Capability, Data Source: Out of Band Discovery Task, Values used: _ResourceGuid - UUID

3. Database: Altiris, Data-class: AeX AC Location, Table: Inv_AeX_AC_Location, Data Source: Basic Inventory Agent, whether from Basic Inventory function or Hardware Inventory from Inventory Solution, Values used: _ResourceGuid - Fully Qualified Domain Name

8. The Query accomplishes the following: It takes the UUID from csti_amts, uuid and looks for a match in Inv OOB Capability, uuid. If a match is made, it takes the _ResourceGuid from the same table and makes a match of the same columns name to AeX AC Location. With the match it then reads the values stored under Fully Qualified Domain Name (I'm not sure why they didn't just label this column FQDN...).

9. Next, oobprov.exe hands back the FQDN it's read from AeX AC Location, Fully Qualified Domain Name and passes it to SCS. SCS takes this value and inserts it into the IntelAMT database at csti_amts, fqdn for the matching resource.

10. Next, oobprov.exe fetches the automatic profile set within Out of Band Management Solution. This is done in the Altiris Console under View > Solutions > Out of Band Management > Configuration > Provisioning > Intel AMT Systems > Resource Synchronization. This policy needs to be enabled for this step to work, and a default profile configured and selected under the dropdown labeled ‘Intel AMT 2.0+ to profile:'.

11. The profile provides the operational data for management of the AMT system. After AMT accepts the profile, the Provisioning process is now complete. Before this step, AMT functionality is not available on this system, and after this step only properly authenticated functions will be able to use Intel vPro on the target provisioned systems.

Troubleshooting

The following items can be considered break points for this process. If you've done provisioning you may have run into the symptoms produced by the following items. These are compiled as common areas of trouble in this process.

• The "Hello" packets only transmit for 24 hours, on a back-off schedule, before stopping altogether. If the Server is unable to provision in that time, with IP refreshes becoming more frequent, the system can be in a limbo state. See this article for steps to rectify: http://juice.altiris.com/article/3612/using-intels-rct-tool-restart-amt-hello-packets-enterprise-provisioning

• IP Address changes, refreshes within DHCP during a system's build process can leave SCS with an out of date IP Address for a system that needs provisioning. Coupled with the preceding issue this can leave the system in an unprovisioned state, leaving no ability of the SCS to contact the system to finish the process.

• Remote Configuration certificate is not properly installed on the server, producing authentication failure messages in the AMT logs.

• Oobprov.exe is unable to fetch the FQDN. The AMT system needs the Altiris Agent installed, have sent Basic Inventory when it had a valid FQDN (for example a system in the process of being built might not have a valid FQDN yet), OOB Discovery Task downloaded and executed, and data populated into the OOB Capability data class from the task in order for oobprov.exe to be able to fetch the FQDN. Conversely you can use the option in Resource Synchronization labeled, ‘Use DNS IP resolution to find FQDN when assigning profiles'.

A good resource for troubleshooting issues can be found here:

• http://juice.altiris.com/book/3699/troubleshooting-altiris-manageability-toolkit-vpro-technology

Conclusion

Knowing the underline mechanisms can help when troubleshooting or even when planning your environment. While not all details are provided here, the most essential are.

In part 2 we introduced the Server components used in Provisioning, including some key items to be aware of. In this installment we'll cover troubleshooting the server components in a symptom - cause - resolution format. The methodology should also allow help you understand how these components work for further troubleshooting efforts, or for simply understanding how the data is moving through the Provisioning process. This specific article covers the Console and the common errors that can appear.

Introduction

Once the server components are installed, and the AMT systems are in a correct Setup Mode, one must access the Provisioning Console to manage the Provisioning process. This console is located in the Altiris Console under View > Solutions > Out of Band Management > Configuration > Provisioning. This part of the series covers errors in the console, specifically to common errors scene after the installation has taken place. These errors can also surface due to environmental changes in the infrastructure.

Symptoms

This section lists all the symptoms covered in this article. Use this list to guide you if you are working on a specific issue.

• Provisioning Console Access Forbidden - Generally this is a 403 error on most of the Altiris Console Provisioning Nodes

• Provisioning Console Connection Closed - All the Provisioning Nodes show an error that the underlining connection was closed

• Provisioning Console User Not Authorized - This error relates to the access rights to the actual Provision Nodes, and can happen even if a user is an Altiris Administrator

• Provisioning Console Timeouts - We've seen timeouts occur in the console, when accessing the Intel AMT Systems list

Provisioning Console Access Forbidden

Problem

When accessing the Provisioning Console, the following error is thrown:

The request failed with HTTP status 403: Forbidden

Cause

When installing Intel SCS, the manual install defaults to HTTPS, using TLS for secure communication. If the environment is not setup for TLS/HTTPS, the Altiris Provisioning Console will be unable to authenticate to Intel SCS, throwing this error.

Resolution

1. On the Notification Server where Intel SCS is installed, open up IIS Manager.

2. Browse down into the Default Web Site and select AMTSCS.

3. Right-click on AMTSCS and choose Properties.

4. Select the Directory Security tab.

5. Click the Edit button under the Secure communications section.

6. Uncheck the box labeled ‘Require secure channel (SSL).

7. Click OK.

8. Click Apply and then OK.

Provisioning Console Connection Closed

Problem

The error ‘The Host Name cannot be resolved', or ‘ the remote connection was closed' appear when accessing the Provisioning Console.

The problem can also be seen when using the Test functionality on the DNS Configuration node. It may show a failed to obtain IP message.

Cause

When our Console tries to resolve the name to the Intel SCS Server (even when Altiris and SCS are on the same server) it fails and one of these errors are thrown. The difference can be in the perceived FQDN for the Server. Altiris is attempting to acquire the right IP address so it can communicate with SCS.

Resolution

There are two ways to fix this if a reinstallation does not correctly set the SCS identity within Altiris.

LMHOSTS or HOSTS files - We can update one or both of these files to contain the FQDN we're using to try and translate the IP Address. The difficult part is finding out what Altiris is attempting to connect to. Use the process below to find out what it is looking for:

1. See Part 1 concerning the use of OOB trace logging and Debug View.

2. Enable trace logging in OOB and launch dbgview.exe.

3. Try to access the console and produce the error.

4. Stop trace logging.

5. This is the difficult part. Normally I scan through the log looking for the host name of the server. Usually this shows up as part of an FQDN. One example of this is Altiris called Servername.domain, which did not respond, but Servername.domain.com was a valid name.

6. Do a Search for the Host Name of the system (Not FQDN as it may not be using the valid one). For example, MyServer.

7. Once complete, access the file named lmhosts (no extension). Place a line in the file with the Server IP Address and invalid name:

• 10.10.10.1 Servername.domain

8. Whatever invalid name was located in step 5, the above sequence can be used to give the computer the correct IP Address resolution. This resolves the issue. However there may be other steps needed. If this doesn't resolve the issue, continue to step 8.

9. Access the Service Location node in the Provisioning Console.

10. Change the option to ‘Alternate URL:'.

11. Specify a new location changing the name to one that resolves, for example:

• Previous URL: http://Servername.domain:80/AMTSCS

• Fixed URL: http://Servername.domain.com:80/AMTSCS

12. Click Apply to save the changes.

The difficult part in this process is locating what Altiris believe the name of the Intel SCS Server is. Since Altiris and SCS are not integrated, they do not have a mechanism that shows if they are on the same server or not. This is why this issue surfaced.

Provisioning Console User Not Authorized

Problem

After installation or after credential changes the typical error structure appears with the message:

• Current User can't view this page.

• Current user can't change settings on this page.

Note that the error does not have the Red error typically associated with other console errors.

Cause

After installation only the user who conducted the Intel SCS install has rights to the console nodes. Until other users are added, only this user (usually the Notification Server Application identity) has rights to these nodes. Notification Server role and scope security does not apply to the populating of the data to the right of these nodes (although it does control access to actually showing the nodes themselves in the left-hand tree).

Resolution

Follow these steps to give the necessary users rights to the Provision Console nodes:

1. Log into the Altiris Console as the Notification Server Application Identity, or the user used to manually install Intel SCS (one of these will usually be the authorized user).

2. Access the Altiris Console under View > Solutions > Out of Band Management > Configuration > Provisioning > Configuration Service Settings > Users.

3. Note the users who already have rights.

4. Click the blue + icon to add a user.

5. Click the ... browse icon to see a typical Notification Server Domain user and groups search window.

6. Add a group or user and click OK.

7. Under the Role: give Enterprise Administrator rights unless you want to limit which nodes are operable.

8. Click OK to complete adding the user.

If no user can access these nodes, the Intel SCS installation needs to be run again under the correct user. Run through these steps to complete this:

1. Log onto the Notification Server directly (or with the /console switch if you're using Remote Desktop) with the NS Application Identity.

2. In Add/Remove Programs, locate ‘Intel® Active Management Technology Setup and Configuration Service and remove it.

3. On the Notification Server, browse to install_path\Program Files\Altiris\Notification Server\NSCap\Bin\Win32\X86\OOB\IntelSCS\.

4. Launch the file AMTConfServer.exe and walk through the install. Be sure to use the Application Identity as the credentials for SCS.

5. When prompted for the database credentials, if permissible use the Application Identity.

6. Once completed log into the Altiris Console with the Notification Server Application Identity, then move back to step 1 of the previous sequence to add other users as necessary.

Provisioning Console Timeouts

Problem

Even in small environments we've seen timeouts on the Intel AMT Systems node, and much less frequently on the other nodes. The timeout throws a .NET error and the page is replaced by a timeout error.

Cause

The cause is not known at this time. The timeouts do not seem to occur always at particularly busy times for the Notification Server, so it is difficult to know what causes them. When there are plenty of resources available the timeouts generally do not occur, though if the server is extremely busy it doesn't always occur. It appears to be caused by varying factors.

A refresh after the timeout error often loads the page just fine. This suggests the loading the page gets into a loop or hung state, instead of a true processing timeout issue.

Resolution

No full resolution is known at this time, but a few items can help minimize the impact of the issue.

1. Remote Consoles - We've seen remote consoles perform better than having the console loaded directly on the Notification Server

2. Refresh - Normally the timeouts occur without loading any of the frames within the page. If you click on the link or hit the refresh for the Intel AMT Systems page and no frames load within a minute, refresh the page. Often when the page is refreshed it then loads correctly, even quickly.

Conclusion

Once the console has been restored, the Provisioning process can be configured and initiated. Because of the all or nothing nature of most of these issues, they must be overcome before even being able to properly setup and configure Intel SCS for the Provisioning process. The above resolutions cover the methods used to resolve these issues at multiple sites.

*This is a repost of this article to the Activation section of the Site*

Troubleshooting issues with the Intel® AMT Provisioning process can be a daunting prospect. This series walks through the troubleshooting methods to pinpoint where problems originate and how to fix them. Use Part 1 to troubleshoot the AMT systems when provisioning is not occurring. If the issue is on the client side, this document should provide the tools to diagnose and fix the issue.

Introduction

There are several modes a vPro capable system can be in when it arrives at the customer site. The modes are:

1. AMT disabled

2. AMT enabled, not in Setup Mode (factory default)

3. AMT enabled, not in Setup Mode (Password has been changed in the MEBx)

4. AMT enabled, in Setup Mode for TLS-PSK

5. AMT enabled, in Setup Mode for Remote Configuration

6. 4 and 5 in ‘Hello' Packet Mode disabled

Each of the modes have their own quirks, and understanding the modes will help determine what state a system is in, and how to change a system from one state to another.

Versioning

It is important to understand the different versions of not only the local AMT build, but of Altiris' Out of Band Management with the Intel SCS Component. See the following table:

Note the following points when working with the different versions:

• Versions 2.0, 2.1, 2.5 do not support Remote Configuration

• Versions 2.5 and 2.6 are notebooks

• Versions 2.2 and 2.6 are upgrades to versions 2.0, 2.1 and 2.5 respectively and provide the additional functionality of using Remote Configuration for Provisioning

• Intel SCS version 1.2 was unstable. It's recommended to upgrade to 1.3 or upgrade OOB to 6.2.

• Versions 2.2 and 2.6 are not supported for Remote Configuration unless Intel SCS is upgraded to version 3.2.1. Check the following KB articles for more information:

• Intel Communities

• Intel Communities

AMT Setup

Each mode for AMT sets the system in a specific state. See the brief descriptions below of how AMT acts in each state:

1. AMT disabled - In this situation AMT must be enabled either manually by looking into the Intel MEBx (Ctrl+P at startup) or by using the RCT Tool. The following article covers the use of this tool, including data on the command-line switch that can be used to enable AMT:

• http://juice.altiris.com/article/3612/using-intels-rct-tool-restart-amt-hello-packets-enterprise-provisioning

2. AMT enabled, not in Setup Mode (factory default) - This is the required mode to use USB One-Touch for provisioning. If a user or the OEM has logged into the MEBx and changed the password, the system is no longer in factory default and the One Touch method will not work.

3. AMT enabled, not in Setup Mode (Password has been changed in the MEBx) - One Touch will not work, but manually entering the PSK or setting into Remote Configuration mode will allow the system to enter Setup Mode.

4. AMT enabled, in Setup Mode for TLS-PSK - All Provisioning is encrypted using TLS, however the inner security workings can differ. For Pre-shared Key (known as PID PPS) a public and private key are used. The manufacturer can set a specific PID PPS on the system or a user can auto-generate them. The key is that both the client and server have to have the key in order for authentication to work.

5. AMT enabled, in Setup Mode for Remote Configuration - All 2.2, 2.6, and 3.0 version AMT systems come in this mode unless the OEM is explicitly instructed to set it differently. The point of Remote Configuration is to avoid visiting the AMT system in order to get it provisioned for manageability use.

6. Modes 4 and 5 in ‘Hello' Packet Mode disabled - This is common if the system is not immediately hooked up to the production network. All systems will fall into this state if they transmit the ‘hello' packet for 24 hours.

Troubleshooting Tools

Before we get into the actual symptoms, we'll cover the tools used to determine where the problem is coming from. While not easy to use, the logging capabilities allow us to verify if the correct processes are functioning on the local system.

AMT Logs

The Altiris Console has direct ties into the AMT Logs captured in the IntelAMT database as a normal part of operation. The Logging level is set in the Altiris Console under View > Solutions > Out of Band Management > Configuration > Provisioning > Configuration Service Settings > and select General. Debug Warning is recommended so you get both Errors and Warnings.

The logs are accessed from Provisioning > Logs > and select ‘Log'. Entries here will reveal problems during the provisioning process and other Intel SCS functions.

OOB Trace Logging

Out of Band Management has the ability to log trace details to a debugging program. See the following KB article on details on how to set this up:

• Intel Communities

Trace logging will log everything from console accesses, to oobprov.exe calls from IntelSCS. When oobprov.exe is called, all actions are logged to trace, which can capture problems with the provisioning process.

Wireshark

While the two above tools are distinctly for Out of Band Provisioning, Wireshark tells the whole story of what is coming and going across the wire. It's important to know what the AMT clients are sending, especially in the ‘Hello' packet, and what the server is responding with.

Wireshark can be obtained from: http://www.wireshark.org/. While this is the recommended tool, any network trace capture program can be used to examine the network traffic between the AMT client and the Provisioning Server.

Altiris Knowledgebase

All know errors and issues we've run across have been documented in the Altiris Knowledgebase. If you have a specific error, search in the KB and see if we have a documented fix for it. Access it directly here:

• https://kb.altiris.com/

Symptoms

The following symptoms point to problems with the local AMT system or its ability to communicate to the Provisioning Server so that Provisioning can occur.

System Missing

A common symptom for new AMT client systems is that the system, even if believed to be in Setup Mode, doesn't show up in the Altiris Console under Intel® AMT Systems. The causes vary, but the following methodology should help pinpoint where the problem originates.

Is the system sending ‘Hello' packets? Walk through this procedure to determine if it is or not:

1. Does the AMT Log contain entries for the system requesting Provisioning? The identifier in the logs is the UUID. One example of an error that would prevent a system from showing up is ‘failed to find PID mapping', meaning the requesting system is trying to authenticate with a PID that the Server does not have. Either import any keys provided by the OEM or other provider, or manually enter in the PID PPS under the ‘Security Keys' section of the Provisioning Altiris Console.

2. If no entry appears for the system, place Wireshark on both the AMT client and the Server. Now initiate a restart of the ‘Hello' packet sequence by turning the AMT client off and unplugging it from power. Drain the capacitors by pressing the power button while unplugged. Generally the power LED will light for a moment before fading dark. Plug the system back in. Does the Server show hello packets (sending on port 16994, with destination port 9971) coming in from the system?

3. If the server doesn't show any incoming ‘Hello' requests, fire up Wireshark on the local system to see if we see any ‘Hello' packets heading out. If they are actively leaving, something is blocking the traffic from reaching the Notification Server. These ports are standard TCP calls. See the next section labeled ‘Provision Server'.

4. If no ‘Hello' packets are being sent, the system may be in a non-Setup State. At the AMT system access the Intel MEBx by pressing Ctrl+P at startup. Is the password what was setup during Setup Mode, or will it only accept Admin? If none of the valid passwords work, this machine may be in an unworkable state. Unplug the CMOS battery for 15 seconds to put the machine back in Factory Default Mode, and Setup as necessary.

Provision Server

With Wireshark we can prove a system is sending ‘Hello' packets out on the wire. The destination is an important distinction as usually this will be simply the name ProvisionServer. By default, Remove Configuration and TLS-PSK will target the simple name ProvisionServer. It's up to the administrator to properly direct that Hello packet to the Notification Server.

1. If you ping ProvisionServer from a command-prompt, do you get the IP Address of the Notification Server? A CNAME record needs to be created in DNS to correctly direct the hello packets. Check page 21 of the Admin guide located at this KB article: Intel Communities for more information.

2. Another place you can test the DNS functionality is under Provisioning in the Altiris Console. Select the ‘DNS Configuration' node. Click the ‘Test' button to initiate the test. A correct IP Address signifies that DNS is working correctly from the Notification Server. The ping test is still important to signify that the client can also resolve the name.

1. If the network cannot support this CNAME, only two methods remain. You can set the Provision Server IP in the MEBx directly. You can also use the RCT tool to simulate the Hello packet and send it to the NS directly (see the previous link to the article on RCT usage).

Conclusion

Part 2 of this series covers the Server components for Provisioning. If you've read all the symptoms and suggestions, you'll note that there is crossover when troubleshooting between the client and the server, regardless of where the problem lies. See Part 2 for the continuation of Provisioning Troubleshooting.