If you're reading this blog posting, hopefully you've read my blog post on CIRA last week -

http://communities.intel.com/community/openportit/vproexpert/blog/2009/11/10/cira-and-fast-call-for-help--what-is-it-where-can-i-find-it

Firstly, I wanted to share with you that over the past week we have actually worked through an entire end to end setup with a real-world customer (i.e. not just inside Intel labs) and now we have CIRA and AMT functionality over CIRA working successfully!

If you're wondering which of the management consoles and MPS/vPro Gateways were used - it was LANDesk 8.8 SP3 in this case (remember that LANDesk bundle their own MPS/vPro Gateway offering). If you're looking to get this to work in your environment (CIRA with LANDesk specifically) please do get in touch and I can share some specific current LANDesk pointers with you (that are not mentioned in this blog posting).

Some of the things we came across last week which are good pointers to pay attention to:

1. There are 4 ports that get configured with the MPS which are fully configurable (i.e. they are not restricted to being a specific port number) - however, you cannot re-use the same port number, you need to have 4 distinct port numbers (sounds trivial, but it happens).
2. You can use port 16993 as one of the port numbers, even though that is the port that is used for https connections in AMT (there is no conflict)
3. In the httpd.conf file - instead of havinga deny all and allow specific IP addresses, you might want to change to allow all
4. CIRA relies on the DHCP option 15 that is allocated where the vPro client is to be different than what it was pre-configured with - that is how the system knows it is outside the corporate environment. If DHCP option 15 happens to be blank where your vPro clients connect from - that is good enough. Blank is considered different and CIRA works fine.
5. Currently, you should install the LANDesk agent after provisioning is completed
6. Check through selecting the 'vPro Status' operation on a provisioned vPro client to ensure all the LANDesk NED settings have been deposited properly on the vPro client prior to taking it out of the corporate environment.

Btw, the CIRA connection is established through a user click at the OS-level using the IMSS utility.

So the bottom line is we now have close to 100 systems that are confirmed to be have full AMT functionality working over a CIRA connection in a real live environment - it works! (

The 2nd part of the blog can be considered a more 'advanced topic' and is devoted to what happens if your management console of choice doesn't currently support CIRA...

One Management Console for example that is currently not supporting CIRA is Microsoft SCCM (even with SP2).

The options as I see them, are:

1. Contact your software vendor and ask them whether they support Intel - Intel works with multiple software vendors on incorporating support for various Intel vPro features (CIRA amongst others) - they can hear it from us, but it is much better if they hear it from you.
2. Your software vendor might have plans to introduce support for CIRA, however it is further down the line - so it is just a question of time.  
3. Try and engineer something yourself to have CIRA work in the environment you have setup

At least for testing your environment for what CIRA would look like, you could leverage the WebUI tool. You would need to have an MPS installed and configured first of all. Thereafter, all that you need to do is configure the proxy settings in the web-browser you are using to the IP address/FQDN of where you have your MPS installed and also enter the default http proxy port of 8080 - that will be sufficient for getting your WebUI to work over a CIRA connection.

If you use Microsoft Internet Explorer you are limited only to the http proxy portion which will allow several of the AMT operations to work over a CIRA connection, but not SOL/IDER for example.

If you are using Mozilla Fire Fox for example, you can configure a SOCKS proxy as well, which can handle routing SOL/IDER traffic as well.

If we take the example of Microsoft SCCM, what you can do is to use the scripting framework that has been used successfully for something like: providing out of band 802.1x in Microsoft SCCM SP1 (it is natively supported now in SP2) - http://communities.intel.com/message/10877

You can configure the correct settings for the vPro client to be able to contact the MPS Proxy Server and establish a CIRA connection between the MPS Server and the vPro client, however you will still need your management console to integrate and be aware of this CIRA connection to be able to do something useful.

What you could do at this point is to configure a 'transparent proxy' - what that would typically entail is to configure the MPS IP address/FQDN as a proxy routing that will be inserted in the headers of packets that go through the router to which the Server that is hosting the management software. You can use something like Cisco WCCP (Web Cache Control Protocol) to set this up. At this point, Microsoft SCCM will not be aware that the packets it is sending are actually being re-routed through the MPS to the vPro clients (which is aware of the remote vPro client) and that is why this is called a transparent proxy.

A caveat/disclaimer I would add though is that albeit technically feasible you would need to put together the full working solution yourselves and support it yourselves.

What is it?

When vPro and more specifically AMT was initially designed and engineered it was architected to work on an internal corporate network which allowed for the Server to client communications model. The problem was that many organisations have client PCs that are actually situated outside the corporate environment and were excluded from the reach of the vPro benefits available to systems residing within the corporate network. The reason for this is that client PCs that are not on the corporate environment would be sitting behind a home router and would actually posses a private local IP address that is not publicly addressable - i.e. it is not unique and the Management Console has no way of reaching that remote client. The solution to this situation is what is called CIRA - Client Initiated Remote Access.

The term Fast Call for Helpis what we refer to the use case that is enabled by CIRA (which is a means to an end, but not a use case on its own). It specifically addresses a help desk type scenario where the PC is broken and it is being fixed from remote by an administrator or technician.

How does it work?

It works on the principle that as with any usage of a PC behind a NAT'd router, once the client initiates a request (say for a web page) and the information returned comes back to the router, the router knows locally which PC to forward the information back to. The important distinction from the analogy used is that this connection is created Out of Band and does not rely on the operating system or some local software client agent being available or in a healthy state.

The connection that is initiated by the client arrives at the vPro Enabled Gateway which needs to be 'publicly reachable' - so it would typically reside in a DMZ and by protected by an external firewall which might have some port forwarding.

The management console has a listner for incoming CIRA connections and once such a connection arrives it can perform AMT commands on the remote vPro client.

The high level flow is as follows (with a graphical representation below):

1. The user of the remote vPro client initiates the connection to a component that acts as a proxy Server and is called the vPro Enabled Gateway (aka MPS - manageability presence server).
2. The connection can either be initiated manually by a user in an OS level utility or pre-OS level with a key combination
3. Alternatively, the connection can be scheduled to automatically be initiated according to a pre-determined time frequency
4. Once the connection reaches the Gateway, a secure encrypted tunnel is established back to the vPro client
5. At this point the Management Console which is sitting inside the corporate environment is notified of the incoming connection from the vPro client
6. The administrator/technician which is using the Management Console can now initiate any AMT command to the remote vPro client

What components are required for getting CIRA and Fast Call for Help to work?

1. vPro systems
2. Management software that has built in support for Fast Call for Help
3. vPro Enabled Gateway

In addition, you should also be aware that there are configuration files that need to be edited for the vPro Enabled Gateway, some configurable ports need to be open and that AMT provisioning (with CIRA profiles) are a pre-requisite.

Which vPro Hardware do I need to take advantage of Fast Call for Help?

Any vPro system that has AMT Firmware 4.0 and above supports Fast Call for Help. That means any 4.x, 5.x and now the up and coming 6th generation of vPro which is being released in the 1st quarter of 2010. The new capability which is being introduced in 2010 is that this CIRA connection can be initiated over a wireless network interface as well, whereas today it is limited to being initiated over a wired network connection.

Which manageability software is available today for implementing a utilise CIRA capabilities?

1. Symantec Management Suite version 7 (formerly Altiris Management Console and aka CMS7) Beta II
2. LANDesk Management Suite 8.8 SP3
3. Setup and Configuration Service (SCS) 5.x and above (including the Intel DTK) also support CIRA

Which vPro Enabled Gateway products are available today for setting up a CIRA capable infrastructure?

1. Checkpoint Secure Gateway (interoperable with the Symantec Management Console, but not with the LANDesk console)
2. LANDesk Gateway which is embedded inside the LANDesk Management Console (however does require to run specific installer for MPS)

Why am I blogging about this now?

CIRA and Fast Call for Help were actually supported in Intel Firmware from version 4.0 which was released about 1.5 years ago. Unfortunately all the components required to make Fast Call for Help work were either unavailable or had stability issues. However, today the components exist and are validated to work successfully (with a few known issues that are being addressed). Therefore, if this is of interest to you then you are in a position to implement Fast Call for Help in your environment today. We would welcome anyone out there that is interested in trying to implementing this

Is this everything I need to know?

There are more technical details required for a successful implementation, however this should provide a good introduction and starting point. If you have any questions, please don't hesitate to contact me.

If you have experienced WMI related issues with the SMS Addon or are worried about potential WMI related issues you might want to read this...

The SMS Addon relies on calls to the WMI service for interacting with SMS and it calls the WMI service more than you might think. For some customers it has been observed that the default frequencies of these calls coupled with the amount of data/entries (such as number of advertisements, number of collections etc.) causes a very high load on the WMI service and a subsequent crash of the service that requires a full reboot of the entire Server (not just the service). Obviously that is not an acceptable situation for a production customer, however there are 8 specific instances where the SMS Addon makes calls to the WMI service which can be tweaked to provide a potential resolution.

You won't see these 8 registry keys in the registry as the SMS Addon uses default values inside the SMS Addon code. You would need to generate these keys yourselves and provide values in order to override the defaults used. The list of these keys and the what they are for are as follows:

All registry keys need to be created in: HKLM\SOFTWARE\Intel\Intel(R) AMT Add-on\GEN\ folder and Dword values need to be entered in minutes. Any value but 0 is legal. There is no disable value and hence 'disabling' is achieved by setting a very high value such as 5 (2,628,000 minutes) or 10 years (5,256,000 minutes).

These is the list of the scheduled WMI related operations:

1. NewSysScanInterval used to identify new systems in SMS (discovered by SMS discovery methods like AD discovery) – by default runs every 60 minutes – not needed if no systems are managed by SMS Site (such as Central SMS Site).

2. DeletedSysScanIntervalused to remove information about systems removed from SMS – by default runs every 60 minutes – not needed if no systems are managed by SMS Site (such as Central SMS Site).
3. SCSSystemScanIntervalused to find new machines in SCS – by default runs every 60 minutes – not needed if no systems are managed by SMS Site (such as Central SMS Site) and also this doesn’t apply if add-on isn’t integrated with SCS.
4. SystemChangeMembershipScanIntervalused to identify systems membership in groups (for system defense settings) – by default runs every 60 minutes – not needed if no systems are managed by SMS Site (such as Central SMS Site) or if system defense isn't being used.
5. OffLineSystemScanInterval – used to run commands on machines that were offline (used for system defense and event registration) – by default runs every 5 minutes – not needed if no systems are managed by SMS Site (such as Central SMS Site) or if system defense or event registration aren't being used.
6. SCSrequestScanInterval – used for tracking requests from add-on to SCS (unprovision, reprovision) – by default runs every 5 minutes – not needed if no systems are managed by SMS Site (such as Central SMS Site) or no un/reprovision commands are run from the add-on; also this doesn’t apply if add-on isn’t integrated with SCS.
7. UpdatedAdvertisementsScanIntervalused for reading advertisement definitions – by default runs every 2 minutes – not needed if no systems are managed by SMS Site (such as Central SMS Site) or if advertisement wakeup or system defense aren't being used. Runs frequently to pick up any changes to advertisement settings.
8. AdvertisementChangeScanInterval used for identifying status of systems with system defense settings on advertisements – by default runs every 5 minutes – not needed if no systems are managed by SMS Site (such as Central SMS Site) or that don’t run system defense on advertisement.

Tal

If you are deploying vPro have you thought about what might happen if you are running virtual machines on your vPro clients? The virtual machine will have its own virtual IP address which will be different from the actual host IP address which is the one we want when using vPro. What was happening was that the vPro machine was getting provisioned with SCS, but then when discovery was attempted with the SMS Addon, the provisioned vPro system could never be discovered and sub sequentially managed because the systems showed up as 'Outside Site Boundaries'. It seems that the SMS Addon had a 'flaw' in that it assumed that a machine will only have a single IP address. I cannot say based on my experience whether sometimes the host IP address will be picked up and the problem won't manifest itself or whether it will always be the virtual IP address that will be picked up.

I emphasize that I have experienced this situation when using the SMS Addon and not with any other management consoles. That doesn't mean the problem won't manifest with them as well or whether the design of those products caters for this. If anyone has come across virtual machines on vPro clients and this has or has not caused issues I would be interested in your feedback.

At a high level, the nature of the fix will be that the SMS Addon will not assume there is a single IP address and will loop through the IP addresses. If an IP address comes up as outside the site boundaries the sms addon will loop onto the next available IP address. This process will continue until it has exhausted all the IP addresses available on that local vPro machine and only at that point 'give up'.

The fix within the SMS Addon will be incorporated into an official hotfix which will be published soon; I'll provide an update once it has been released.

Some of you might find that you need to provision your vPro systems with hostname only as opposed to the generally recommended and accepted fully qualified domain name - FQDN. The typical reason for using hostname only would be because the FQDN that appears at the OS level of a client will not match what will appear in the DNS forward and reverse lookup zones.

You might ask - how can that be? An example from one of my customers is that they have a network domain that is appended to entries in DNS via DHCP option 15. However, their DNS/DHCP is not integrated with their Active Directory and a different AD domain exists. Therefore when using the activator tool to extract the FQDN of the system and write it into the SCS DB as part of the provisioning process, it will conflict with the FQDN that appears in DNS. This will not cause a problem for provisioning per se, but for trying to access the machine later on using a management tool/interface it probably will.

Note: if you are using TLS certificates (server or mutual authentication) then hostname only provisioning will probably not be a viable option for you in any case because of the scrutiny of certificate exchanges that check whether domain suffices match.

So what does all this have to do with SCS 5.0? In 3.x versions of SCS there wasn't anything special you needed to do to allow hostname only provisioning, but with SCS 5.0 there is. When you construct your provisioning profile, in the optional settings you will need to select domains and make sure you select the checkbox of ‘Allow configuration when platform has no domain name'.

What if you are doing AD Integration?

If you are opting for AD Integration (as many will), you will have a problem with SCS 5.0. The reason for this is that the Kerberos protocol will insist that the request ticket will rely on DNS for your FQDN entry, whereas your object within the AMT OU in Active Directory will have a service principle name field (SPN) that will not match that FQDN - the SPN will have hostname and the request ticket will be whatever it is in DNS and at that point the kerberos negotiation will fail. It is important to stress that provisioning will work fine and you will see no hint to these issues, however once you try and access the machines via WebUI or any management console, you will have a problem.

The fix for (for now) will be to edit the SPN field for each provisioned machine's object in AD to what will appear in DNS. The SPN field has an entry for each port number that could potentially be used for AMT purposes (16992,16993,16994,16995,623,664) but if you are not using SOL/IDER and are not using certificates (you won't if you've opted for hostname only in any case) then the only field you should need to change is for the 16992 port. You can edit this field manually using ADSIedit. A more scalable solution will require a script to run as a post provisioning step to edit the SPN. A solution within SCS 5.0 is not available at present.

Ta

SCS 5.0 is the latest version of the Intel Setup and Configuration Service. This new version boasts a number of fundamental and exciting additions to the world of vPro:

1. You can enjoy the benefits of Active Directory Integration without the need to extend the Active Directory Schema!

2. You can use Windows Authentication to communicate with the SCS Database

3. The SCS Console version 5.0 has a much nicer and professional looking user interface

4. The performance, stability and logging capabilities of the application have notably improved

5. You have the ability to dynamically create collectoins of AMT Systems based on different filter conditions

6. This is still early days for AMT Firmware versions 4 and 5 and the use of CIRA (Client Initiated Remote Access) and MPS (Management Presence Server) but it supports them

Note: If you are using SMS as your Management Software you will need to use the Intel (R) Client Manageability Addon version 5.0 which is available for download from the following url: http://downloadcenter.intel.com/Filter_Results.aspx?strOSs=All&strTypes=All&ProductID=2609&lang=eng&OSFullName=All%20Operating%20Systems

To emphasize the point - you will not be able to use SMS Addon version 3.3 with SCS 5.0. SCS version 5.0 will be bundled already for you with the Addon version 5.0.

Some potentially useful technical insights that I have gathered through my experience of being an early adopter of SCS 5.0 through trying to deploy it at a large-scale enterprise customer:

1. If you opt for having windows authentication (as opposed to the dummy SQL account which was part of the design up until SCS 3.3) you will need to opt for the custom installation path. In there you will be prompted to specify twice the user for running the AMTSCS and AMTSCS_RCFG virtual directories in IIS. You will need to specify the same username and password of the accounts that are running your IIS services where your SCS is being installed. Pay attention to this step - if you specify any user other than the user that is running the IIS services: this could a local account for example and not a domain account, then you will not be able to log into SCS via the SCS console.

2. When you opt for the windows authentication to DB you wil not be able to use the default website on IIS. If you are creating a new website and you are going to opt for https connection, make sure your new website is setup with the server ssl certificate. You will also need to remember to stop the default website and have your new website running.

3. You will need to remember to delegate permissions to the account that is running the SCS service on the AD OU for AMT objects, but this time it will be for objects of type 'Computer Objects'. There will not be a conflict with the Host OS level computer objects as these AMT Computer Objects are seen as user objects.

4. You have the option to create the DB separately using an SQL Standalone DB script (i.e. not as part of the install wizard) however even if you are opting for windows authentication to your SCS DB, you can achieve this by only running the wizard (the custom install path). If you have created the DB prior to SCS install, you can point the SCS service to this DB instance during the install wizard.

5. A general point to note that would apply to any provisioning with SCS (not just SCS 5.0) - when you are creating a profile

6. Another point to mention is that the profile ID number is not fully deterministic if you don't run through the config of a new profile without pressing cancel at any point. For example, if you have the default profile as profile ID #1 then when you try and create an additional profile and at some point click cancel and then try and create a new profile it can eventually have a profile ID of #5 for example. This can start becoming a problem if you rely on the profile ID number as part of your provisioning process using the Activator Utility for example, as you can only pass the profile ID as far as the SCS API is concerned, yet if you've hardcoded the profile ID in some file on the vPro client where your Activator Utility will run then you cannot know for sure until your profile has been created in SCS what its profile ID will be. If you are editing an existing profile, its ID number won't change. You also cannot go into the DB and change that value manually as it is a primary key and is auto generated as part of an indexing mechanism in the SCS code. - this one might be a bit tricky, so contact me if you need me to clarify.

7. I don't know whether you've noticed any sluggishness in the past when trying to install 3.x versions of SCS - for example with one of my large customers it would take 1.5 hours to install SCS because of looking up users in a rather large Active Directory; whereas with SCS 5.0 it takes 5 minutes at most.

8. Whilst I haven't taken advantage of the capability to create collectoins of AMT systems I wanted to point out one of the main benefits of this feature. I have been faced in the past with situations where I need to perform an operation through SCS on many machines, but not all machines. Therefore the global operations in SCS 3.x versions only gave me the possibility of running the command on a single or all machines. Now I can tailor which machines I want to perform operations on.

My overall recommendation to you is to give SCS 5.0 a go. It is easily the best SCS version that has been released. I have blogged about it as part of my first hand experiences - I have had nothing to do with its development and I am speaking out of the objective view of a user. Hope you find this useful.

Ta