Intel Communities : All Content - The Data Stack

Powering the Way to Exascale Infographic

webadmin@intel.com — Mon, 17 Jun 2013 20:41:09 GMT

HP Moonshot: Lifting Off with Intel Atom and Xeon Processors

webadmin@intel.com — Fri, 14 Jun 2013 20:31:28 GMT

On my way back from Vegas now – a jam packed, 36 hours where I never saw the sun and never left the hotel (and didn’t sleep much) – but well worth the trip all the same. I attended HP’s Discover conference for two days with a primary focus to listen to HP’s top executives and a number of their customers discuss the new Moonshot system that launched just this past April. And it wasn’t hard to do – Moonshot certainly was the rock star of the event!

Honestly, I was expecting a lather, rinse, repeat from the recent launch, but was pleasantly surprised to see HP do a double click down into exactly when, where and why a customer would want to use Moonshot…and even more importantly, where not to use Moonshot. This type of clarity was exactly what the audience was looking for in my opinion and Paul Santeler, VP and GM of HP’s Hyperscale division did an excellent job of it in his break out seminar. A few key points:

Moonshot is not a replacement for their traditional server systems like the DL/SL line and blades – it’s an augmentation. Designed for light weight, scale out workloads and consumable only in a fully loaded chassis (45 Intel® Atom® S family SOC-based cartridges today, with option to quadruple the density with 180 cartridges with the next gen “Avoton” based cartridges coming later this year) – it’s a platform targeted at web-scale environments with applications written and optimized for light weight, distributed workloads
The application is “back in charge”. Moonshot is configurable across compute, network and storage based on the application need. Instead of running every DC app on a general purpose server, light weight, scale out applications can run on a flexible, easy to configure infrastructure. One can now optimize across such workloads as static web, batch analytics and simple content delivery, all by turning the knobs on the HP Moonshot system. And although there is only one production cartridge shipping today (based on the world’s first 64-bit, 6W, data center class SOC - the Atom S 1200), soon there will be a range of cartridges that span software compatible Atom to Xeon® processors to address a broader range of workloads.
Good things come from collaboration. Intel and HP have been working together on Moonshot for a number of years, since the initial design and development stages. We are proud that HP chose Intel to be the lead partner for Moonshot and we will continue to innovate together and deliver more value to our end users. Together we bring decades of experience, a full platform portfolio and a developed ecosystem just waiting to explore Moonshot. The best is yet to come.

Congrats to HP on a successful show. Big kudos to Meg for weaving in Kevin Bacon into the opening keynote (hit me up on twitter at @RaejeanneS if you want more on this!). And thank you for choosing Intel as your collaboration partner.

What’s next for Intel in this space? Well, my whirlwind travel schedule continues next week at the ODCA Forecast’13 and GigaOm Structure’13 events in San Francisco, a great confluence of enterprise experts talking cloud adoption at Forecast and industry insiders discussing the future of cloud infrastructure innovation at Structure. In the meantime, for the latest on what Intel is doing with our products, solutions and customers, please visit the IT Center on Intel.com.

Partners Extend the Benefits of Intel Distribution for Apache Hadoop Software

webadmin@intel.com — Thu, 13 Jun 2013 16:57:28 GMT

There’s more to the Intel^® Distribution for Apache Hadoop* software (IDH) than meets the eye.

Intel is building an entire partner ecosystem around IDH that extends optimized hardware, data storage and analytics support to help ensure that IT orgs get the value and intelligence they need out of big data.

IDH helps organizations store and analyze big data by providing an open source data management platform with the hardware-level security, manageability and performance acceleration features of Intel^® Xeon^® processors. It also comes with technical support, training, and professional services from Intel. IDH is the distribution of choice for enterprises seeking to deploy open source Hadoop for processing big data at multi-petabyte scale.

However, the compute intensive processes of Hadoop require a combination of hardware and software optimizations and specialized analytics and visualization tools to deliver the insights, scale and ROI demanded of big data. In addition to server architectures and IDH, Intel provides a number of tools to help manage Hadoop, including Intel^® Graphbuilder, which enables distributed graph analytics on top of Hadoop. However, Intel turns to its partners to help create a larger vendor ecosystem of optimized and co-engineered solutions and to build a more complete IDH computing environment.

RainStor Takes Complexity, Cost out of Big Data Querying

I estimate that up to 90 percent of organizations that have deployed Hadoop clusters today are just using them as an ETL offload. In other words, they are using Hadoop to store big content, but haven’t gotten around to taking advantage of Hadoop’s benefits as an engine for big data analysis. That’s where RainStor comes in. RainStor helps organizations achieve business insights at lower costs than other data stores, and uses familiar query and BI tools to reduce the complexity of big data analytics.

Rainstor for Hadoop* is a big data infrastructure that runs natively on the Intel Distribution for Hadoop. It’s made to not just handle the velocity and growth of today’s data, but also tackle the changing nature of data itself—log files, web clickstreams, Twitter content, machine generated data, and more, all in great volume.

But most IT admins don’t want to become query specialists or data scientists just to analyze their big data. Using RainStor, they can run real SQL queries on Hadoop stores, taking much of the complexity out of big data analysis. Rainstor is standards-based, and it uses specialized JDBC and ODBC drivers to peer into persistent Hadoop data. It can then run queries through the data using the familiar SQL environment—and DBAs don’t have to be stuck with HiveQL. To some that could be like telling a C++ developer that JavaScript coding requires no further training.

Rainstor also offers data compression and de-duplication capabilities that can lower the storage footprint by as much as 20 to 40 times. These compression features not only reduce the hardware needs and costs of big data, they also speed up querying. Rainstor is a great example of how partners are building out the Intel Hadoop ecosystem with innovative technologies. Learn more about Rainstor for Hadoop, and follow Tim and the growing #Intel #BigData Hadoop community at @TimIntel.

Intel Atom S1200 helps HP Moonshot to lift off in Vegas.

webadmin@intel.com — Fri, 07 Jun 2013 14:39:11 GMT

HP and Intel have been collaborating for a few years on project Moonshot to bring new levels of density, efficiency and TCO for light weight web workloads such as static web and dedicated hosting. Recently, HP unveiled their first generation Moonshot systems and I am excited to see that the first and only production HP ProLiant Moonshot servers available today are based on the Intel Atom S1200 processor family.

HP chose to lead with the Intel Atom processor for many reasons. First, HP and Intel have a long history of collaboration and we have brought many innovations to market first on Intel and HP’s platforms. Second, the Intel Atom S1200 processor is the industry’s only available 64-bit SoC with critical data center class features such as full 64-bit software ecosystem support, ECC and Intel Virtualization Technology - all within an ultra-low power 6W TDP. This means that today the ProLiant Moonshot servers using Atom S1200 can drop into any environment and software applications will run seamlessly on the server without porting needed. The lower power you want, with the software applications you need. Third, this SoC was designed for targeted lightweight web scale workloads, including low-end dedicated hosting, simple content delivery, and offline batch analytics making Intel Atom S1200 the perfect SoC solution for HP Moonshot’s target markets.

Moonshot servers with the Intel Atom S1200 are shipping to customers today and receiving great reviews. We look forward to sharing more results from customers going forward as these implementations go public. We also look forward to seeing Moonshot systems that will take advantage of higher density HP ProLiant Moonshot servers using Intel’s next generation Atom SoC coming later this year. The next gen servers will be built on Intel’s 2nd generation 64-bit Intel Atom SoC, code named “Avoton”. Avoton is built on Intel’s leading 3D tri-gate 22-nanometer (nm) process technology and is based on a new microarchitecture codenamed “Silvermont”. It will feature an integrated Ethernet fabric controller and deliver improvements over today’s Intel Atom S1200 in performance per watt and energy efficiency through a combination of new capabilities, new microarchitecture and leadership manufacturing technology. Avoton is now being sampled to customers and the first systems are expected to be available in second half of 2013. Moonshot servers using Avoton will quadruple the density (4 Avoton SoCs per server) vs. the current generation just announced using Intel Atom S1200.

2013 is and will be a great year for Intel and HP Moonshot. We have not only enabled the first Moonshot system to lift-off but with Avoton we will also bring HP Moonshot’s customers a revolution in energy efficiency and performance per watt to drive major TCO improvements when processing lightweight web scale workloads.

Learn all about it at this year’s #HPDiscover in Vegas June 11-13. Sign up for the exclusive NDA session BB BB4355 on Wednesday, June 12th at 12:45PM in Murano 3206 and follow all the action on Twitter at @IntelITS.

19 Years of Data Center

webadmin@intel.com — Thu, 06 Jun 2013 16:58:22 GMT

I woke up this morning and realized that yesterday marked my 19th year at Intel. Just as turning 29 and 39 freaked me out more than the actual decade birthday in my life, year 19 at Intel woke me up that soon I’d be 2 decades into a career and quite honestly, most of it went by so fast that I can’t quite say how I got here some days. Should I have tried other companies/industries, should I have tried other paths than marketing or should I have ventured beyond the data center ?!

So, I did some deep contemplation during my [very long] commute this morning and here is what I came up with….I love my job, I am proud to work for Intel and despite the [very long] commute – one couldn’t ask for a better employer and career than with Intel’s data center group. But, I also realized that there were a few things along the way that helped me stay “me” as I’ve gone through it. Listed in increasing or of importance:

I will not wear a "logo’d" polo shirt, or for that matter any logo’d apparel, as I work. Polo shirts make me look like a teenage boy and quite honestly, high tech logos really aren’t my style. I will also not wear a sweater set or sensible shoes to work. If I am not meeting with a customer, I wear jeans. Period. Even better when paired with 4 inch heels. And don’t even get me started on the “lanyard” concept…
I will not apologize or make an excuse why I have spent most of my career in the Enterprise / Data Center domain. If you think consumer, tablets, phones, etc are “sexier”…well you obviously haven’t really looked into the big cloud service providers, rack disaggregation or what is powering the voice/gesture recognition technologies on those sexy phone and clients. Game changing tech trends that are rooted in the data center
I choose my manager as much as I choose my job. During my 19 years I have had some exceptional managers. The commonality across them – I honestly believe they care as much about me as they care about my output. They’ve had my back and in return I will always have theirs. They also build their organizations with that culture – respect, no tolerance for politics, investment in career development and support for work/life balance. I am now a working mom with elementary age twins and I travel too much. Did I also mention the very long commute? That means I only have time to work on real work. I couldn’t do what I do at home and at work if I had to worry about watching my back or battling bureaucracy.

So although my life has dramatically changed over 19 years, and Intel with it, I am glad to say that I have landed in a highly relevant, fast paced, super cool technology space where I feel supported to be ME – data center junkie, lover of new tech, Intel loyalist and devoted mom who never wants to miss a talent show or soccer game.

Question to you –

How have you struck balance and kept interest in your career over one, two or more decades in the data center business?

How to Optimize Hadoop Performance on Intel Architecture

webadmin@intel.com — Thu, 06 Jun 2013 00:11:20 GMT

If you are dealing with data analytics, you have probably heard about or are using Apache Hadoop*. By leveraging the compute power of multiple nodes in a cluster, Hadoop can analyze practically unlimited volumes of data—both structured and unstructured—with lightning-fast speed.

But did you know you can optimize Hadoop to deliver even better performance on Intel architecture? The key is to tune the underlying Java so that it takes advantage of capabilities in Intel hardware. When you do that, you can expect to see up to 70 percent faster performance on Hadoop sort operations. Keep reading to learn how.

Understanding Hadoop’s Java Foundation

A Hadoop cluster comprises a master node and multiple slave nodes, where data is stored and where analytics processing occurs. An incoming analytics request invokes several Java* services that enable efficient replication and large-scale analytics across nodes in a cluster.

Processing begins with two Java services on the master node. These two services then communicate with additional Java services on the slave node where the data are stored—see the image below for a simple view of what’s going on.

When MapReduce* completes all the assigned analytics tasks, it returns the results to the master node, which compiles results from multiple nodes and returns them to the requester.

Important note: Hadoop spawns a new Java Virtual Machine* (JVM) for each MapReduce function on each slave node. This means that a large analytics job can result in the creation of thousands of individual JVMs. Because Hadoop does not share memory resources across nodes, each JVM and Java service must perform optimally. Reduced performance on any single node can hamper data analytics performance across the cluster.

Tune Java to Optimize Hadoop Performance

Given this foundation of Java services, it is easy to see that optimizing Java for Intel® architecture can deliver significant Hadoop performance enhancements. Optimizations are built into Java and Intel architecture, which make these improvements easy to achieve. When Intel releases a new microarchitecture and platform, Intel and Oracle software engineers work together to tune the JVM to take advantage of the new hardware advances. These optimizations can provide faster Hadoop performance on each Intel-based node as well as on the entire cluster.

How much faster? Up to 50 percent faster on a TeraSort* benchmark test and 70 percent faster on the Hadoop Sort benchmark. Read the white paper, Optimizing Java and Apache Hadoop for Intel Architecture, to see the details.

Since 2007, Oracle and Intel have improved Java performance up to 14 times by tying specific Java optimizations to advancements in the underlying hardware. Read the paper to see a complete list of optimizations, but here’s a taste of some that are important to Hadoop performance:

Fast CRC increases file checksum and compression/decompression checksum, which increases Hadoop network and file system performance.

Large-page usage increases performance of large analytics jobs.

Intel® Advanced Vector Extensions improve the performance of MapReduce operations that contain array and string manipulation, such as sub-string or character searches; also improves integer and floating point calculations.

Intel® Integrated Performance Primitives (Intel® IPP) compression increases compression performance, which reduces network and disk input/output (I/O) across the Hadoop cluster.

If you’re using Hadoop, or analytics on large data sets are part of your job, then check out the white paper for a full description of the Java optimizations that are available to you to enhance Hadoop performance on Intel architecture.

Follow me @TimIntel.

Go Ahead and Virtualize IBM InfoSphere* Information Server

webadmin@intel.com — Thu, 23 May 2013 15:42:13 GMT

If you want to virtualize IBM InfoSphere* but have put it off, I have some new information that might give you the bump you need to move forward: How does 98 percent throughput sound?

In a nutshell: extensive testing shows that InfoSphere can perform very well when virtualized. How well? We achieved between 90 and 98 percent of the throughput that we typically see in a physical environment, with an overhead tax of only about 10 percent on I/O-intensive workloads.

You can get all of the configuration and results details in the white paper about InfoSphere virtualization here, but I’ll hit the highlights for you in this post.

While virtualizing InfoSphere Information Server is cool, here’s the icing on the cake: virtualization with VMware vSphere* is now supported as an IBM PureApplication* pattern. “Patterns” are a growing ecosystem of software stacks for IBM PureSystems* running on Intel® Xeon® processors. Hence, adding the leading VMware vSphere 5.1 stack indeed broadens the use of these rich appliance systems versus other industry systems. PureSystems are innovative, quick-to-uptime, and run industry-standard software stacks.

Testing InfoSphere Virtualization

IBM, VMware, and Intel teamed up recently to see how IBM InfoSphere performs when it’s virtualized. Specifically, we looked at the runtime performance of:

IBM InfoSphere DataStage* 8.7
Running on VMware vSphere 5.0
On a server powered by the Intel Xeon processor E7 processor family

The tests found that InfoSphere DataStage scaled smoothly as we cranked up the number of virtual CPUs (vCPUs), while clocking throughput at up to 98 percent of that found in a physical environment.

More good news: we saw only a slight performance difference when using VMFS (Virtual Machine File System) versus RDM (Raw Device Mapping) data stores. This means that you can reap the benefits of VMFS for storage provisioning with virtualized workloads without concern over performance.

Now let’s get down into some of the details so you can put these results in context.

InfoSphere Virtualization Test Configuration

Server

We used an IBM* System x3850 X5 with a network-attached IBM System Storage* DS5300:

Four socket system
40 physical cores
Intel Xeon E7-8870 processors
Configured as “optimized for performance”

To get a good comparison between physical and virtual environments, we controlled RAM and processor availability to the native environment so that we could match the virtual environment as closely as possible. We enabled all of the virtualization-related options, except for Intel® Hyper-Threading Technology (Intel® HT Technology), which we disabled to simplify the comparison.

IBM InfoSphere

We installed all InfoSphere DataStage components on one physical server in the native environment and on one virtual machine in the virtualized environment:

IBM WebSphere* 8.1 Application Server (WAS)
XMeta repository
DataStage engine

We found that DataStage engine running in a single virtual machine had higher throughput. The specific reasons for this result weren’t clear, but we’re hopeful that further testing will provide more detail.

ETL Workload Virtualization Test Results

As expected, throughput in both environments increased as the number of processor cores increased, with performance in the two environments varying only between 2 and 10 percent. The overhead for the virtual environment was only 10 percent for all tested configurations.

Host Server Memory-Management Test Results

We also wanted to see what would happen with DataStage performance when overcommitting the host server. We measured a 34 percent drop in throughput when the system was 100 percent committed, and throughput continued to drop as the host processors were further committed.

Working with VMware engineers, we determined that this drop was because the eight vCPUs were not mapping neatly onto the 10-core, Non-Uniform Memory Access (NUMA)–node design of the Intel Xeon processor E7 family’s microarchitecture.

To work around this issue, you could use a five vCPU configuration instead of an eight vCPU configuration. Five vCPUs would map well to the 10-core NUMA nodes. Another workaround would be to turn off NUMA scheduling in BIOS or VMware vSphere, which would allow all of the CPU cores to be used, though you would see a lag in memory performance. The lesson? Understand NUMA configuration to optimize VM performance.

We repeated this over-commitment test with the VMware vSphere “reserve CPU” option for the DataStage guest set to maximum, and the result showed minimal performance impact. However, this move can potentially impact the performance of the other non-reserved virtual machines running on the same host.

If your system is overcommitted and you’re not seeing the DataStage runtime performance you want, the best option would be to increase the host system capacity or to move the VM to another host.

Go Ahead and Virtualize InfoSphere

Our tests showed InfoSphere Information Server runtime performed very well in a virtualized environment hosted on a platform powered by the Intel Xeon processor E7 family and using VMware vSphere 5.0. So if you’ve wanted to virtualize your InfoSphere applications but were concerned about performance, now might be the time.

Take a look at the white paper to see the test configuration, procedure, and results. And follow me on Twitter, @TimIntel, to get more useful InfoSphere and DB2* tidbits. You can also get the latest news and technology updates at the joint Intel and IBM DB2 website.

S5000XVN and >16gb RAM

webadmin@intel.com — Fri, 03 Aug 2012 22:42:49 GMT

Got an S5000XVN running dual 2.83ghz E5440's and currently 16gb of Hynix ECC 667, 4x 4gb PC2 5300. Damn thing is a beast!

I got 4 more identical 4gb memory sticks and when I put them in, i get the solid amber light of death at POST. I'm running in RAS Disabled and the sticks are A1 A2 and B1 B2 (the blue ones)

If I put any more in it, even 1gb sticks, it hangs. I know these do 32gb and I would like to double up to host a website. In the documentation it states that it only supports 16gb as mirrored but im not running them mirrored, at least I don't think I am.

Also the tech documentation was kind of vague on whether or not it supports X-series cpus. Would be nice to have 16 logical processors (as if 8 wasn't enough)

Server Storage Caching Considerations

webadmin@intel.com — Wed, 22 May 2013 14:00:16 GMT

Caching is storage. Here’s how Wikipedia defines caching:

“… .a cache (pron.: /ˈ k æ ʃ / KASH) is a component that transparently stores data so that future requests for that data can be served faster.”

With the data explosion and consumers creating content and wanting to access it immediately, caching is becoming more and more important.

My colleague, Susan Bobholz is a Marketing Director in Intel’s Data Center Software Division and talks about the considerations for server storage caching.

Right now, one of the hottest storage topics is storage caching. It seems hardly a week goes by without some type of caching software showing up in the press. I thought I’d spend a bit of time talking about this trend and provide some things to think about when choosing caching software.

In many datacenters, the hottest, most frequently accessed data is stored on 15K serial attached SCSI (SAS) hard drives. But those hard drives can become a bottleneck because they are mechanical devices with moving parts. They simply can’t move fastest enough to keep up with some application demands. One solution to resolve this is to replace all those 15K SAS hard drives with Solid State Drives (SSDs) but this can be an expensive undertaking. The beauty of storage caching is that it protects your investment in hard drives, because your application performance is improved without replacing all those hard drives with SSDs.

To be as simple as possible, storage caching allows the hottest, most important data to be stored in a SSD instead of hard drives, allowing that data to be accessed significantly faster. Often caching is implemented as a server application, but sometimes it’s actually part of the firmware on a RAID HBA

So you’ve decided you want to implement storage caching. There are several caching options out there. Other than cost, what are some key questions to consider about when deciding which to use?

Where does your cache physically reside?

As mentioned above, some caching solutions are integrated into a RAID HBA. This means that the Cache SSD must be attached to the RAID HBA itself and only data on hard drives connected to that RAID HBA can be accelerated.

Other caching solutions allow the Cache SSD to be anywhere inside the server itself. This provides additional flexibility as the data being cached can be anywhere on the server - behind a RAID HBA, behind a SAS HBA or even attached to the chipset SATA ports.

In addition to being able to have the Cache SSD inside the server, some caching solutions allow the Cache SSD to be outside the server, in a SAN or NAS. This is important in virtualized servers as this allows virtual machine migration to occur automatically. The cache remains active while the virtual machine moves from host to host.

Consider where you want the cache SSD to connect to your server when choosing a caching solution.

Which OSes are supported?

Think about what OSes exist in your datacenter. Windows? Linux? Virtualized OSes such as VMware ESX or Xen? Think about whether it’s important to have a common caching solution from one vendor across all these environments. Not all caching solutions support all these OSes.

Being able to choose what goes into the cache

This may sound unimportant, but imagine having an SLA with a customer that requires you to deliver the lowest latency to the data associated with that application. What if you could guarantee that specific data was always in the cache, ready to be accessed? Several caching solutions available today offer proprietary ways to pin data into the cache. This is becoming so important that standards bodies such as ANSI T10 are looking into ways to standardize ways to determine whether data should be kept into a cache at all times.

Read caching or write caching?

Look at the applications you want to accelerate. Do they mostly read data from hard drives or do they mostly write data? Or is it a mix? Some caching solutions are better are accelerate reads, others are better at writes. Choose a caching solution that meets the needs of the applications you want to accelerate.

Caching algorithms aren’t all the same

We all learned about Least Recently Used caching in school. Just as the name implies when the cache is full but new data needs to be added to the cache, the data that has been sitting in the cache the longest without being use will be evicted to make room for the new data. This can be an effective algorithm and is very common. But some caching solutions add intelligence to the caching algorithm and are able to decide to keep specific most popular/active data in the cache longer, protecting it from being evicted by more recent, but less popular data. This reduces the probability that important data is evicted from the cache, improving overall application performance.

So, these are just some of the areas to consider when choosing a storage caching solution. What is important to you when you choose a caching solution? Let me know!

Full disclosure: Intel has its own caching solution: Intel® Cache Acceleration Software that works with Intel® Datacenter SSDs. We think it’s pretty cool. A 30 day trial is available on intel.com.

Susan Bobholz is a Marketing Director in Intel’s Datacenter Software Division. She’s been with Intel for 20 years, doing everything from software development to initiative management to product marketing, focused on storage technologies and products. Prior to joining Intel, Susan developed software at Siemens Medical Labs and firmware for Motorola cell phones. She graduated from the University of Wisconsin with a BS in Electrical and Computer Engineering. She holds 3 patents.

At TDWI: Finding a middle way between Hadoop and relational data warehousing

webadmin@intel.com — Fri, 17 May 2013 22:23:37 GMT

If you’re thinking big data analytics will solve all your BI challenges, you may be looking at it wrong…

I realized this when I was in Chicago the first week of May, attending The Data Warehousing Institute (TDWI) conference, where the theme was “Preparing for the Practical Realities of Big Data.”

There are high expectations around big data at the moment. Many people in marketing, product development, and analytics teams can’t wait to get their hands on big data intelligence to better understand and target audiences.

However, TDWI isn’t their show. At TDWI, the focus instead is on traditional database administrators, data analysts, and data scientists, and it’s a very technical conference firmly based on OLAP and OLTP analytics and hands-on issues of data warehousing.

With this in mind, I attended the keynote address by Ken Rudin, director of Analytics at Facebook—a leader in cool, cutting-edge big data processing and analysis addressing a conference of (what some would consider) old-school DBAs. The message from Rudin, who has held senior leadership positions at Zynga, Salesforce.com, and Oracle, was fascinating: Don’t get caught in the tyranny of either/or when it comes to data analysis—businesses need both traditional relational database processing and Hadoop*-based big data analysis.

@krudin said that Facebook started out relying almost solely on Hadoop and big data when the social media giant first launched, but now is increasingly incorporating OLAP and OLTP processes into its analytics. Hadoop is best at exploring huge data sets—putting all the data into one system to discover patterns. Traditional relational data analytics is best at business, looking at focused data to derive metrics and more actionable, granular analysis. Both are valuable technologies: which one is best depends on what kind of impact you are looking for.

So the question is not, how do you get from old-school to cutting edge as soon as possible. Rather, ask which technology is right to generate impact out of data.

This was a conclusion that appealed to many at TDWI, as several people I spoke to registered a bit of skepticism about the value of Hadoop as an engine for analytics. For them, the main attraction of Hadoop is its potential to act as a backend data storage mechanism, where unstructured data can be warehoused.

Then the question becomes: What’s the best way to integrate data stored in Hadoop into a traditional OLAP or OLTP infrastructure for processing? The answer is just around the corner.

At the SAP booth, I presented a discussion of the newly released joint solution from SAP and Intel that has optimized Intel^® Distribution of Apache Hadoop* software for the SAP HANA* in-memory database. Using SAP Smart Data Access* (watch for availability in coming weeks), this big data solution is able to leverage all types of data for processing in analytical applications.

And since it’s built on Intel^® architecture, and leverages the full power of Intel^® Xeon^® E7 processors, HANA has hardware-enhanced performance and security built in, with solid-state drives and cache acceleration for blazing speed and stability. Watch this interview from TDWI to learn more about SAP HANA and how the database helps address big data challenges.

If you’re looking for the technology to get the most impact out of analytics, look no further.

Follow Tim and Twitter @TimIntel and the SAP analytics team at @SAPAnalytics.

Additionally, @TDWI has some very interesting DW feeds.