Questions about memory controllers / maximum bandwidth for CPUS

[idata]

I actually have two questions before i begin. First is the interaction of the memory controller versus the motherboard variant for memory interaction. My second question is more or less a spoiler request. But when is the Broadwell/Skylake LGA 2011-V3 variant going to be released? I am looking to upgrade soon from my current rig. Core 2 Quad Q6600 CPU based so yea.....

From what i have been reading, using the Haswell E 5820k as an example. the memory controller on the CPU has a maximum memory clock of 2133 MHZ with quad channel support. Now my question is, if you have an X99 Board using DDR4 Dimms in excess of 2133, what happens to the bandwidth rating? According to my calculations the Haswell E quad channeled at 2133MHZ has a theoretical max memory bandwidth of about 68 Gigs per cycle. That means per channel, you have about 17 Gigs times the number of channels you are running. So single channel is 17 gigs, quad equals 68 gigs. Now...

If you overclock the Dimms above 2133, what happens to the memory bandwidth? I ask this because Memory controllers on certain X99 boards can do 3200MHZ per channel (overclocked)... Will the bandwidth of the CPU still be capped to 68 Gigs? Or does the memory controller of the motherboard take over? I do realize that the higher you clock the Dimms, the higher you need to set latency timings as well as voltage (CAS and RAS) to have it run synchronous to the CPU. So will overclocked Dimms exceed the processor's max bandwidth ratings? say 3200MHZ x8 bits, would i get 25.6 Gigs memory bandwidth per channel on overclocked Dimms? up to a max of 100 Gigs of memory bandwidth quad channeled on a Haswell E 5820k?

Also, assuming the Broadwell/Skylake E series use that LGA 2011 socket type on high end gaming and server based application, when will they becoming out? I am waiting very patiently because i don't want to pre-purchase any hardware only to find out the pin density is going to change again. I speculated Q1 of 2016, but given rumors that are out there, its tough to tell. Spoilers plz anyone ? =)

If i can hear the explanation directly from one of the Engineers that would be perfect. Feel free to hesitate on simplicity. I want to know exactly how everything works. Complex terminology is mandatory. i wouldn't even mind trying to find the engineering blueprints if i could.... I am an IT guy who just wants to build an ultra high end system to get the most out of it. That is curbing physical bottlenecks between the buses on the motherboard and interactions between CPU and chip sets.

If something is unknown to me, i will research it. Term and conceptual wise. This post is a little TLDR but thank you all for your patience.

[idata]

OK it's been a week, i know its TLDR so ill rephrase the question.

What does the Haswell 5820, or 5930k CPU do when you run overclocked DDR4 2400 or greater DIMMS?

Mind you the internal memory controller for Haswell only supports a max clock of 2133. What happens?

Does the motherboards memory controller take over? Or is additional latency configured to run 3200mhz? Will the bandwidth be capped at 68 gigs quad channeled or will it go up to 100gigs like i predict it would?

This question would determine the ram i am going to get for the Broadwell E So any engineering explanations would be greatly appreciated.

Anyone???

[Allan_J_Intel1]

I would recommend to wait for a community member that may have experienced same issue, please bear in mind that Intel does not provide information for overclocking, in other words, if the components are running out of specifications would be considered out of scope of support.

Allan.

[idata]

Indeed, the reason i ask is because the internal memory controllers say one specification max. While a motherboard, that may have an external memory controller, independent of the CPU from a bus pov, can list an OC frequency outside of the processors internal memory controller range.

This doesn't mean that the modules will not work, however it may do more I/O cycles. My question is what exactly will the CPU do in the event that the motherboard runs DIMMS outside of the CPU'S processing limitation?

I am guessing higher CAS and RAS latencies keep the dimms synchronous to the processor. To avoid memory collisions. But will you achieve greater bandwidth than 68 gigs on 2133? say using 2400 for a max bandwidth of 76 gigs quad channeled? Or will the bandwidth just be capped at 68 gigs all together regardless of the frequency clocked, 2400 or greater?

Allan, has Intel done any lab work on testing these? My question is actually more of a physical engineering from a knowledge pov than a problem. I am creating a high powered build configuration for a future PC. My theory is since FrontSideBus no longer exists, i think DMI 2.0's frequency will still control the North Bridge's operating frequency via DDR4's memory clock.

What i am trying to achieve when i purchase hardware is to avoid physical bottleneck limitations across each and every individual Bus. That is, if the CPU can only do a clock of 2133 max, don't buy 2400 or greater dimms. Or From a PCIE pov, if I plan on doing SLI don't buy a cpu lower than the 5930k. That is 28 PCIE lanes vs 40 for dual x16 mode.

[idata]

I forgot to mention i have no intentions of overclocking. Because to get the best processing speed safely, you need to buy larger more densely packed cpus. Since overclocking requires a voltage bump to increase operation frequency, you can probably get a 10-20% faster performance improvement. However at the end of the day, the physical bus, or medium you are working on will be the hindrance. My rule of thumb though is to never OC, since it requires more power while simultaneously generating more heat.

I like to invest in ultra high quality hardware for my gains. Providing, i can create a custom configuration script to use it from a software point of view.

[AJana1]

Hello GreeneMan,

Well, in your words,

While a motherboard, that may have an external memory controller, independent of the CPU from a bus pov, can list an OC frequency outside of the processors internal memory controller range.

this external memory controller does not exist in mainboards designed for current Intel processors. As to the overclock, the integrated memory controller has its own clock generator. Overclocking it would not influence other components. But just like what allan_intel said, "if the components are running out of specifications would be considered out of scope of support." Overclocking the system is not always the safe thing, it might potentially damage your processors permanently, if that happens, your issue of processor replacement would be rejected. I suggest you contact with the motherboard manufacturers about the detail information on overclocking before customising your new computer. Good luck!

Best Regards,

Aaron Janagewen

[idata]

Ah. I figure one would exist. one thing i left out was that the memory kits i was referring to are factory over clocked. If an external memory controller doesn't exist on the motherboard, what is it that determines the motherboards support to run 2600, 2800 or 3200mhz dimms? Just the firmware that the BIOS uses?

You are right in that sense, because older Intel chips did not have an on die memory controller...

So i figure something on the motherboard must exist that will allow either manual overclocking, or factory based over clocked DIMMS to run with the Haswell E's internal controller. I am using this tech example for my question because i had planned on building a broadwell based system using DDR2400 Dimms. If factory overclocked dimms work in the sense that it can increase data transfers i would go with it. But if the CPU is still limited by its max internal controller clock speed than i wouldn't bother. That is 68 gigs under DDR4 2133, or 76 gigs under DDR4 2400.

In terms of actual over clocking on ram. I could by a 2133 kit look up the latency timings and manually do it myself. I have no intent on doing so, because at the end of the day i still don't know if the cpu can and would utilize that increase in frequency, tis what i am trying to find out. I know at a minimum i am going by the max supported internal clock of 2133, or 2400 if Broadwell releases like i think it will.

[idata]

All though some of my hardware knowledge is dated more towards 2008, not referencing specs from newer products. The only thing i know of that gets a direct benefit to higher clocked memory transfer rates is the north bridge. QPI and DMI i do need to research and see the main difference. The board i am getting is DMI based for the internal control hub from a bus pov. So direct CPU access to RAM would benefit in quicker read/writes as well as processing and output. The ram clock itself in theory should not affect the south bridge. Though internal processing can.

Ill probably just go with the internal controllers max clock support. But if factory clocked memory kits in excess of 2400mhz benefit my chip with little risk in circuit burn out. I would love to capitalize on it. I just want to know if the maximum memory bandwidth listed is the hard limit, or soft limit.

The patterns i have seen is, the higher you clock the ram, the higher you need to set both CAS and RAS latencies as well as voltage. Which is actually bad because it creates delay between memory I/O even if its on the scale of 1 billionth of a second. 1/1,000,000,000.

So i guess ill just have to wait and see what people do with the memory profiles.

[AJana1]

Hello GreeneMan,

Thank you for your feedbacks. In your words,

Ah. I figure one would exist. one thing i left out was that the memory kits i was referring to are factory over clocked. If an external memory controller doesn't exist on the motherboard, what is it that determines the motherboards support to run 2600, 2800 or 3200mhz dimms? Just the firmware that the BIOS uses?

You are right in that sense, because older Intel chips did not have an on die memory controller...

but the external memory controller is not likely to be provided by the chipset and/or mobo manufacturers for Intel processors. You know something like SPD and/or XMP, well, this time just treat them as the ID (identification) card of memory module. When cold boot is starting, the firmware codes read those data to make appropriate configurations for memory controller. If a processor just only support DDR4 2133, but you have purchased a DDR4 2600 memory module, it will work under the DDR4 2133 specification if your mobo does not support memory overclock or someone else has already disabled such overlock related functions before. This time if you enable the overclock function by yourself, the mobo would make your memory module work under DDR4 2600 specification. In this situation, the thing overclocked is not your memory module but the memory controller integrated into processor. There is no harm towards your memory module, but potentially towards the processor.

QPI is different from DMI in nature. DMI is a PCI-e protocol based bus connecting between North Bridge and South Bridge chipsets, later processors integrates the entire North Bridge logic into its processor, exposing itself towards the external world with DMI connection. It is very rare to find such an external memory controller implemented onto PCI/PCI-e bus. QPI was initially designed to succeed the traditional Front Side Bus, someone might also treat it as a serialised enhanced version of Front Side Bus. The most common example is the first generation of Core i5 processor, two dies (processor cores and northbridge) are connected with QPI link on the CPU chip. And the external (speaking to processor cores) memory controller is really implemented on the external logic based on QPI bus. Today, QPI is only provided for the interconnection among processors, seldom used to connect to the chipsets, even if you are using XEON or future Itanium processors.

I wish this reply could answer most of your questions.

Best Regards,

Aaron Janagewen

[idata]

I think you answered my question. Thanks for the tech reply as well for QPI and DMI. From what i am interpreting, the CPU's memory controller can actually exceed its recommend clock cycle. This is what i wanted to know =). I was wondering what the CPU would do.

Janagewen wrote:

If a processor just only support DDR4 2133, but you have purchased a DDR4 2600 memory module, it will work under the DDR4 2133 specification if your mobo does not support memory overclock or someone else has already disabled such overlock related functions before. This time if you enable the overclock function by yourself, the mobo would make your memory module work under DDR4 2600 specification. In this situation, the thing overclocked is not your memory module but the memory controller integrated into processor. There is no harm towards your memory module, but potentially towards the processor.

I have heard of XMP, i never really looked into how it works. That is if its a config you can upload or if its just a data sheet. I was under the impression that they were just memory configuration profiles optimized for speed and stability. But this statement, this actually changes how i look at ram which is what i wanted. I was under the impression if you buy a memory kit, it was clocked to a specified frequency and latency timing.

Now I did read a while back that some motherboard bios settings can throttle the DIMMS if they don't support a specific clock rate, which goes back to the context of this quote. I thought the clock was set on the DIMM itself and throttled by the CPU if it couldn't handle it. I knew that instances like that can happen, i just never understood what causes it and why up until now.

But if it is in fact the memory controller, which nowadays are integrated in the CPU that changes but not the DIMM. Than this means i now know exactly what i want to buy.

But out of curiosity thought a DDR4 3200 kit just has the XMP profile settings and a better heat spreader for the clock? Otherwise it should in theory be the same from an engineering pov as a DDR4 2133 kit?

[AMohr1]

You should buy a kit of high quality DDR4-2133 RAM.

Like: Kingston Hyper X DDR4-2133 Savage 4x8GB

If you install more than 4 modules (8 modules), the clock frequency on the RAM channel do decrease to 1333MHz.

In server case, REG buffers do decrease load capacity.

[idata]

Interesting... Is that because the slots are parallel in each memory channel? 4 channels 8 slots. I never would have thought the clock would diminish if you use the 2nd slot on the channel for additional buffering space. Each channel should have 128 bits in total.

64 bit DIMM slots.

[KTran10]

It is about memory load, 2 DIMM per channel will drag it down by 266MHz, it also depends on motherboard too. For your memory bandwidth

if it run @2133 then Max is 68G for 4 channel

The formula for bandwidth = DIMM frequency * 68 / 2133.

If you try to buy DIMM 3200 to overclock, and if DDR ratio can change from 16(2133) to 24(3200) then your bandwidth will follow the formula above. However, I doubt CPU allow to change DDR ratio.

Overclock by busclock, which is not very stabled because other components not work such as SATA

[idata]

hmm interesting formula. From the calculation i got, 3200 mhz should be about 100 gigs rounded quad channeled using that. I thought the 68 and 2133 for the memory controlling stand point would change though. Is that a constant regardless of the clock you choose to run? be it 2133 or 3200? OCing will increase theoretical bandwidth, but latency timings will effect that to so just don't know what to go off of.

The way i used to calculate ram, bandwidth = Dimm frequency * 8bits to get the bandwidth in GB/s. Then GB/s * (X) where X is number of active memory channels in your system. (1-4) as of this point in time.

The DDR ratio i actually need to research this, its new for me. Is it a pipe lining stage for the CPU's memory controller? Since the base clock on ram has a multiplier? The CPU clock also has those as well.

For my Q6600 quad, i think i have a multiplier of 8 and a base clock of 300. for a total of 2.4 GHZ. Or it was 12 with a base clock of 200, cant recall now. I have it at stock speed. But the same principle applies to Dram none the less?

[AJana1]

Hello GreeneMan,

I am sorry to delete your replies by mistakes, I am out of here! Sorry again!

Best Regards,

Aaron Janagewen

[idata]

All good dude. Just acknowledging that i like everyone's school of thought on this thread. As well as the information i am reading to make more intelligent decisions on my planned PC build. Be it the information being accurate or not.

Everything here is all theoretical based. When calculating bandwidth for best possible performance gain. What i wanted to know is given the changes that are occurring to the clock rates of DRAM. is the formula

Dimm frequency * 8bits = Bandwidth GB/s That is for an old school ex.

DDR2 800 x 8bits= 6.4 GB/s per Dimm slot per channel. since i am using 2 memory channels ATM. should be 12.8 gigs bandwidth. or 12,800 megabytes of I/O.

That was my previous post. Accidents can and occasionally do happen. With that said stick around. Any talk on bus technologies, operating protocols and how they operate are important for hardware enthusiasts. The talk from my pov is much appreciated no matter how complex it gets. Discussing physical and logical limitations too about a product are the up most importance as well. So good practices knowledge and the likes are all welcome.

At this point in time though, i am still itching for that Broadwell E series chip.