1 of 1 people found this helpful
Ok. I just tried a "batch process" of RAW files from my Canon 20d from Digital Photo Professional to Maximum Quality JPG's in Photoshop. I exported 80 Raw files totalling 811MB to Maximum quality JPGs which were then opened in Photoshop. I'm using a 2Ghz Core2Duo Laptop with 2gb of ram and an internal 100gb 7200 rpm hard drive. It took 23 minutes to convert the Raw files to JPG and open all of them in Photoshop.
I also did the same process but instead of using Digital Photo Professional, I used Photoshop CS3's image processor to convert the RAW files to JPG. That conversion only took 7 minutes. I then opened all the JPG's in Photoshop and it took 9 minutes to open all of them. So, converting this way was significantly faster for me.
I think the most important thing is CPU and RAM for image conversion and opening in Photoshop. I also have a desktop computer with an Intel Q6600 Quad Core processor and 4gb of ram, and a 640gb 7200 rpm hard drive. For instance, opening the same 80 JPG files all at once on the desktop pc with a faster CPU and more RAM does it in 6 minutes instead of 9. So, firstly I would see if your CPU is as fast as you can afford and there should be sufficient RAM (4gb or more) so that the system doesn't need to resort to using the pagefile.
Also, I think it's always a good idea to convert and save to a separate hard drive. For instance, using one hard drive to read all the RAW images, and using another hard drive to write all the JPG's should prove to be faster. The reasoning behind this is that a normal hard drive will waste time if it has to read and write at the same time. So, if the hard drive with the RAW files is just reading and another hard drive is just writing then the performance should be better.
My understanding of the new Intel 160GB G2 SSD is that the biggest benefit is using it as a system/application drive as it is the fastest in 4k random read/write, which is what I plan on using it for. Booting up, loading programs, thumbnails of images, loading image galleries will be much faster than a standard 7200rpm hard drive. However, needing to work with gigabytes of RAW and JPG files might better be served with standard hard drives short stroked in a RAID 0 setup.
My instinct is telling me that the CPU/RAM is what is slowing the system down. Giving the CPU faster access to the data from the hard drive can never hurt and the Intel SSD will give you that for sure, but it's the CPU that needs to process the image, and that processing is where I think the bottleneck is happening. If I am wrong and in fact the hard drives are the bottleneck, then yes the SSD or a short stroked RAID 0 array would serve to speed up the process greatly.
If anyone else could chime in on this I'd love a second opinion.
Thanks very much for the well-thought out reply to my concerns. Actually the interesting thing is I am using my desktop which has similar specs to what you listed - a Q6600 quad core and 4GB of RAM.
Normally I've exported the photos to secondary drives before but the main one I'm using is a 1 TB 7,200 RPM Western Digital which for the most part is fast enough when viewing and editing. I don't have any other drives currently available where I can have enough space free at the moment for both the RAWs and Jpegs but I could try exporting the jpegs as you said onto another physical drive to see if that speeds it up. If I take my main drive (WD Raptor 150 GB, 10,000 RPM) and export to that, I can certainly give you the times once I know.
Maybe it's simply a CPU bottleneck as you said. I can process images from my 10D (similar to your 20D) in a relatively much faster time than the images from my 5D Mark II. It's a speed-up in the processing of the 5D II files that I'm looking for, and so my question regarding Intel SSDs.
Let me repeat your test then here if I may. I will use the same # of 10D files (RAWs from that are about 6MB and I thought the 20D's were about 8MB each)... I can export them to the same drive and then to a faster one and see the time difference as well as opening them all in Photoshop like you did.
Thanks again for the help as it is very much appreciated.
Ok between taking care of a few things here, I ran some numbers for you to match with your test data. Here's a simple copy/paste of what I put in a document.
Batch from RAW to Jpeg at max quality
Export 80 RAWs (at 811 MB for his 20D)
23 mins to convert as well as open in Photoshop on laptop
6 mins to open in Photoshop but not sure long it took to export on desktop.
Try Photoshop’s Image Processor?
Save on same drive and on different drive.
80 5D Mark II RAWs = 1.71 GB
WD Raptor = 10,000 RPM, 150 GB HDD
WD Black (WD250GB) = 7,200 RPM, 250 GB HDD
WD Caviar – 7,200 RPM, 1 TB HDD
Primary Drive = WD Raptor
Secondary Drive = WD Black
Primary Drive = WD Caviar
Secondary Drive = WD Black
Only the Raptor and WD Caviar have OS installed. The WD250GB Drive (Black) used to have an OS but became corrupted so won’t always boot. Running Windows 7 Professional 64-bit on the Raptor and Windows XP Home Professional 32-bit on the Caviar drive.
Photoshop exports faster with the image processing tool but upon entry with RAW capture does slightly change the image. Even changes image output with same color profile set to both programs. I tried to make sure the color profiles were the same on both drives, but for some reason Photoshop would literally import the RAW with different settings than how it looks for me in DPP. Thus though the tool is useful for fast processing, I'd rather retouch by hand via DPP and work with a true original - not what Photoshop "thinks" is a true original, when I then edit and export...
WD Raptor DPP Export RAW > Jpeg Same Drive = 16m 01sec
WD Raptor DPP Export RAW > Jpeg to WD250GB 7K-RPM Drive = 18m 35sec
WD Caviar DPP Export RAW > Jpeg Same Drive = 15m 45sec
WD Caviar DPP Export RAW > Jpeg to WD250GB 7K-RPM Drive = 15m 58sec
WD Raptor Opening 80 Jpegs in Photoshop CS4:
(Photos on Raptor) = 2m 49sec
(Photos on WD250GB) = 2m 52sec
WD Raptor Opening 80 Jpegs in Photoshop CS4:
(Photos on Caviar) = 4m 17sec
(Photos on WD250GB) = 4m 18sec
The biggest things I'm concluding from this are 1. The exporting speed is different because one drive is 10K RPM and one drive is 7,200 RPM. The amount of free space may offest the numbers slightly but I'm thinking not enough to really impact a result horribly. The Raptor is about 66% full and the Caviar is about 50% full (both fully defraged and running no excessive tray services and extra garbage at startup). 2. The opening time in PS is very similar because both disks can read quite fast and irregardless of which disk is being used for scratch, read times are far less of an issue.
This brings us down to wondering if say, a 15K RPM drive or SSD (faster write thoroughput) would improve export performance, or if I'm hitting a CPU bottleneck and it simply needs more horsepower per se to crank through the media faster.
Unfortuantely with this architecture though (Dell XPS 410 using a LGA 775 socket for my Intel Quad Q6600), I don't believe I can actually increase the processor type without having to jump to LGA 1366 (Nehelem) or basically replacing the board. This makes me more inclined at the moment to buy an SSD as opposed to changing out a lot of the components in my system due to funds available.
If you need me to repeat all the tests with 80 images from my 10D, I certainly can, but I think they'd all just be lower with obviously the same results.
Hopefully this is good enough for your review and thanks for the help Davem.
1 of 1 people found this helpful
Looks like a hard drive that could write faster may help. Although, in sequential write speeds the Intel X25M SSD's are not the fastest compared to the competition.
Maybe for this application a 7200 RPM RAID 0 array short stroked would give you the best bang for the buck. The Raptor is a great drive, although an X25M SSD would even be faster for those small file reads and writes especially for an OS. I just ordered a 160GB X25M for that purpose. I'm going to put the OS on the X25M and then copy a max of 4 gigs of RAW photos at a time to it. Then I'll export to JPG to a 7200 RPM Raid 0 array. Therefore I'll be able to read and write at over 200mb/s...Also reading and opening the 4Gigs of photos on the X25M SSD should be extremely fast....
My sequence will be:
4GB Compact Flash Card RAW files in media reader--->Intel X25M RAW File editing, then export JPG---->RAID 0 pair of 7200 RPM Hard drives then send everything--->External Backup
Thanks for the quick reply on the topic! Yeah that was more or less my thought too - I would use the X25-M to take batches of media (RAWs) to and work on them, then export them out either to the SSD or a secondary drive.
Though I would plan to write far beyond 4 GB a day (the typical wedding for me so far has put the file counts at 60+ GB per event), I figure the SSD will help with working on more events faster while the drives will fall in price through next year. Not saying they wouldn't last a long time still with my useage but I don't blame you for figuring out how much you can write per day due to the lifespan of the cells.
I don't know how long it'll take for you to get your drive in but if you could run say, the same exporting test of 80 of your shots, would you be able to reply with the times again?
I'm so excited to get one of these drives - at the same time though if it's not really going to help me accomplish what I need then why keep it (though I honestly don't see how given the speed comparing to a HDD). Even barely topping raptor speeds are fine in my book for now... I've only seen the drive here top anywhere nearing 70-80MB/sec when it's empty. Right now it's sitting more like 20-50MB/sec depending on task so again I don't think I can go wring. I'd just wish for an equivalent test with an SSD so I know how quickly I need to take the plunge, heh.
I am building a new machine, so as soon as I get all the parts and the assembly done, I'll let you know the speed results of the Intel SSD. Realistically, this could be in a week or so.
As your tests showed, there really isn't much difference in reading and writing to the same drive. Maybe doing a 4 drive 7200rpm Raid 0 array short stroked, would be the most cost effective and yield tremendous speed in read and write. This way, you could just dump the 60GB on the 4 drive RAID 0 array, and then edit and convert right on the array. For $200 you could be done, and as well with writing 60GB of data each day, you won't have the high SSD cost/wear/TRIM issues since you can't TRIM raided SSD's....
Thanks again for the prompt reply. I'll certainly await your rebuild and stats if you don't mind! It'll be interesting to know for myself (and many other I'm sure) as to how things compare to the SSD.
Though I'd love to build a raid array, my current case is a pre-built BTX form factor and I've long been out of drive bays to mount more than two drives at a time, let alone four! Though raiding is also fairly cheap, don't disks have to be the same size and, forgive me if I'm mistaken, but raid 0 will fail if one drive crashes, correct?
Granted with the support concerns, I'm not saying SSD technology is at a perfected point where it's infalible either, however you're right at least when it comes to probably being cheaper and still offering a huge performance boost.
I guess I'm just really set mentally on jumping to the SSD since I've never raided drives together before. It doesn't seem hard to do - I've just been wary of having drives fail and the loss of capacity a raid would give. Right now though since I do have the WD 1 TB drive, do you think I could always just pick up another one and raid those two? That way I still have tons of space and a performance gain without as much a headache as four drives?
I suppose at any rate though, it'll have to be something I'll think over and observe to see if the performance gain with either method justifies the expense. For an SSD I'm sure it will and likewise for a raid I don't see why it wouldn't help tremendously. I guess I'm still just unsure.
Thanks again for all your recommendations.
A raid 0 array will fail if one drive fails, but, since you would have all your data backed up or at least on the Compact Flash cards, the only thing you would really lose is time. If your 1TB drive failed you'd lose time too, so while it's something to consider, I don't think it's worth giving too much weight to.
You could just buy the same 1TB drive and raid 0 that drive, which would be even cheaper. For less than $100 or so, it's a no brainer. Then you could see the performance boost, and better judge what you need. You could use them separately too for extra data storage. At 60GB per wedding, it seems to make sense to have a second one.
We'll see how the SSD works and then revisit this....
In the meantime, you could also see if some other photogs have solved this. I sometimes check/use the forums at www.luminous-landscape.com or www.dpreview.com
All the Best!
Fair enough. Again I'm somewhat unfamiliar with raiding drives so I just thought a raid was inherently more prone to failure than a single drive. Guess it really doesn't matter though since the photos are indeed backed up as it is.
I will certainly have to consider the second drive raid option. That's a lot cheaper than an SSD and may suffice. I will still wait to hear for your research though if you don't mind. I'm still tempted by an SSD only because along with the photos, I do do a lot of graphic / web design, 3d rendering, video work and gaming - all aside from just taking photos. But hey, then again a raided drive would help all those too.
Certainly will look at the forums man, thanks. I had checked them awhile ago but since these drives are becoming more affordable, people may offer some insight on them now.
All sounds good.
I found this on the Adobe website. Thought you might enjoy looking through this: http://kb2.adobe.com/cps/404/kb404439.html
They talk about RAID and scratch disks, and even touched upon the ideas we were having in this thread about fast access to files.
Also, now that you mention Video/3d rendering, here's what Adobe recommends for Premier Pro: http://kb2.adobe.com/cps/405/kb405744.html
I'm starting to think that it might be more likely that giving Photoshop the fastest access to the files since they are no more than 20MB, is mostly a random read/write operation. Many sites describe sequential read/write as video files that are like 700MB...
Here's a review that touches on the Intel X25M random and sequential read compared to other drives: http://www.legitreviews.com/article/1022/8/
Here are more 50MB and 1000MB file tests on various SSD and raid 0 setups: http://www.hardforum.com/showthread.php?p=1034948741
Will be interesting to see. I just got my X25M today, it's so tiny and weighs less than my small flip open cell phone....
It would be great to really understand exactly what is happening to cause these performance differences in Photoshop...Is it disk access time, random read/write, Iops etc...
Hopefully we'll figure it out...
Wow that's tiny! Oh, also thanks for those links. Some of them are new to me whereas others I've already passed through when researching the drive and comparisons. Yeah I figured if I got the 160 GB SSD that I'd put all my PS files on it, 3ds Max and all the texture sets, other photo tools, and anything else needing the horsepower to crunch through for work.
Thanks again and I'll keep an eye posted as to how your testing and install goes. Good luck and enjoy your drive!
I asked our same questions an another forum and got a couple responses.
All the Best!
Thanks very much for looking on these forums about it too and to take up so much of your time. Needless to say I've learned a bit more about drive settings from the people there!
How about this then, I'll test the same 100 RAW / Jpeg test from my 10D then. Granted, 6 MP isn't exactly 10 MP like the Sony A700 like IdPlease is using but it's still a far cry from 21 MP of a 5D Mark II.
Also, first time color-recalibrating video from some rushed footage from a weekend event... The video clip was about 3 minutes long, and took 3 hours to reconvert (changed color temperature) on my standard WD Caviar drive here.
Thanks again for the help and will get you more tests. How's your SSD treating you so far?
Hopefully you're doing well with the drive. Sorry for bumping the thread but I was curious to know how your progress has been?