Friday, April 11, 2008

Work In Progress— The Ultimate Raid

I've been writing quite a bit about storage over the past couple of years going all the way back to this first roll your own raid article to a more recent review of an 8-drive array from Dulce Systems with more general storage articles (check all of the out on www.dv.com).

Currently waiting for publication is a review of the 3Ware Sidecar.

This time I wanted to build an ultimate raid. But just what does an editor need? Well, what I needed was a single workstation unit with lots of capacity, with redundancy and with the speed to handle capture and playback of 10 bit uncompressed HD. 2K capabilities would be a nice bonus but I really only see myself working in 2K just in technology demo rather than production at the moment.

I had built a Raid 1+0 based upon the CalDigit eSata card for my MacPro and a Sonnet card in my PowerMac G5, connecting to a 5 enclosure port-multiplier box I bought from CoolDrives.com.

The CalDigit and Sonnet cards are software based, utilizing port multiplier chips from Silicon Image but with the raid itself configured and managed through the OS (in my case, MacOS X) and SoftRaid, a superb management utility.

I wanted to go the hardware route this time, so I chose to create a solution based upon Highpoint Technology's RocketRaid 3522 card (note-- this is the Mac page, but it works as well with XP and Vista-based PC's) and and the Enhance Technology E8 MS enclosure.

The Highpoint RocketRaid card is hardware based with an onboard Intel IOP 81341 processor with 2 external mini-SAS ports, each capable of supporting 4 drives in Raid 0, 1, 3, 5, 6, 10, 50 and JBOD. It is a PCIe 8x card which works as well in a 4x PCIe slot and includes browser-based raid management. It will work with any enclosure and with SAS or SATA drives.

An added bonus with the Highpoint card is an ethernet port which allows remote access to the card to configure the raid or even initiate rebuilding remotely in the event of failure.

Hardware raid means, in brief, that all processing is performed by the card's processor taking load off the computer's processors. Hardware raid level 5 represents a good compromise between speed, capacity and redundancy. Effectively, parity is spread across all drives and the raid capacity is reduced by the size of one drive. Raid 5 allows for the failure of one drive, while raid 6 allows for two drive failures.

The Enhance E8 MS enclosure allows easy mounting and removal of drives and has two miniSAS connectors. It will accept either SAS or SATA I/II drives.

I have been testing with 8 Seagate 7200.11 1 tb drives as well as with 8 1 tb Samsung F1 drives. Stay tuned for the detailed comparison of these drives but catch a sneak preview below.

Preliminary testing is very favorable with some amazing speeds, 3-4 layers of 10 bit uncompressed HD without rendering and seemless playback of 7 layers of 10 bit uncompressed. This can be attributed not just to the quantity of fast drives but in great measure to the Highpoint card.

One problem so far-- my MacPro (early 2008) dual 3.2 will not recognize the RocketRaid card in Slot 2 (the 16x slot). In fact, it will not recognize any card in Slot 2 other than a 16x video card. I suspect there are some issues with this machine on Apple's end and I look forward to solutions from Apple. I suspect that I will obtain even faster results once the card works in the 16x slot. Hurry up and fix the incompatibility, Apple.

Meanwhile, here are some AJA System Test results for both the Seagate (first graphic) and Samsung drives.





Not bad, eh?

Oh, and the cost to create all of this: About $550 for RocketRaid 3522; $650 for Enhance E8 MS; $250 each for 1 TB Seagate or Samsung drives. Total-- around $3200. Compare that to pre-configured 8 tb turnkey systems costing over $8000.

The upside-- you save money. The downside-- assembly required (but a lot easier than your kid's bicycle), configuration required and no unified support. There remains a great deal to be said for buying a turnkey system from a storage vendor like Dulce Systems, Sonnet or CalDigit (just to name three which I have evaluated recently). Roll your own, though, does have its place in the market.

Stay tuned.. more coming on this site and eventually in print.

Rolling Shutter Effect In Sony EX-1

In the April, 2008 issue of DV Magazine I reviewed Sony's EX-1 camera.

I love the camera, warts and all, and it is high on my post-NAB purchase list. This is coming, you understand, from an unabashed user of the Panasonic HVX200 and long-GOP skeptic.

But one of the concerns that many users have raised about the EX-1 is its CMOS "rolling shutter" which effectively scans the entire image progressively rather than buffers the image as does a CCD-based device.

This rolling shutter effect can produced partially-exposed frames under certain conditions, primary among them being strobing or flash photography.

I was able to reproduce this in my testing subsequent to the article. I'm shooting at night at the Rockefeller Center ice rink using existing light (those half-inch chips really do well in low light situations). Of course, every parent needs to snap their kid doing multiple axils.

Take a look first at a single frame before the flash.








Now, the next two frames contain the flash.





Can you see the rolling shutter effect? Look at "The Concourse..." You can see that the area is normally exposed in frame 1 and brighter in frame 2. So, the flash effectively "rolled" from the bottom of the frame to the top.

But does it really make any difference in this scenario?

Take a look at a 4 second clip shot 720p24 at 60 fps for a slow-mo effect. The flash occurs about 3 seconds into the clip.


video

The flash is certainly visible but even when played at full HD resolution the shot still holds together.

I'll keep trying to break the rolling shutter. Stay tuned...