Work In Progress-- RAID Expansion Chasis

The June issue of DV Magazine will feature my article on 3 RAIDS in a range of price and features. I'll be looking at a 2-drive unit from CalDigit, an 8-drive RAID 5 from Sonnet Technology and finally a Fibre-Channel server-level device from Dulce Systems.

Due to space limitations in print, I noted that I'll also be reviewing for the web an extension of an 8-drive array using SAS-Expander chassis and controller technology.

He is the testing so far.

For this test, I'm using another HighPoint RocketRaid 4322 cards, referenced in my Ultimate Raid article here on the blog as well as in DV. The HighPoint 4322 is my choice of RAID controller cards. It's dedicated Intel processor is faster than the prior generation; it is expandable with a battery backup daughter card; it supports both eSata and SAS drives through miniSAS 4-channel connectors; it is expansion chassis ready. My tests have already shown that the 4322 is faster than the 3xxx series. And at a price of $599, there is simply nothing on the market that comes close to its price/performance benefits.

I'm testing a 16-drive chassis from AIC, the XJ1100, which retails around $1600. It is a sturdy chassis with pop-out swappable drive trays requiring screws to secure the drive to the tray. Unlike many enclosures I've seen or tested, the drives trays are not lockable.

The unit is effectively 2 8-drive devices connected internally by an expansion circuitry. 4 miniSAS connectors on the rear allow dual connections to the controller card and dual connections to other 16 drive expansion chassis. Chaining multiple enclosures together can create some significant storage with blazing speed!

Like any enclosure of this class the XJ100 contains heavy-duty fans. Heat is, after all, a hard drive's greatest enemy. If you are using this enclosure for editing, you will definitely want to locate it in another room or within a soundproof enclosure. It sounds like a windtunnel. In justfication, however, all large enclosures such as this are noisy as they really are designed to be servers.

I'm testing this enclosure with 400gb Seagate Cheetah 15K SAS drives and Seagate 1tb ES-class eSata drives.

This is a preliminary report on testing the Seagate drives.

The Seagate Cheetah drives came pre-initialized as a RAID 6 and formatted to a little over 6 tb.

Note that 15K SAS drives are not available in the capacities of eSata drives, but the advantages are significant. The SAS drives are built for 24/7 heavy use with 1.6 million hours MTBF. Seagate designs them with specific error-checking algorithms. They are dual-ported, allowing the next drive in the array to take over immediately in the event of failure. For purposes of video, and this would apply specifically if the drives were being used in a SAN configuration, latency is a mere 2ms as opposed to a 4.1ms latency of eSata drives.

Because of this low latency and immediate response, EditShare only uses SAS drives in their turnkey SAN systems. I am certain this is the case for other vendors as well.

A faster rotational speed will naturally pump data faster and more reliably.

Take a look at the AJA System Test results for the Cheetah drives. Note as well that this speed is not just a function of the drives; the RocketRaid 4322 contributes significantly.

All I can say about results like that is WOW!

To test this in the real world, I copied to the drive several Red R3D files and exported 2K Quicktime files. I brought these into FCP which transcodes the files to ProRes 422 HQ. These 23.98 fps clips played without a single glitch. I also transcoded to Blackmagic 2K 2048x1556 4:4:4 RGB. All footage played without a hitch. Now something else-- I was able to play these files in FCP with scopes open and updating in real time. Even on my 8-drive array, I will sometimes drop frames while playing back live scopes (seems to be a common FCP problem). I always suspected the culprint was drive speed and this array helped prove my point.

As I write this entry, I'm now initializing a raid using the 16 Seagate ES drives. I'll report on those in the next installment and then the whole article will reprint on DV.COM once my June article appears.

Who is the logical customer for this technology?

For the RocketRaid card, the answer is any of us wishing to create the fastest raid with parity that relatively little money can buy.

The XJ1100 is definitely an enclosure for someone who stores massive amounts of data and needs the luxuries of speed, capacity, and redundancy. As one documentary filmmaker friend who dropped by the studio as I was testing the unit said, "I'd fill this in a year."

The Seagate 15K Cheetah drives are suitable for high end work. Their speed and dependability makes them candidates for SAN applications, whether in this enclosure or in another system. Any user loking for the top performance in any hard drive regardless of the system, should strongly consider the Cheetah drives. Keep in mind that the 400gb 15K Cheetahs run in the $525 range. But speed and data security... priceless.

Stay tuned for results with the Seagate ES (Enterprise Series) eSata drives.

RocketRaid 4322

The heart of any RAID system is its controller card. You can buy the highest capacity, fastest drives on the market and install them in the most sturdy, power-redundant, shock resistant enclosure you can buy. But if you controller card is merely a software-based "dumb" controller or if your "smart" controller with onboard processor does not deliver optimized performance, then you have less than optimal performance. Mediocre performance in a video raid can mean dropped frames. Worse than that, it could even mean loss of data in the event of disk failure.

On this blog as well as in the pages of DV Magazine, I created what I called The Ultimate Raid based upon the HighPoint Technologies RocketRaid 3522 card, a ProAvio 8-drive MiniSAS enclosure and 8 Seagate 1TB drives.

But I should have known better than to call it my Ultimate Raid. The minute I created the title, HighPoint Technologies went beyond ultimate with their release of the RocketRaid 4322.

Priced at $699, the RocketRaid 4322 delivers all of the performance that the most demanding editor would ever require.

Like its previously-ultimate sibling RocketRaid 3522, the 4322 is based around an on-board Intel processor. But the 4322 is based around Intel's IOP348 rather than the 3522's IOP341. This new speed demon clocks out at 1.2 ghz.

As a result of this faster and more powerful chip, the 4322 boasts a cooling fan rather than the heatsink design of the 3522. Yet the fan runs so quietly that I cannot tell the difference between the two chips.

The 4322 gains speed as well by now installing in an 8x PCIe slot in your Mac or PC workstation. It also supports both eSata as well as SAS drives thorugh its dual 4-lane MiniSAS connectors, just like the 3522.

Likewise, the 4322 supports every imaginable RAID configuration-- JBOD, 0, 1, 3, 5, 6, 10, 50 an single drive.

As I had written in the prior article, I recommend Raid 5 for video work. This level of protection which writes parity across all drives and can sustain the loss of one drive represents the optimal compromise between capacity and performance. The less-secure among us might want to configure Raid 6, reducing capacity and possibly speed to a degree but insuring against the loss of 2 drives.

Now, that about speed.

Let's take a look first at my earlier results from the RocketRaid 3522 in the above mentioned configuration using the AJA System Test utility, positing 1920x1080 10-bit uncompressed video.




Not shabby results at all. No wonder I thought that nothing could ever top this and could call it Ultimate. That data rate could sustain and in fact did in real-world work 10-bit uncompressed full HD without dropping a frame or missing a beat.

But look at the results from the 4322, this time assuming 2K video.



Wow! My read/write speeds increased by 50%! I can edit Red footage!

One more significant result comes from the Blackmagic Disk Speed test. Take a look at its read/write results as well as the maximum potential frame rate for a range of video scenarios.

What a difference a chipset and bus-speed bump can produce.

How do you say The Ultimate Raid Controller-- And Then Some?

If you are rolling your own Raid, this is the card which can give you complete confidence in everything from DV through 2K and in the data protection which it affords.

It earns my Ultimate recommendation.

HighPoint Technologies
www.highpoint-tech.com
Street Price around $699

Abaltat Muse 2.0

I've been playing with this remarkable composition tool for some months now and honestly have had so much fun with it that I could hardly break away to review it!

Muse is an intelligent scoring application which analyzes the color values and other properties of your video and then creates a music bed appropriate to the video.

On the surface, it seems quite simple. Open a Quicktime movie in Muse and click Compose. A dialog box will appear with choices of 7 different library styles, each with several variations. In seconds, Muse creates a music bed. Don't like it, modify it using keyframes which pop up choices of instruments, volume, transposition, scale or pan. Add a color tracker to alter the score at that timeline point to the color selected.



It that basic composition mode is insufficient, click Compose on the main preview window and then click Advanced. Options include style of composition (melodic or freestyle), color tracking options, tempo, complexity and jingle (repetitive elements within the track).

When done, export as a wav, aiff or Quicktime file. Bring it into your NLE and sync with the clip. And it's all royalty free.

To illustrate, I started with a quick edited sequence of a fast-paced ride through Dallas shot on a Sony EX-1 at 1 fps. Naturally, I would want a fast paced track to accompany it.



I selected Retro Tech from the Band pop up and experimented with a few presets, adding a few keyframes and bringing in instruments, fx and some fades. The video seemed to lend itself more to a freestyle composition than a Melodic method, Melodic following traditional more structured rules of composition.

In just a few minutes of tinkering, I produced what I consider to be a very fitting music bed.

One of the complaints that editors often voice about royalty free music is a certain mundane sameness of the tracks. No one could ever accuse Muse of producing such auditory boredom. It is as if you actually had a composer sitting beside you scoring to the music. Of course, a side value here is if you actually do have a composer, the Muse track can be a guideline to tell your composer "I want something that sounds like this."

At $299, it is a bargain. Anyone who has licensed music knows that $299 doesn't go very far in buying clips. And those clips are of stock lengths and don't always match up to the video. Muse produces music beds which just sound like custom compositions and with the almost-infinite variations which can be applied, no two clips ever need sound the same.

I would wish that future versions could include a greater variety of libraries and that a preview function of various styles could be added. My other significant wish-list items would be the ability to track luminance values in addition to color as well as to have a color-picker to choose within the video a more exact color.

I love it; I commend it to you. There is no other product even remotely like it. Muse is a simple and fun tool to add tremendous value to any production.

By the way, here's the clip with the final music bed--




Check it out at www.abaltat.com

Petroff 4x4 Mattebox on Sony EX-3

I've been too busy enjoying my new Sony PMW-EX3 even to post!

There has been some chatter on the discussion forums that the Sony EX1, EX3 and indeed even Red exhibit an unfortunate trait common to CMOS cameras-- poor rejection of the IR spectrum. This could potentially result in color shifts or inaccurate reproduction of colors.

I'll be testing both for this site as well as for a future article on the EX-3 the IR rejection capabilities of the B+W True-Cut 750 IR filter. This new filter is touted by Schneider Optics to be superior to its earlier 486 and 489 IR cutting filters, reducing the potential for a greenish tinge which could potentially occur with the 486 and 489.

The IR 750 is currently only available in a 4x4 filter and that meant using a mattebox.

The Petroff 4x4 has long been my favorite and courtesy of Petroff, you see a Petroff 4x4 mounted to the EX3 using a lens mounting ring which Petroff developed for the EX1. Petroff had not yet tested the EX3 but I was convinced that there would be no issue with fit since the lenses are identical other than the interchangeable feature of the EX3. I was right.

The unit I am testing here is a two stage 4x4 model with optional side wings and a French flag. It is mounted with the Zacuto baseplate and rod system, my unqualified favorite plate and worth every penny.

Why do I like this product so much?

It is sturdy yet lightweight. The mattebox is made from a combination of metal and virtually-indestructable polymide. It has flexibilty while also being able to bend without feeling that you are permanently bending metal.

It is modular and can be assembled/disassembled without tools. You can add or remove stages, insert/remove filters, assemble/disassemble the mattebox and not have the need of a screwdriver.

Each stage can rotate or be locked in place. In the case of the IR 750 filter, I would not want to rotate the stage. A 4x4 polarizer would be an entirely different manner. Those same thumbscrews over which the lock fits are also the screws you would loosen to remove the filter holder. Then it is a simply matter to drop the filter in the holder and return to the mattebox, tightening the screw.

Note one concern with the design of the EX1/EX3. That infernal (and, in my opinion useless) built-in mike protrudes too far forward. It does partially block the thumbscrew in the stage closest to the camera. It is not a major issue but a bit of an annoyance requiring deft small fingers or a needle-nose plier to operate the screw. I suspect it might be possible to design a donut to extend the entire assembly slightly. Petroff does manufacture a "universal" mattebox designed for Sony HDV cameras where the built-in mike constitutes a more significant issue. EX owners might want to consider this universal model.



In this shot, note how tightly the mattebox components fit together and how securely the lens adapter, large lens mount screw and rod attachment both hold the mattebox together as well as secure it to the Zacuto baseplate.

No light leaks, no wobbling, secure snap in filter holders give the user of this mattebox complete confidence that it is doing what a mattebox is supposed to do-- light control, hold filters, and fit securely to the supporting rods.


The Petroff mattebox accomplishes all of these tasks and then some.

The P44 two stage as shown lists at $1370. Follow-focus is also available.

Check out the full product specs and product line at www.petroff.ws

Check out the baseplate, rods and tons of other related accessories at www.zacuto.com. Conveniently enough, Zacuto is also a Petroff dealer.

That's the mattebox part of this installment. Will start shooting with the filter in the coming days to determine whether the IR problem is pervasive or just infrequent.

Quick Update on Ultimate Raid

Highpoint Technology has updated the firmware for the RocketRaid 3522 to allow booting from Mac OS X 10.5. Now, this is not quite as significant to the video editor since we would not be capturing to a startup volume, but it is a significant advance in the evolution of the RocketRaid card for those users who require large start-up volumes.

This is one impressive raid card. Bootable (for those whose applications need bootable raids), easily updated via firmware patches, ethernet port for remote control of raid and computer functions, multiple raid levels, optional battery backup, but most importantly its Intel IOP 81341 on-board processor. It is a true hardware raid controller with all of the speed, security and management capabilities that any editing installation would demand.

Work In Progress— The Ultimate Raid

I've been writing quite a bit about storage over the past couple of years going all the way back to this first roll your own raid article to a more recent review of an 8-drive array from Dulce Systems with more general storage articles (check all of the out on www.dv.com).

Currently waiting for publication is a review of the 3Ware Sidecar.

This time I wanted to build an ultimate raid. But just what does an editor need? Well, what I needed was a single workstation unit with lots of capacity, with redundancy and with the speed to handle capture and playback of 10 bit uncompressed HD. 2K capabilities would be a nice bonus but I really only see myself working in 2K just in technology demo rather than production at the moment.

I had built a Raid 1+0 based upon the CalDigit eSata card for my MacPro and a Sonnet card in my PowerMac G5, connecting to a 5 enclosure port-multiplier box I bought from CoolDrives.com.

The CalDigit and Sonnet cards are software based, utilizing port multiplier chips from Silicon Image but with the raid itself configured and managed through the OS (in my case, MacOS X) and SoftRaid, a superb management utility.

I wanted to go the hardware route this time, so I chose to create a solution based upon Highpoint Technology's RocketRaid 3522 card (note-- this is the Mac page, but it works as well with XP and Vista-based PC's) and and the Enhance Technology E8 MS enclosure.

The Highpoint RocketRaid card is hardware based with an onboard Intel IOP 81341 processor with 2 external mini-SAS ports, each capable of supporting 4 drives in Raid 0, 1, 3, 5, 6, 10, 50 and JBOD. It is a PCIe 8x card which works as well in a 4x PCIe slot and includes browser-based raid management. It will work with any enclosure and with SAS or SATA drives.

An added bonus with the Highpoint card is an ethernet port which allows remote access to the card to configure the raid or even initiate rebuilding remotely in the event of failure.

Hardware raid means, in brief, that all processing is performed by the card's processor taking load off the computer's processors. Hardware raid level 5 represents a good compromise between speed, capacity and redundancy. Effectively, parity is spread across all drives and the raid capacity is reduced by the size of one drive. Raid 5 allows for the failure of one drive, while raid 6 allows for two drive failures.

The Enhance E8 MS enclosure allows easy mounting and removal of drives and has two miniSAS connectors. It will accept either SAS or SATA I/II drives.

I have been testing with 8 Seagate 7200.11 1 tb drives as well as with 8 1 tb Samsung F1 drives. Stay tuned for the detailed comparison of these drives but catch a sneak preview below.

Preliminary testing is very favorable with some amazing speeds, 3-4 layers of 10 bit uncompressed HD without rendering and seemless playback of 7 layers of 10 bit uncompressed. This can be attributed not just to the quantity of fast drives but in great measure to the Highpoint card.

One problem so far-- my MacPro (early 2008) dual 3.2 will not recognize the RocketRaid card in Slot 2 (the 16x slot). In fact, it will not recognize any card in Slot 2 other than a 16x video card. I suspect there are some issues with this machine on Apple's end and I look forward to solutions from Apple. I suspect that I will obtain even faster results once the card works in the 16x slot. Hurry up and fix the incompatibility, Apple.

Meanwhile, here are some AJA System Test results for both the Seagate (first graphic) and Samsung drives.





Not bad, eh?

Oh, and the cost to create all of this: About $550 for RocketRaid 3522; $650 for Enhance E8 MS; $250 each for 1 TB Seagate or Samsung drives. Total-- around $3200. Compare that to pre-configured 8 tb turnkey systems costing over $8000.

The upside-- you save money. The downside-- assembly required (but a lot easier than your kid's bicycle), configuration required and no unified support. There remains a great deal to be said for buying a turnkey system from a storage vendor like Dulce Systems, Sonnet or CalDigit (just to name three which I have evaluated recently). Roll your own, though, does have its place in the market.

Stay tuned.. more coming on this site and eventually in print.

Rolling Shutter Effect In Sony EX-1

In the April, 2008 issue of DV Magazine I reviewed Sony's EX-1 camera.

I love the camera, warts and all, and it is high on my post-NAB purchase list. This is coming, you understand, from an unabashed user of the Panasonic HVX200 and long-GOP skeptic.

But one of the concerns that many users have raised about the EX-1 is its CMOS "rolling shutter" which effectively scans the entire image progressively rather than buffers the image as does a CCD-based device.

This rolling shutter effect can produced partially-exposed frames under certain conditions, primary among them being strobing or flash photography.

I was able to reproduce this in my testing subsequent to the article. I'm shooting at night at the Rockefeller Center ice rink using existing light (those half-inch chips really do well in low light situations). Of course, every parent needs to snap their kid doing multiple axils.

Take a look first at a single frame before the flash.

Now, the next two frames contain the flash.





Can you see the rolling shutter effect? Look at "The Concourse..." You can see that the area is normally exposed in frame 1 and brighter in frame 2. So, the flash effectively "rolled" from the bottom of the frame to the top.

But does it really make any difference in this scenario?

Take a look at a 4 second clip shot 720p24 at 60 fps for a slow-mo effect. The flash occurs about 3 seconds into the clip.




The flash is certainly visible but even when played at full HD resolution the shot still holds together.

I'll keep trying to break the rolling shutter. Stay tuned...