[multicat-devel] Dynamic directory for output

[Jeremy Poulter]

Hi,

Sorry if this does not immediately sound like a multicat query, and 
maybe to some degree it is not but please bear with me ;-)

I am trying to setup a machine to archive weeks worth of everything that 
is broadcast. I have a machine setup with 12 x 3TB discs and I have a 
combination of dvblast and multicat receiving the data and saving to disc.

My problem is how donI get it to be distrobuted accross the 12 discs?

First of all let me describe some of my requirements.

- Recording time is more important than redundancy, ie I would prefere 
to have more disc space with no redundency.
- I would like to be able to replace a single discs in case of failure 
or upgrade.
- I would like to expand the capacity as 4TB and bigger become affordable.

So for these reasons I am trying to avoid use of LVM and or RAID as I 
can see no way to replace discs without giving up space for redundancy.

I found what I thought may be a solution in MHDDFS which provides a 
virtual view of multiple discs nicely disctabuting files across the 
discs, but this has too much of an overhead so reading and writing to 
the same disc can cause packets to be dropped.

So we get to the multicat bit. It occured to me that all I need to do is 
change the directory multicat is writing to each chunk then the chunks 
can be written accross multiple discs at full speed with out any 
intermediate program hindering performance.

My current thinking is that I will add an option to allow specification 
of an external utility to be called each time a new chunk is created (in 
OpenDirFile?) which will take as inputs the output dir and the new file 
number. This will then return the actual file name(s) to use.

This method has the added addvantage (for my senario anyway) that this 
can also be used as a trigger to add the file to a database, etc.

So am I missing anything obvious that would help in this case? Anything 
I sould be aware of befor implementing this? I am rather worried about 
the time it would take to execute an external process as I don't want to 
be dropping packets. Anything else to worry about?

Cheers,

Jeremy