View Post [edit]
Poster: | foundation | Date: | Nov 10, 2004 11:50pm |
Forum: | petabox | Subject: | Re: Filesystem |
I've been thinking about writing something like this for my company's image repository, and have been pondering how to represent the physical topology of the network so as to place duplicate copies not on the same drive, controller, server, rack or location if possible.
Reply [edit]
Poster: | brewster | Date: | Nov 11, 2004 12:39am |
Forum: | petabox | Subject: | Re: Filesystem |
we also sync between datacenters on different continents.
this gets around alot of types of errors for us while keeping it simple.
-brewster
Reply [edit]
Poster: | foundation | Date: | Nov 11, 2004 1:18am |
Forum: | petabox | Subject: | Re: Filesystem |
Also, if a box goes down do you have a spare box, that then gets rsync with the mirror to become the host? or does that have to happen manually?
Reply [edit]
Poster: | brewster | Date: | Nov 11, 2004 1:40am |
Forum: | petabox | Subject: | Re: Filesystem |
Reply [edit]
Poster: | dunno | Date: | Jun 16, 2005 1:55pm |
Forum: | petabox | Subject: | Re: Filesystem |
just a spur of the moment thought, but you could have a 2 tier data system, where the first tier is JBOD, and is generally the front end, and a second tier that has the same dataset as the first, except that it used RAID 5 at some level... maybe it could also be stagered time wise, the backup could be 2 days behind the first tier, with a rather reliable pool keeping the changelog between the first tier, and the 2 day old backup tier... that way you'd have all your information in two place, and you'd have some measure of protection against virii type corrpution that bypasses safeguards like redundancy... an well.
Reply [edit]
Poster: | foundation | Date: | Jul 15, 2005 5:08am |
Forum: | petabox | Subject: | Re: Filesystem |
and we are looking at mogilefs. Mogilefs uses mysql to track file locations, and automatically replicates the number of copies required. So you can say I want at all times there to be 2 copies of this data and three copies of this other data. And when you lose a server, it detects a copy is inaccessible and starts replicating a new copy. It does the transfers over http or nfs. Because we have written the front end, we don't need a posix compliant file system, we can use the client libraries. Something to consider for people implementing large systems, and a way to avoid RAID (it's raid-ish over the network really).