Multiple composr cms's across wireless network on multiple SSD's

IHi,

Am looking to see whether I can use multiple instances hosted on SD's, and sync the datasets for each instance across the network?

"The primary issue I was concerned about: Rsync, and generally built-in SQL functionalities for repeated datasets, rely on a master and a slave. These aren't permanent roles but for every data update, there is a single node's database that is being used as the template to copy to another. This is as opposed to Git, for example, which can merge files in general to preserve new inputs from multiple sources. The unidirectional master-slave relationship would be fine for reads but could lead to a write race condition if a second post/message/etc. is made before the first one can propagate throughout the network, unless of course there is only one master, active database. This isn't my area of expertise either, though, so if an expert is confident with the solution I'm on board!"

Any pointers would be extremely helpful!

Hi,

Right, this is a very tricky area. Master/slave would be how to do it, even though that is indeed imperfect. You can hand-configure _config.php so that it has a read database connection to a slave and a write database connection to the master.

Usually it is large enterprise or VC-funded projects that need to have multiple geo-distributed servers, and on these kinds of situations companies such as ocProducts might be brought in to help.

I would really like to be able to have out-of-the-box support for a network of lazy-synched geo-distributed machines to serve a single Composr site, but this would require a lot of work, particularly the auto-increment IDs we use everywhere would need to be replaced with UUIDs. I'm hoping we can do this with version 12, but we don't have version 11 finished yet.

(I saw your messages on IRC, sorry I wasn't at my machine then)

Chris,

Thanks for the informative reply, much appreciated!

What I am trying to acheive is this:

Communities throughout NZ will be left with no connectivity if we get hit by more natural disasters. Christchurch proved that our current infrastructure is woefully inadequate.

So I run wireless networks, in particular mesh networks. Idea is that every third node (access point) has an SSD, on which I stick Composr. So whilst network can have multiple SSID's, only SafetyNet will continue to be accessible via browser when disaster strikes. Should mention that nodes are solar powered and have 4 day back up.

What I wanted to acheive was the multiple instances of Composr, syncing so to speak across the network.

Have created very simple demo that would sit on SSD: demo.portmeshwireless.nz

Chris,

Wondered whether we could utilise either Git, Mecurial or Bazaar to achieve the same, whilst your wish for a lazy-synched geo-distributed, maybe a little way away?

Ok, well, the problems trying to disk sync are…

1. You would need to somehow have the database (which is a binary blob in any serious database system, for performance neccessity) go through distributed revision control. These systems are designed to work on lots of individual text files.
2. You still are going to have the ID sync problem.

So you could try and install Composr with the XML database, which we warn is just for developers, and put that in git. This will actually solve these problems. But the performance is probably going to be awful, and you'll probably be having a lot of bugs.
You'd then have to have some kind of background process that continually tries to push/pull the database to get the different servers in sync.

This is really on the edge of my expertise, as I've never had a client ask me to create a serious system like this. I imagine though there are solutions in MySQL. For example, I would guess there's some kind of lazy MySQL replication option. And I know people workaround the autoincrement ID by manually making each server start from a different offset through the ID space (which is a horrible hack, but it's what people do, lol).

Chris,

Thanks for that, and that was skirting on the edge of your expertise. Am seriously impressed!

Ok, if we went with the other config, as in master / slave, I wonder whether we would have the necessary speed over the network, bearing in mind that it would be the only network operating.

Or alternatively look at either Mecurial or Bazaar instead of Git?

If you can make it do lazy synching, then speed should not be an issue anyway. At least I'd assume not. It'd just be done in the background.

Mercurial and Bazaar do pretty much the same thing as Git, i.e. distributed revision control system intended for synching lots of individual assets rather than one big database.

Chris,

Thanks for your input, much appreciated. Will see how we get on, may just try both and see which comes out with the best performance!

Matt