VoodooPad's WebDAV Syncing Tech
(Or, how VoodooPad does its page syncing)
Miscellaneous infomation you need to know
- Every document in VoodooPad has a UUID associated with it. And every page in that document also has a UUID associated with it.
- A "client" is just a VoodooPad document on the desktop, iPad, or iPhone.
- A "sync id" is a unique identifier for each VP document. For VoodooPad on the desktop, it is based on the volume UUID that the document in sitting on, along with "file system number" of the document. This allows you to have multiple clients for the same sync data on the same machine. For the iPad and iPhone, it is the UUID of mobile device.
- This document was written with VoodooPad 4.3 in mind. Changes might have occurred after that release.
The document structure
When you setup VoodooPad to sync its pages over a webdav server, it will create a directory structure with the following hierarchy:
VoodooPadSync/ - This is the top level directory that documents are synchronized to. Inside it are only document folders.
VoodooPadSync/<doc uuid>/ - This directory represents a document, and contains a couple of files and a pages directory. The UUID is of the first document to sync to this location.
VoodooPadSync/<doc uuid>/Info.plist - This contains information about the document, which is retrieved when you pair up a document for a first sync. It has the name of the document, and some other tidbits about it.
VoodooPadSync/<doc uuid>/syncinfo.plist.gz - This contains information on all the pages in the document, a list of clients (documents), and pages to be deleted.
VoodooPadSync/<doc uuid>/pages/<page(s) uuid.plist.gz> - this is a collection of the pages, which are gzip'd binary plists.
VoodooPadSync/<doc uuid>/Lock/ - This directory is created when a document syncs to the server. If this directory is present, that means another client is currently either reading or writing to the sync location. If the Lock directory is already present when VoodooPad tries to sync, then it just spins in the background waiting for it to clear. However, VoodooPad does peek at the Lock/ directory create date and if it has been over 8 minutes (by default) since it was made, it assumes that the last client must have crashed. It then deletes the Lock/ folder, recreates it, and moves along with the sync.
The Lock directory is created even when there is nothing to push up. If we're reading data, then we don't want another client changing it when we're pulling it down.
A flaw- since nothing is written to the Lock directory when we read, then if it takes over 8 minutes and another client is waiting on the lock to clear... well, our lock will be deleted. A fix would be to write some piece of data in there when reading, and update it every 7 minutes or so, to keep other clients from deleting the Lock.
Push/Pull
When syncing, VoodooPad first looks on the server and then pulls down any changes. If there is a page name conflict, VP renames the local page and assumes the one on the server is the "right" one. Once everything has been pulled down, and deleted pages have been removed, VP then will push up the local changes.
One other trick VP does- it makes a hash of the pages, and compares the server version along with the local version. If the server version is newer, but the data hash is the same, VP won't bother pulling it down. The same thing for pushing.
How does it know what is new?
When VoodooPad is done syncing, it will ask the server what the modified date is for the last file it wrote. VoodooPad will then store this data, and the next time it talks to the server, it figures out what pages have been updated since that last date.
Other Stuff
And here is what a typical VoodooPadSync folder would look like:
VoodooPadSync/
8a513e9e-da86-4fc7-9c2a-6da29c2f091d/
Info.plist
syncinfo.plist.gz
pages/
./05cb8826-b9e6-4151-9ec1-46187a92f1ef.plist.gz
./0e7c7775-4f5d-47d7-8379-21eff0c8530a.plist.gz
...
./3d357f41-2d9f-4d3a-be16-87ee943186af.plist.gz
fca0e46f-f8f3-43c1-829d-95aad0cfdb25/
Info.plist
syncinfo.plist.gz
pages/
./d7e9db5e-e5a0-4631-b9a3-084fcb147ccc.plist.gz
./dcb06838-99e8-4990-b0e6-2b1545a977da.plist.gz
...
./f9813210-3ae0-4dd3-b3d7-c71f7bd8275b.plist.gz
If you're a cocoa developer and you would like to investigate using WebDAV for your syncing needs, make sure to check out Flying Meat's FMWebDAVRequest class: http://code.google.com/p/flycode/source/browse/trunk/davtest/ . It doesn't do everything, but it does what Flying Meat Developers need and use in VoodooPad.
