lights FAQ Forum github.com/luapower/rsync
rsync

Networking
socket
bnet
socketloop
libcurl
libssh2
rsync

rsync

rsync algorithm in Lua


local rsync = require'rsync'

The RSYNC algorithm in Lua.

This library can be used to synchronize similar-but-not-identical files between two machines with little bandwidth costs. This implementation is CPU-bound on Gigabit networks even with a good CPU.

API

stream-based API
rsync:gen_sigs_file(read, write [, block_len]) -> sig_count compute and save block signatures
rsync:load_sigs_file(read) -> sigs load block signatures
rsync:gen_deltas_file(read, sigs, write[, block_len]) -> false_alarms compute and save file differences
rsync:patch_file(read_delta, seek, read, write[, block_len]) patch original file with file differences
config
rsync:new([config]) -> rsync new rsync module with optional overrides
rsync.block_len default block length (1024)f_at
rsync.mem_len default buffer length (64K)
rsync:weak_sum() -> sum create a weak sum digest (default is rolling sum in Lua)
rsync:strong_sum() -> sum create a strong sum digest (default is blake2.blake2sp_digest)

The file functions operate on abstract read and write functions:

  • read(buf, len) -> read_len assumed to read at most len bytes into buf, returns 0 to signal EOF, raises errors on failure.
  • write(buf, len) assumed to always write exactly len bytes from buf. should raise an error otherwise.

Callbacks are not allowed to yield but they can use coro for control inversion.

HOWTO

To update file1 on machine1 from the updated file1 on machine2, first call rsync:gen_sigs_file() on machine1 with a read function that reads file1 and with a write function that sends its bytes to machine2. On machine2 call rsync:load_sigs_file() with a read function that reads the bytes sent through the network from machine1. Then (still on machine2) call rsync:gen_deltas_file() with the signatures got from the previous step and a read function that reads file1 and a write function that sends its bytes to machine1. Finally on machine1, call rsync:patch_file() with a read_delta function that reads the bytes sent through the network from machine1, a seek(offset) function that seeks in file1, a read function that reads from file1, and a write function that writes the resulting bytes to a new file. The optional arg block_len, if given, should be identical on all calls.


Last updated: 35 hours ago | Edit on GitHub

Pkg type:Lua+ffi
Version: r1-2-gc1ea780
Last commit:
License: PD
Requires: luajit 
Required by: none

Top