| Age | Commit message (Collapse) | Author |
|---|
|
|
|
|
|
An improvement could be adding "remote_root_dir" option to ssync-fetch
so we can use it in cases of rsync
|
|
|
|
Use config to generate the necessary options for each sub process
running:
1) ssync-index REMOTE
2) ssync-index LOCAL
3) ssync-queue
4) ssync-fetch
|
|
|
|
|
|
|
|
This version will require two separate calls to index local and
remote.
Local should be `ssync-index -b -o /output/local.idx ROOT_DIR`
Remote should be `ssync-index -r -w N -o /output/remote.idx ROOT_DIR`
|
|
Breaking ssync into 3 sub-processes:
1) ssync-index - indexes remote and local dirs
2) ssync-queue - generates queue of yet-fetched files
3) ssync-fetch - downloads the queue
Which will ultimately be executed using ssync which will allow for
unified config files and transfer locking.
The rewrite is being done in hopes of preventing "missing files"
during large queues and ensure completeness.
The breakdown into multiple files should also help with narrowing the
logic and improving the process without interfering with the execution
and readability of the other stages.
This commit has complete subprocesses - though ssync-index needs
remediation to remove the config file - as we need predictable I/O to
be able to pass the index files into the queue process
|
|
When we find files but nothing to sync keep the file around.
|
|
For debugging it can be helpful to see when we've listed files but
nothing to sync vs when we've found nothing to get at all.
|
|
We don't need the run file to know there was nothing to fetch
|
|
This prevents duplicate fetching from occuring
|
|
|
|
Wrap the remote connection with " " to avoid breaking when there is a (
or other shell characters in the file
|
|
Typos in the fetch file variable and not using the last run file
variable.
|
|
|
|
|
|
|
|
|
|
|