Monday, February 4, 2013

LFTP Usage

LFTP usage

There are two main commands used for downloading: pget for files and mirror for directories.

There are two main commands used for uploading: put for files and mirror -R for directories.

To cancel a transfer, use Ctrl+c

To resume partially downloaded files/directories, you will use the -c switch: mirror -c Directory pget -c Video1.mp4 put -c Video1.mp4 . We can also use --continue

Segmented Downloading


Note: This guide uses 3 segments as an example. You may use as many as needed or desired.

A pget command using segmentation is pget -n 3 Video2.mp4, where 3 is the number of segments.

A mirror command using segmentation is mirror --use-pget-n=3 Directory, where 3 is the number of segments.

You can use jobs -v to see the speeds of the individual segments as well as the total speed.

We can type multiple commands and make it in the queue.

first need to stop the queue
queue  stop
type the commands
queue start

delete the transfer from the queue or command using queue -d

To stop and delete a running transfer, use the kill command. If you set lftp to allow multiple transfers simultaneously, kill #, where # is the number listed by jobs, will stop and delete the specified transfer. kill all will stop and delete all transfers


lftp sftp://user:password@host  -e "get remote-file.name; bye"

In case the port for SSH connection is changed, then try:

lftp sftp://user:password@host:port  -e "get file.name; bye"

To send a file to a remote server:

lftp sftp://user:password@host  -e "put local-file.name; bye"
Check most of the lftp options using ?


lftp :~> ?
    !<shell-command>                     (commands)
    alias [<name> [<value>]]             bookmark [SUBCMD]
    cache [SUBCMD]                       cat [-b] <files>
    cd <rdir>                            chmod [OPTS] mode file...
    close [-a]                           [re]cls [opts] [path/][pattern]
    debug [<level>|off] [-o <file>]      du [options] <dirs>
    exit [<code>|bg]                     get [OPTS] <rfile> [-o <lfile>]
    glob [OPTS] <cmd> <args>             help [<cmd>]
    history -w file|-r file|-c|-l [cnt]  jobs [-v]
    kill all|<job_no>                    lcd <ldir>
    lftp [OPTS] <site>                   ls [<args>]
    mget [OPTS] <files>                  mirror [OPTS] [remote [local]]
    mkdir [-p] <dirs>                    module name [args]
    more <files>                         mput [OPTS] <files>
    mrm <files>                          mv <file1> <file2>
    [re]nlist [<args>]                   open [OPTS] <site>
    pget [OPTS] <rfile> [-o <lfile>]     put [OPTS] <lfile> [-o <rfile>]
    pwd [-p]                             queue [OPTS] [<cmd>]
    quote <cmd>                          repeat [OPTS] [delay] [command]
    rm [-r] [-f] <files>                 rmdir [-f] <dirs>
    scache [<session_no>]                set [OPT] [<var> [<val>]]
    site <site_cmd>                      source <file>
    torrent [-O <dir>] <file>            user <user|URL> [<pass>]
    version                              wait [<jobno>]
    zcat <files>                         zmore <files>

Please also take a note of this set -a
command which has many useful options

We can set the aliases,bookmarks in command line or set aliases in /etc/lftp.conf
We can also specify to only transfer newer files so we can launch the transfer as many times as we want and it will only transfer changed / new files (like rsync).
Syntax:
$ lftp -u username,password –e “set net:timeout 0” –e “mirror --reverse –verbose -n /local/directory remote/dir”  sftp.yourftpservername.com

 Here
  • -u specifies the username and password for the server you are connecting to
  • -e allows you to ‘queue’ commands. So, instead of waiting for lftp to be ready for the next command, we can write it all in one statement and it will just run them in order.
  • The first command is “set net:timeout 0” – this is just to disable the timeouts with lftp. Some of my directories contained hundreds and hundreds of subdirectories and it would timeout when reading them.
  • The second command is the good one, its for mirroring two directories and its commands are;
    • -revserse just switches the order of the directories (ie local remote, not remote local)
    • -verbose just displays what lftp is doing
    • -n copies only new files 
    also we can use
    "mirror --delete --only-newer --verbose"

 To download files in parallel


mirror --parallel=10
  
lftp has a few other clever tricks up its sleeve. The at switch can come in handy when you want to run the backup at a specific time 


lftp at 00:00 -u username,password -e "mirror --delete --only-newer --verbose path  ftpsite &

No comments: