Shell script to create multiple files at a time

Create multiple files at  a time using below script.


cat file.sh

for i in `seq 1 1 15`;

do touch guest$i.txt;

 done

#1 is the starting number. 1 is the incrementing order, if 2 it increments even, 15 upto 15 no it will create it.

Shell script to create multiple users

cat  useradd.sh

#!/bin/sh

for i in `more userlist`
do
echo $i
useradd $i
echo tester!23 | passwd $i --stdin
echo; echo “User $username’s password changed!”

done

Create a file userlist and specify the username you would like to create the user account and run the above script. Then the user account are created with the default password 'tester!23'

LFTP,SFTP password less login Linux to Windows

LFTP can be used as any typical FTP client but it also provides the ability to connect without asking about the password. The solution is simple however a bit unsafe as the password is given explicitly as text, like below. Thus anybody who can read the script can also steal the password, so make sure your script is readable for you only. Also, do not connect as root. Create a special user for this without root super-powers. The example below shows how to get a file from a remote server via secure SFTP protocol (bye at the end closes connection):

lftp sftp://user:password@host:port  -e "get file.name; bye"

We cannot able to upload multiple directories using SFTP but i achieved uploading multiple directories using LFTP,SFTP

Here are the steps i tried to make password less login from linux to windows.

Created the public and private key using putty in windows.

Copied the keys to my linux machine.

Converted the private key to linux key format using puttygen command

puttygen mywinkey.ppk -O private-openssh -o mykey.

Here mywinkey.ppk is the windows private key and mykey is the key created in linux format.

Move mykey file to the .ssh/  folder or copy the key and paste it .ssh/authorized_keys and change its permission

chmod 700 .ssh/authorized_keys

Tried now but prompts for the password, Type Enter key it will enter in to the ftp server

 lftp  sftp://username@sftpservername/
password:

To avoid prompting for password , please follow the below steps.

Enter in to the ftp server and type the below commands

set bmk:save-passwords true
bookmark add testing (Here testing is the bookmark name)
bookmark list

To delete the bookmark, type the below command

bookmark del bookmark name
type help bookmark for more details.

 history -c clears history
                                           
Now we can able to login the windows remote ftp server with out password and avoid prompting for password.


Type

lftp testing 
or
lftp testing sftp"//username@sftpservername.com

For automatic backups also we can use the lftp like rsync .Click here to see more lfp usage commands.

LFTP Usage

LFTP usage

There are two main commands used for downloading: pget for files and mirror for directories.

There are two main commands used for uploading: put for files and mirror -R for directories.

To cancel a transfer, use Ctrl+c

To resume partially downloaded files/directories, you will use the -c switch: mirror -c Directory pget -c Video1.mp4 put -c Video1.mp4 . We can also use --continue

Segmented Downloading

Note: This guide uses 3 segments as an example. You may use as many as needed or desired.

A pget command using segmentation is pget -n 3 Video2.mp4, where 3 is the number of segments.

A mirror command using segmentation is mirror --use-pget-n=3 Directory, where 3 is the number of segments.

You can use jobs -v to see the speeds of the individual segments as well as the total speed.

We can type multiple commands and make it in the queue.

first need to stop the queue
queue  stop
type the commands
queue start

delete the transfer from the queue or command using queue -d

To stop and delete a running transfer, use the kill command. If you set lftp to allow multiple transfers simultaneously, kill #, where # is the number listed by jobs, will stop and delete the specified transfer. kill all will stop and delete all transfers

lftp sftp://user:password@host  -e "get remote-file.name; bye"

In case the port for SSH connection is changed, then try:

lftp sftp://user:password@host:port  -e "get file.name; bye"

To send a file to a remote server:

lftp sftp://user:password@host  -e "put local-file.name; bye"Check most of the lftp options using ?

lftp :~> ?
    !<shell-command>                     (commands)
    alias [<name> [<value>]]             bookmark [SUBCMD]
    cache [SUBCMD]                       cat [-b] <files>
    cd <rdir>                            chmod [OPTS] mode file...
    close [-a]                           [re]cls [opts] [path/][pattern]
    debug [<level>|off] [-o <file>]      du [options] <dirs>
    exit [<code>|bg]                     get [OPTS] <rfile> [-o <lfile>]
    glob [OPTS] <cmd> <args>             help [<cmd>]
    history -w file|-r file|-c|-l [cnt]  jobs [-v]
    kill all|<job_no>                    lcd <ldir>
    lftp [OPTS] <site>                   ls [<args>]
    mget [OPTS] <files>                  mirror [OPTS] [remote [local]]
    mkdir [-p] <dirs>                    module name [args]
    more <files>                         mput [OPTS] <files>
    mrm <files>                          mv <file1> <file2>
    [re]nlist [<args>]                   open [OPTS] <site>
    pget [OPTS] <rfile> [-o <lfile>]     put [OPTS] <lfile> [-o <rfile>]
    pwd [-p]                             queue [OPTS] [<cmd>]
    quote <cmd>                          repeat [OPTS] [delay] [command]
    rm [-r] [-f] <files>                 rmdir [-f] <dirs>
    scache [<session_no>]                set [OPT] [<var> [<val>]]
    site <site_cmd>                      source <file>
    torrent [-O <dir>] <file>            user <user|URL> [<pass>]
    version                              wait [<jobno>]
    zcat <files>                         zmore <files>

Please also take a note of this set -a command which has many useful options
We can set the aliases,bookmarks in command line or set aliases in /etc/lftp.conf
We can also specify to only transfer newer files so we can launch the transfer as many times as we want and it will only transfer changed / new files (like rsync).
Syntax:
$ lftp -u username,password –e “set net:timeout 0” –e “mirror --reverse –verbose -n /local/directory remote/dir”  sftp.yourftpservername.com

 Here

• -u specifies the username and password for the server you are connecting to
• -e allows you to ‘queue’ commands. So, instead of waiting for lftp to be ready for the next command, we can write it all in one statement and it will just run them in order.
• The first command is “set net:timeout 0” – this is just to disable the timeouts with lftp. Some of my directories contained hundreds and hundreds of subdirectories and it would timeout when reading them.
• The second command is the good one, its for mirroring two directories and its commands are;
  • -revserse just switches the order of the directories (ie local remote, not remote local)
  • -verbose just displays what lftp is doing
  • -n copies only new files 
  also we can use
  

  "mirror --delete --only-newer --verbose"

  
  

 To download files in parallel

mirror --parallel=10

lftp has a few other clever tricks up its sleeve. The at switch can come in handy when you want to run the backup at a specific time 

lftp at 00:00 -u username,password -e "mirror --delete --only-newer --verbose path  ftpsite &

Install and Configure LFTP

LFTP is sophisticated ftp/http client, file transfer program supporting a number of network protocols. Like BASH, it has job control and uses readline library for input. It has bookmarks, built-in mirror, can transfer several files in parallel. It was designed with reliability in mind.

You can download this free software from http://lftp.yar.ru/get.html

tar -zxvf lftp-4.4.0.tar.gz
cd lftp-4.4.0
./configure --with-openssl --without-gnutls --with-debug

checking whether inet_aton is declared... yes
checking whether strptime is declared... yes
checking whether strtok_r is declared... (cached) yes
checking whether dn_expand is declared... yes
checking whether memmem is declared... (cached) yes
configure: creating ./config.status
config.status: creating Makefile
config.status: creating src/Makefile
config.status: creating lib/Makefile
config.status: creating doc/Makefile
config.status: creating po/Makefile.in
config.status: creating m4/Makefile
config.status: creating trio/Makefile
config.status: creating lib/config.h
config.status: executing depfiles commands
config.status: executing po-directories commands
config.status: creating po/POTFILES
config.status: creating po/Makefile
config.status: executing libtool commands

make
make check
make install
[root@test lftp-4.4.0]# updatedb
[root@test lftp-4.4.0]# which lftp
/usr/local/bin/lftp

Ensure that lftp is built with ssl by checking with the below command (libssl.so)

[root@test lftp-4.4.0]# ldd /usr/local/bin/lftp
        linux-gate.so.1 =>  (0x0081a000)
        libexpat.so.0 => /lib/libexpat.so.0 (0x00cf9000)
        libssl.so.6 => /lib/libssl.so.6 (0x04c1b000)
        libcrypto.so.6 => /lib/libcrypto.so.6 (0x04ad8000)
        librt.so.1 => /lib/librt.so.1 (0x00129000)
        libreadline.so.5 => /usr/lib/libreadline.so.5 (0x05d81000)
        libutil.so.1 => /lib/libutil.so.1 (0x009e6000)
        libtermcap.so.2 => /lib/libtermcap.so.2 (0x009ec000)
        libresolv.so.2 => /lib/libresolv.so.2 (0x009b4000)
        libdl.so.2 => /lib/libdl.so.2 (0x00bb2000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00134000)
        libc.so.6 => /lib/libc.so.6 (0x00a15000)
        /lib/ld-linux.so.2 (0x009f6000)
        libpthread.so.0 => /lib/libpthread.so.0 (0x00bb9000)
        libgssapi_krb5.so.2 => /usr/lib/libgssapi_krb5.so.2 (0x05a4a000)
        libkrb5.so.3 => /usr/lib/libkrb5.so.3 (0x05a7a000)
        libcom_err.so.2 => /lib/libcom_err.so.2 (0x009cb000)
        libk5crypto.so.3 => /usr/lib/libk5crypto.so.3 (0x059ac000)
        libz.so.1 => /lib/libz.so.1 (0x00b9d000)
        libkrb5support.so.0 => /usr/lib/libkrb5support.so.0 (0x009db000)
        libkeyutils.so.1 => /lib/libkeyutils.so.1 (0x009d6000)
        libselinux.so.1 => /lib/libselinux.so.1 (0x00446000)
        libsepol.so.1 => /lib/libsepol.so.1 (0x004ba000)

[root@test lftp-4.4.0]# ldd /usr/local/bin/lftp | grep libssl
        libssl.so.6 => /lib/libssl.so.6 (0x04c1b000)

[root@test ~]# /usr/local/bin/lftp sftp://username@xxx.example.com(remote server name)
Password:
lftp username@xxx.example.com:~> lcd /root/
lcd ok, local cwd=/root