Monday, April 1, 2013

Check the httpd status using curl


Check the  httpd status using curl

[root@test]# curl -I http://yahoo.com
HTTP/1.1 301 Redirect
Date: Sat, 09 Mar 2013 05:52:09 GMT
Connection: close
Server: YTS/1.20.13
Cache-Control: no-store
Content-Type: text/html
Content-Language: en
Location: http://www.yahoo.com/
Content-Length: 211

[root@test]# curl -I http://yahoo.com | grep HTTP
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0   211    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
HTTP/1.1 301 Redirect
[root@test]# curl -Is http://yahoo.com | grep HTTP
HTTP/1.1 301 Redirect
[root@test]# curl -Is http://gmail.com
HTTP/1.0 403 Forbidden
Content-type: text/html; charset=UTF-8

[root@test Downloads]# curl -Is http://facebook.com
HTTP/1.0 403 Forbidden
Content-type: text/html; charset=UTF-8

[root@test]# curl -Is http://facebook.com | grep HTTP
HTTP/1.0 403 Forbidden
[root@test]# curl -Is https://facebook.com | grep HTTP
[root@test]# curl -Is http://google.com | grep HTTP
HTTP/1.1 301 Moved Permanently
[root@test]# curl -Is http://linkwithlinux.blogspot.in | grep HTTP
HTTP/1.1 200 OK
[root@test]#

Here I -
 -I/--head
              (HTTP/FTP/FILE) Fetch the HTTP-header only! HTTP-servers  feature  the  command  HEAD
              which  this  uses  to get nothing but the header of a document. When used on a FTP or
              FILE file, curl displays the file size and last modification time only.

-s =Silent or quiet mode.

200 status running fine.
301- redirect temporarily,permanent
403 - forbidden

Nginx Installation in Linux


                                                             
Download source file                                                                
                                                               
                                           
wget http://nginx.org/download/nginx-1.0.15.tar.gz

[root@test src]# tar -zxvf nginx-1.0.15.tar.gz

[root@test src]# ./configure --prefix=/usr/local/ --with-http_ssl_module
checking for PCRE library in /usr/pkg/ ... not found
checking for PCRE library in /opt/local/ ... not found

./configure: error: the HTTP rewrite module requires the PCRE library.
You can either disable the module by using --without-http_rewrite_module
option, or install the PCRE library into the system, or build the PCRE library
statically from the source with nginx by using --with-pcre=<path> option.

[root@test nginx-1.0.15]# yum install pcre*
[root@test nginx-1.0.15]# ./configure --prefix=/usr/local/ --with-http_ssl_module
Configuration summary
  + using system PCRE library
  + using system OpenSSL library
  + md5: using OpenSSL library
  + sha1: using OpenSSL library
  + using system zlib library

  nginx path prefix: "/usr/local/"
  nginx binary file: "/usr/local//sbin/nginx"
  nginx configuration prefix: "/usr/local//conf"
  nginx configuration file: "/usr/local//conf/nginx.conf"
  nginx pid file: "/usr/local//logs/nginx.pid"
  nginx error log file: "/usr/local//logs/error.log"
  nginx http access log file: "/usr/local//logs/access.log"
  nginx http client request body temporary files: "client_body_temp"
  nginx http proxy temporary files: "proxy_temp"
  nginx http fastcgi temporary files: "fastcgi_temp"
  nginx http uwsgi temporary files: "uwsgi_temp"
  nginx http scgi temporary files: "scgi_temp"

make

make install

cp conf/nginx.conf '/usr/local/nginx//conf/nginx.conf.default'
test -d '/usr/local/nginx//logs'                || mkdir -p '/usr/local/nginx//logs'
test -d '/usr/local/nginx//logs' ||             mkdir -p '/usr/local/nginx//logs'
test -d '/usr/local/nginx//html'                || cp -R html '/usr/local/nginx/'
test -d '/usr/local/nginx//logs' ||             mkdir -p '/usr/local/nginx//logs'
make[1]: Leaving directory `/usr/local/src/nginx-1.0.15'

[root@test nginx-1.0.15]# cd /usr/local/nginx/
[root@test nginx]# ls

conf  html  logs  sbin

[root@test nginx]# cd conf

[root@test conf]# vi nginx.conf


server {
        listen       8081;
        server_name  localhost;

        #charset koi8-r;

        #access_log  logs/host.access.log  main;

        location / {
            root   /var/ww/html;
            index  index.html index.htm;
        }

    listen: Specifies the port on which this virtual host listens.
    server: Lists the host headers for the site.
    location /: Specifies how to handle requests under the location '/', the site root.
    root: The document root for the site.
    index: An ordered priority list of default documents.


to test the configuration file

[root@test conf]# /usr/local/nginx/sbin/nginx -t
nginx: the configuration file /usr/local/nginx//conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/nginx//conf/nginx.conf test is successful

[root@test conf]# cd /usr/local/nginx/sbin/
[root@test sbin]# ./nginx

When you start up your nginx (just go to sbin and type ./nginx in order to start your web server!) you get few more directories:

[root@test sbin]# ps -ef | grep nginx
1222      9940  2436  0 05:05 ?        00:00:01 /usr/bin/kwrite /home/test/Desktop/nginx
root     18195     1  0 05:54 ?        00:00:00 nginx: master process ./nginx
nobody   18196 18195  0 05:54 ?        00:00:00 nginx: worker process
root     18202  3430  0 05:54 pts/1    00:00:00 grep nginx

[root@test nginx]# ls
client_body_temp  conf  fastcgi_temp  html  logs  proxy_temp  sbin  scgi_temp  uwsgi_temp

In config file

TCP nopush setting means that HTTP response hearders are all sent in one packet. Sendfile setting means that Nginx ignores the details of the file it is sending and uses kernel sendfile support instead. Keepalive setting defines how long server waits for users packets. This should be changed only to few seconds on busy sites. Gzip compression saves bandwith on site, depending what kind of packets server is sending.

/usr/local/nginx/html is the doc root.

http://localhost:8081

Welcome to nginx!

Note: If your server currently is configured with another web server, you'll likely need to have Nginx listen on a port other than 80. This is done simply by editing the 'listen' setting in the default server block, as well as any additional server blocks that are created.

Backup with rsnapshot utility

[root@test ~]# cd /usr/local/src/
[root@test src]# wget http://www.rsnapshot.org/downloads/rsnapshot-1.3.1.tar
.gz
--2013-01-14 05:16:24--  http://www.rsnapshot.org/downloads/rsnapshot-1.3.1.tar.
gz
Resolving www.rsnapshot.org... 208.92.64.114
Connecting to www.rsnapshot.org|208.92.64.114|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 215806 (211K) [application/x-gzip]
Saving to: “rsnapshot-1.3.1.tar.gz”

100%[======================================>] 215,806     39.8K/s   in 5.3s

2013-01-14 05:16:31 (39.8 KB/s) - “rsnapshot-1.3.1.tar.gz” saved [215806/215806]

[root@test src]# tar -zxvf rsnapshot-1.3.1.tar.gz
rsnapshot-1.3.1/
rsnapshot-1.3.1/rsnapshot-preamble.pl
rsnapshot-1.3.1/rsnapshot-program.pl
rsnapshot-1.3.1/rsnapshot-diff.pl
rsnapshot-1.3.1/rsnapshot.conf.default.in
rsnapshot-1.3.1/rsnapshot.1
rsnapshot-1.3.1/rsnapshot-diff.1
rsnapshot-1.3.1/AUTHORS
rsnapshot-1.3.1/COPYING
rsnapshot-1.3.1/INSTALL
rsnapshot-1.3.1/README
rsnapshot-1.3.1/TODO
rsnapshot-1.3.1/NEWS
rsnapshot-1.3.1/ChangeLog
rsnapshot-1.3.1/docs/
rsnapshot-1.3.1/docs/HOWTOs/
rsnapshot-1.3.1/docs/HOWTOs/rsnapshot-HOWTO.en.html
rsnapshot-1.3.1/docs/HOWTOs/rsnapshot-Mac-howto
rsnapshot-1.3.1/docs/HOWTOs/rsnapshot-windows-howto
rsnapshot-1.3.1/docs/Upgrading_from_1.1
rsnapshot-1.3.1/Makefile.am
rsnapshot-1.3.1/Makefile.in
rsnapshot-1.3.1/aclocal.m4
rsnapshot-1.3.1/configure[root@gai-1397 rsnapshot-1.3.1]# ./configure --sysconfdir=/etc
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make sets $(MAKE)... (cached) yes
checking for a BSD-compatible install... /usr/bin/install -c
checking for perl... /usr/bin/perl
checking for rsync... /usr/bin/rsync
checking for cp... /bin/cp
checking for rm... /bin/rm
checking for ssh... /usr/bin/ssh
checking for logger... /usr/bin/logger
checking for du... /usr/bin/du
configure: creating ./config.status
config.status: creating Makefile
config.status: creating rsnapshot
config.status: creating rsnapshot-diff
config.status: creating rsnapshot.conf.default
config.status: creating t/support/etc/configtest.conf
config.status: creating t/support/etc/rsync.conf
config.status: creating t/support/etc/gnu_cp.conf
config.status: creating t/support/etc/relative_delete_bugfix.conf
config.status: creating t/configtest.t
config.status: creating t/rsync.t
config.status: creating t/gnu_cp.t
config.status: creating t/relative_delete_bugfix.t

Now type  "make test"    to run the regression test suite.
Then type "make install" to install the program.

After rsnapshot is installed, don't forget to copy
/etc/rsnapshot.conf.default to /etc/rsnapshot.conf



[root@test rsnapshot-1.3.1]# make install
/usr/bin/pod2man -c '' -n 'rsnapshot' -r '' rsnapshot > rsnapshot.1
/usr/bin/pod2man -c '' -n 'rsnapshot-diff' -r '' rsnapshot-diff > rsnapshot-diff.1
make[1]: Entering directory `/usr/local/src/rsnapshot-1.3.1'
test -z "/usr/local/bin" || mkdir -p -- "/usr/local/bin"
 /usr/bin/install -c 'rsnapshot' '/usr/local/bin/rsnapshot'
 /usr/bin/install -c 'rsnapshot-diff' '/usr/local/bin/rsnapshot-diff'
test -z "/etc" || mkdir -p -- "/etc"
 /usr/bin/install -c -m 644 'rsnapshot.conf.default' '/etc/rsnapshot.conf.default'
test -z "/usr/local/man/man1" || mkdir -p -- "/usr/local/man/man1"
 /usr/bin/install -c -m 644 './rsnapshot.1' '/usr/local/man/man1/rsnapshot.1'
 /usr/bin/install -c -m 644 './rsnapshot-diff.1' '/usr/local/man/man1/rsnapshot-diff.1'
make[1]: Leaving directory `/usr/local/src/rsnapshot-1.3.1'

[root@test ~]# cp /etc/rsnapshot.conf.default /etc/rsnapshot.conf.

[root@test ~]# cp /etc/rsnapshot.conf.default /etc/rsnapshot.conf
[root@test ~]# vi /etc/rsnapshot.conf
[root@test ~]# mkdir /test
[root@test ~]# cd /public/
[root@test public]# ls
c  CentOS-6.3-i386-LiveDVD.iso  lost+found  packages
[root@test public]# cd ..
root@test ~]# rsnapshot configtest
Syntax OK

[root@test ~]# rsnapshot -t hourly
echo 5875 > /var/run/rsnapshot.pid
mkdir -m 0700 -p /.snapshots/
mkdir -m 0755 -p /.snapshots/hourly.0/
/usr/bin/rsync -a --delete --numeric-ids --relative --delete-excluded /home \
    /.snapshots/hourly.0/localhost/
mkdir -m 0755 -p /.snapshots/hourly.0/
/usr/bin/rsync -a --delete --numeric-ids --relative --delete-excluded \
    /public /.snapshots/hourly.0/testing/
mkdir -m 0755 -p /.snapshots/hourly.0/
/usr/bin/rsync -a --delete --numeric-ids --relative --delete-excluded /etc \
    /.snapshots/hourly.0/localhost/
mkdir -m 0755 -p /.snapshots/hourly.0/
/usr/bin/rsync -a --delete --numeric-ids --relative --delete-excluded \
    /usr/local /.snapshots/hourly.0/localhost/
touch /.snapshots/hourly.0/
[root@test ~]# rsnapshot -t daily
echo 5885 > /var/run/rsnapshot.pid
mkdir -m 0700 -p /.snapshots/
/.snapshots/hourly.5 not present (yet), nothing to copy


scripts that are executed before and after the backup
Code:

cmd_preexec    /home/test/backup/pre.sh
cmd_postexec    /home/test/backup/post.sh

pre.sh

Code:

screen -S test -X stuff "say Starting backup. World is Readonly! $(printf '\r')"
screen -S test -X stuff "save-off $(printf '\r')"
screen -S test -X stuff "save-all $(printf '\r')"


post.sh

Code:

screen -S test -X stuff "save-on $(printf '\r')"
screen -S test -X stuff "say Backup complete. World back to Readwrite $(printf '\r')"

[root@test ~]# ls -al /.snapshots/hourly.0/
total 16
drwxr-xr-x 4 root root 4096 Jan 14 06:34 .
drwx------ 3 root root 4096 Jan 14 06:33 ..
drwxr-xr-x 5 root root 4096 Jan 14 06:33 localhost
drwxr-xr-x 3 root root 4096 Jan 14 06:33 testing

# crontab -e
0 */4 * * * /usr/local/bin/rsnapshot hourly
30 23 * * * /usr/local/bin/rsnapshot daily

Ethernet bonding in Linux


Ethernet bonding in Linux

Bonding is nothing but integrating multiple ethernet interface in to a single one as bond0. This helps in performance improvement.

Steps for bonding in Fedora Core and Redhat Linux

Step 1.

Create the file ifcfg-bond0 with the IP address, netmask and gateway.  Below is my sample bond conf file.

$ cat /etc/sysconfig/network-scripts/ifcfg-bond0

DEVICE=bond0
IPADDR=192.168. 10.14
NETMASK=255. 255.255.0
GATEWAY=192. 168.10.30
USERCTL=no
BOOTPROTO=none
ONBOOT=yes

Step 2.

Modify eth0, eth1 and eth2 configuration as shown below. Comment out, or remove the ip address, netmask, gateway and hardware address from each one of these files, since settings should only come from the ifcfg-bond0 file above.

$ cat /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes

$ cat /etc/sysconfig/network-scripts/ifcfg-eth1

DEVICE=eth1
BOOTPROTO=none
ONBOOT=yes
USERCTL=no
MASTER=bond0
SLAVE=yes

$ cat /etc/sysconfig/network-scripts/ifcfg-eth2

DEVICE=eth2
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes

Step 3.

Set the parameters for bond0 bonding kernel module. Add the following lines to /etc/modprobe. conf

# bonding commands
alias bond0 bonding
options bond0 mode=balance-alb miimon=100

Note: Here we configured the bonding mode as "balance-alb". All the available modes are given at the end and you should choose appropriate mode specific to your requirement.

Step 4.

Load the bond driver module from the command prompt.

$ modprobe bonding

Step 5.

Restart the network
$ service network restart

check the proc settings.

$ cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.0.2 (March 23, 2006)

Bonding Mode: adaptive load balancing
Primary Slave: None
Currently Active Slave: eth2
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth2
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:13:72:80: 62:f0

Look at ifconfig -a and check that your bond0 interface is active. You are done!

RHEL bonding supports 7 possible "modes" for bonded interfaces. These modes determine the way in which traffic sent out of the bonded interface is actually dispersed over the real interfaces. Modes 0, 1, and 2 are by far the most commonly used among them.

* Mode 0 (balance-rr)
This mode transmits packets in a sequential order from the first available slave through the last. If two real interfaces are slaves in the bond and two packets arrive destined out of the bonded interface the first will be transmitted on the first slave and the second frame will be transmitted on the second slave. The third packet will be sent on the first and so on. This provides load balancing and fault tolerance.

* Mode 1 (active-backup)
This mode places one of the interfaces into a backup state and will only make it active if the link is lost by the active interface. Only one slave in the bond is active at an instance of time. A different slave becomes active only when the active slave fails. This mode provides fault tolerance.

* Mode 2 (balance-xor)
Transmits based on XOR formula. (Source MAC address is XOR'd with destination MAC address) modula slave count. This selects the same slave for each destination MAC address and provides load balancing and fault tolerance.

* Mode 3 (broadcast)
This mode transmits everything on all slave interfaces. This mode is least used (only for specific purpose) and provides only fault tolerance.

* Mode 4 (802.3ad)
This mode is known as Dynamic Link Aggregation mode. It creates aggregation groups that share the same speed and duplex settings. This mode requires a switch that supports IEEE 802.3ad Dynamic link.

* Mode 5 (balance-tlb)
This is called as Adaptive transmit load balancing. The outgoing traffic is distributed according to the current load and queue on each slave interface. Incoming traffic is received by the current slave.

* Mode 6 (balance-alb)
This is Adaptive load balancing mode. This includes balance-tlb + receive load balancing (rlb) for IPV4 traffic. The receive load balancing is achieved by ARP negotiation. The bonding driver intercepts the ARP Replies sent by the server on their way out and overwrites the src hw address with the unique hw address of one of the slaves in the bond such that different clients use different hw addresses for the server.
In this example, we'll manually set up a bonding interface and use round-robin load balancing to send traffc.


Load the bonding module with MII monitoring and round-robin load balancing: modprobe bonding mode=balance-rr miimon=100

Configure the bond0 NIC: ifconfig bond0 192.168.10.12 netmask 255.255.255.0 up

Add the 2 slaves: ifenslave bond0 eth0 and ifenslave bond0 eth1


That's it  Now the bind_addr for JGroups needs to be 192.168.10.12, and traffic will be load balanced between eth0 and eth1. When eth1 does down, only eth0 will be used, and when eth1 comes back up, it will be used again.

 Note that for IP multicasting to work, a route may have to be added to bond0:

route add -net 224.0.0.0 netmask 240.0.0.0 dev bond0


Tools like iptraf can now be used to monitor the physical NICs. We would see that traffic is split evenly across eth0 and eth1, and when a NIC is brought down, all traffic will be sent to the remaining NIC until the down NIC is brought up again.