Pages

Friday, April 28, 2017

Use of shopt command

1) SHOPT

Shopt(SHell OPTions) is a built-in command to change the properties of a shell such as..
Its history behavior
Spell check

Enable special characters for echo command by default, and many more.

This is an excellent command which give more control on Shell for you when you are working on bash and sh shells. This command is available after bash v2 and not available in other shells such as ksh, csh etc. Lets get familiarize with shopt command with some examples.
shopt is a shell builtin command to set and unset (remove) various Bash shell options. To see current settings, type:

# shopt

Sample outputs:

autocd           off
sourcepath               on
xpg_echo                  off

To enable (set) option use the following command:
# shopt -s histappend

To disable (unset) option use the following command:

# shopt -u histappend

Example5: Append history but not over write the history. Many of us see that some of our commands vanish or deleted in our history when we logout and login. We will be shocked to see that they are missing command in history. This is a default behavior of your shell to over write the the previous sessions with the last closed session. For example you initiated two sessions, say it as session1 and session2. When you close session2 first and then session1 next, you will lost session2 history in history command, in other words we can say your session2 is over written by session1 history. If you don’t want lose your all sessions history use histappend option with shopt command as shown below. Keep this command in ~/.bashrc file.

shopt -s histappend

From now on words you can see all your history without any issue(with merged with your the previous sessions). Do you want to know what time you executed your commands in history

PROBLEM WITH SPELL CHECKS?

Example 6: shopt is an excellent command used to check spellings too. With this command we can make cd command to correct any spelling mistakes in the directory structure when changing directories. Enable cdspell option with shopt command as shown below

#shopt -s cdspell

After this if you try to change directory /var/ftp but typed wrong as /var/fdp this error is taken care once the above option is set.
cd /var/fdp
pwd
/var/ftp

PROBLEM WITH ECHO COMMAND?

Example 7: By default echo command will not understand special characters such as /n( for new line), /r( for return) etc. see below example.

echo “This is first line\n now on second line”
This is first line\n now on second line

If you observe \n is displayed literally. To avoid this we can enable xpg_echo option as show below.

# shopt -s xpg_echo
Now if you run above echo command see the output.
This is first line
now on second line

Note: This can be achieved with -e option along echo command

GLOB OPTIONS

Globing is a concept to expand a pattern. By default bash can understand *(For generating multiple alphanumerics ), ?(For generating single alpha-numeric) and [-](For range of chars) globing options. In shopt we have Golb(al) options which are very much useful for wildcards and other stuff. some of the as follows.
dotglob –IF set echo * command will show even hidden files too.
extglob –If set This will exclude the matches of search pattern
failglob –If set, and no matches are found, an error message is printed and the command is not executed.
nocaseglob –If set this will ignore case when listing of files.
nullglob –If set, pattern match to no files to expand to a null string, rather than themselves

There are many another things your shopt command will do. A Good Reference for all the options:

Wednesday, April 26, 2017

List of causes for Kernel Panic Error

Defective or Incompatible RAM
Incomplete,obsolete (or) corrupted kernel extensions
Incompatible , obsolete or corrupted drivers
Hard sisk corruption
Insfufficient hard disk sparc
Incompatibel hardware

Temporarly installed hardware or software

After recompiling a kernel binary image from source code, a kernel panic while booting the resulting kernel is a common problem if the kernel was not correctly configured, compiled or installed.[9] Add-on hardware or malfunctioning RAM could also be sources of fatal kernel errors during start up, due to incompatibility with the OS or a missing device driver.[10] A kernel may also go into panic() if it is unable to locate a root file system.[11] During the final stages of kernel userspace initialization, a panic is typically triggered if the spawning of init fails, as the system would then be unusable

Kernel panics appear in Linux like in other Unix-like systems, but they can also generate another kind of error condition, known as a kernel oops.[14] In this case, the kernel normally continues to run after killing the offending process. As an oops could cause some subsystems or resources to become unavailable, they can later lead to a full kernel panic.

In Linux kernel, a kernel panic causes keyboard LEDs to blink as a visual indication of a critical condition

KDUMP

⁠32.2.3. Configuring kdump on the Command Line
⁠Configuring the Memory Usage
Memory reserved for the kdump kernel is always reserved during system boot, which means that the amount of memory is specified in the system's boot loader configuration. This section will explain how to change the amount of reserved memory on AMD64 and Intel 64 systems and IBM Power Systems servers using the GRUB boot loader, and on IBM System z using zipl. To configure the amount of memory to be reserved for the kdump kernel, edit the /boot/grub/grub.conf file and add crashkernel=<size>M or crashkernel=auto to the list of kernel options as shown in Example 32.1, “A sample /boot/grub/grub.conf file”. Note that the crashkernel=auto option only reserves the memory if the physical memory of the system is equal to or greater than:
2 GB on 32-bit and 64-bit x86 architectures;
2 GB on PowerPC if the page size is 4 KB, or 8 GB otherwise;
4 GB on IBM S/390.
Example 32.1. A sample /boot/grub/grub.conf file

# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,0)
#          kernel /vmlinuz-version ro root=/dev/sda3
#          initrd /initrd
#boot=/dev/sda
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux Server (2.6.32-220.el6.x86_64)
        root (hd0,0)
        kernel /vmlinuz-2.6.32-220.el6.x86_64 ro root=/dev/sda3 crashkernel=128M
        initrd /initramfs-2.6.32-220.el6.x86_64.img
Make sure the system has enough memory
This section is available only if the system has enough memory. To learn about minimum memory requirements of the Red Hat Enterprise Linux 6 system, read the Required minimums section of the Red Hat Enterprise Linux Technology Capabilities and Limits comparison chart. When the kdump crash recovery is enabled, the minimum memory requirements increase by the amount of memory reserved for it. This value is determined by the user, and defaults to 128 MB plus 64 MB for each TB of physical memory (that is, a total of 192 MB for a system with 1 TB of physical memory). The memory can be attempted up to the maximum of 896 MB if required. This is recommended especially in large environments, for example in systems with a large number of Logical Unit Numbers (LUNs).
⁠Configuring the Target Type
When a kernel crash is captured, the core dump can be either stored as a file in a local file system, written directly to a device, or sent over a network using the NFS (Network File System) or SSH (Secure Shell) protocol. Only one of these options can be set at the moment, and the default option is to store the vmcore file in the /var/crash/ directory of the local file system. To change this, as root, open the /etc/kdump.conf configuration file in a text editor and edit the options as described below.
To change the local directory in which the core dump is to be saved, remove the hash sign (“#”) from the beginning of the #path /var/crash line, and replace the value with a desired directory path. Optionally, if you want to write the file to a different partition, follow the same procedure with the #ext4 /dev/sda3 line as well, and change both the file system type and the device (a device name, a file system label, and UUID are all supported) accordingly. For example:
ext3 /dev/sda4
path /usr/local/cores
To write the dump directly to a device, remove the hash sign (“#”) from the beginning of the #raw /dev/sda5 line, and replace the value with a desired device name. For example:
raw /dev/sdb1
To store the dump to a remote machine using the NFS protocol, remove the hash sign (“#”) from the beginning of the #net my.server.com:/export/tmp line, and replace the value with a valid host name and directory path. For example:
net penguin.example.com:/export/cores
To store the dump to a remote machine using the SSH protocol, remove the hash sign (“#”) from the beginning of the #net user@my.server.com line, and replace the value with a valid user name and host name. For example:
net john@penguin.example.com
See Chapter 14, OpenSSH for information on how to configure an SSH server, and how to set up a key-based authentication.
For a complete list of currently supported targets, see Table 32.1, “Supported kdump targets”.
Note
When using Direct-Access Storage Devices (DASDs) as the kdump target, the devices must be specified in the /etc/dasd.conf file with other DASDs, for example:
0.0.2098
0.0.2198
0.0.2298
0.0.2398
Where 0.0.2298 and 0.0.2398 are the DASDs used as the kdump target.
Similarly, when using FCP-attached Small Computer System Interface (SCSI) disks as the kdump target, the disks must be specified in the /etc/zfcp.conf file with other FCP-Attached SCSI disks, for example:
0.0.3d0c 0x500507630508c1ae 0x402424aa00000000
0.0.3d0c 0x500507630508c1ae 0x402424ab00000000
0.0.3d0c 0x500507630508c1ae 0x402424ac00000000
Where 0.0.3d0c 0x500507630508c1ae 0x402424ab00000000 and 0.0.3d0c 0x500507630508c1ae 0x402424ac00000000 are the FCP-attached SCSI disks used as the kdump target.
See the Adding DASDs and Adding FCP-Attached Logical Units (LUNs) chapters in the Installation Guide for Red Hat Enterprise Linux 6 for detailed information about configuring DASDs and FCP-attached SCSI disks.
The vmcore.flat file must be converted
When transferring a core file to a remote target over SSH, the core file needs to be serialized for the transfer. This creates a vmcore.flat file in the /var/crash/ directory on the target system, which is unreadable by the crash utility. To convert vmcore.flat to a dump file that is readable by crash, run the following command as root on the target system:
~]# /usr/sbin/makedumpfile -R */tmp/vmcore-rearranged* < *vmcore.flat*
⁠Configuring the Core Collector
To reduce the size of the vmcore dump file, kdump allows you to specify an external application (that is, a core collector) to compress the data, and optionally leave out all irrelevant information. Currently, the only fully supported core collector is makedumpfile.
To enable the core collector, as root, open the /etc/kdump.conf configuration file in a text editor, remove the hash sign (“#”) from the beginning of the #core_collector makedumpfile -c --message-level 1 -d 31 line, and edit the command-line options as described below.
To enable the dump file compression, add the -c parameter. For example:
core_collector makedumpfile -c
To remove certain pages from the dump, add the -d value parameter, where value is a sum of values of pages you want to omit as described in Table 32.2, “Supported filtering levels”. For example, to remove both zero and free pages, use the following:
core_collector makedumpfile -d 17 -c
See the manual page for makedumpfile for a complete list of available options.
Table 32.2. Supported filtering levels

Option            Description
1          Zero pages
2          Cache pages
4          Cache private
8          User pages
16       Free pages
⁠Changing the Default Action
By default, when kdump fails to create a core dump, the root file system is mounted and /sbin/init is run. To change this behavior, as root, open the /etc/kdump.conf configuration file in a text editor, remove the hash sign (“#”) from the beginning of the #default shell line, and replace the value with a desired action as described in Table 32.3, “Supported actions”.
Table 32.3. Supported actions
Option            Description
reboot Reboot the system, losing the core in the process.
halt     Halt the system.
poweroff        Power off the system.
shell   Run the msh session from within the initramfs, allowing a user to record the core manually.
For example:
default halt
⁠Enabling the Service
To start the kdump daemon at boot time, type the following at a shell prompt as root:
chkconfig kdump on
This will enable the service for runlevels 2, 3, 4, and 5. Similarly, typing chkconfig kdump off will disable it for all runlevels. To start the service in the current session, use the following command as root:
service kdump start

Use of Logsave Command

logsave - save the output of a command in a logfile

Synopsis

logsave [ -asv ] logfile cmd_prog [ ... ]

Description

The logsave program will execute cmd_prog with the specified argument(s), and save a copy of its output to logfile. If the containing directory for logfile does not exist, logsave will accumulate the output in memory until it can be written out. A copy of the output will also be written to standard output.

If cmd_prog is a single hyphen ('-'), then instead of executing a program, logsave will take its input from standard input and save it in logfile
logsave is useful for saving the output of initial boot scripts until the /var partition is mounted, so the output can be written to /var/log.

Options

-a This option will cause the output to be appended to logfile, instead of replacing its current contents.

-s -This option will cause logsave to skip writing to the log file text which is bracketed with a control-A (ASCII 001 or Start of Header) and control-B (ASCII 002 or Start of Text). This allows progress bar information to be visible to the user on the console, while not being written to the log file.

-v This option will make logsave to be more verbose in its output to the user.

Sunday, April 23, 2017

What is Bonding and how to do it

What is Bonding

Linux network bonding is creation of a single bonded  interface by combining 2 or more Ethernet interfaces. This is done for multiple purposes like HA, load distributing etc.... Below are the some of modes of bonding. Based on the mode configured the respective device will act.

Modes of Bonding

Mode 0 – Round robin
Mode 1 – Active Backup
Mode 2 – Balance XOR
Mode 3 – Broadcast
Mode 4  - 802.3as
Mode 5 – Balance TLB
Mode 6 – Balance TLB

List of Confiuration files used -  /etc/modprobe.conf, /etc/sysconfig/network-scritps/ifcfg-bond0,   /proc/net/bonding/bond0

How to do Bonding


(i) Create a  file named <anything>.conf (as long as it ends with a .conf extension) in the /etc/modprobe.d/ directory or add below entry in /etc/modprobe.conf file.

alias bond0 bonding

Above line  creates a interface named bond0 for bonding. Now we need to configure IP address and other details for the interface which will be done by creating a new file file in network-scripts folder.

To be added in configuration file


Other options :  Options bond0 mode=1 miimon=100

(ii) Load the bond driver module from command prompt : modprobe bonding

(iii) Now the IP address and other details need to be added for bond0 device and other interface details need to be changed as slave.

** An Sample entry of  bond interface configuration files **

# Cat /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
IPADDR=192.168.1.1
NETMASK=255.255.255.0
ONBOOT=yes
BOOTPROTO=none
USERCTL=no

** An Sample entry of other interface configuration files **

# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none

# cat /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none

Note: No IP address should be mentioned in slave files

After configuring above details restart the network service and check the status of bonding.

cat  /proc/net/bonding/bond0

To verify whether bonding works, do an ifdown eth0 and check /proc/net/bonding/bond0 and check the current active slave. Meanwhile do a continuous ping to the bond0 IP from different machine and do a ifdown to active interface. Ping should not work.

How to  change slave device in bonding

There may be a situation where we need to change the active device of bond0 from eth0 to eth1. In that case we need to use ifenslave command to do the same.

# Ifenslave –c bond0 eth1   ; Changes the active to eth1
# Ifenslave –d  bond0 eth0  ; Removes eth0 from bonding
# Ifenslave  bond0 eth2

-a, - all-interfaces  show information about all interfaces.
-c, - Change active slave.
-d, - Removes slave interfaces from the bonding device.
-f,  - Force actions to the specified interfaces appears not to belong to an Ethernet device.
-u, - usage. Show usage information and exit.
-v, - verbose. Print warning and debug messages.
-V, - version. Show version information and exit.

It means that if one card fails-second starts to work. mode=1 in your file means the same. 

miimon:
Specifies the MII link monitoring frequency in milliseconds. This determines how often the link state of each slave is inspected for link failures. A value of zero disables MII link monitoring. A value of 100 is a good starting point. Somewhere I read that, it is deprecated. Not very sure. I think now bond driver uses netif_carrier_ok to find the link status of its slaves.

RAID

RAID -  Reduntant array of Independent Disks

Types of RAID

RAID 0
RAID 1
RAID 2
RAID 3
RAID 4
RAID 5
RAID 6
RAID 10
RAID 01

RAID 0
Block are stripped.
Minimum 2 disks
Excellent Performance
Don’t use for critical System

RAID 1
Minimum 2 Disks
Good performance
Excellent Redundancy


RAID 2

Bits Stripped
Uses Error correction codes
Minimum 2 Disks
Good performance
Excellent Redundancy


RAID 3
Bytes striped dedicated parity disk
Minimum 2 Disks
Dedicated disk to store parity and multiple disk to store data disk
Sequential read and write good random read and write worst performance


RAID 4

Blocks striped dedicated parity disk
Minimum 2 Disks
Dedicated disk to store parity and multiple disk to store data disk
Sequential read and write good random read and write worst performance

RAID 5

Minimum 3 disks
Good Performance
Good Redundancy
Best cost effective

RAID 6

Minimum 4 disks dual parity

RAID 01

Mirror of stripes
Requires minimum 3 hard disks

RAID 10

Stripe of mirrors

Sunday, April 16, 2017

How to calculate the usage of Block disks

# cat /sys/block/<dev>/stat

186908    41568  6033917  2408504    91198   509600  4882200  9406764        0  1161304 11848624

Field  1 -- Number of reads issued
This is the total number of reads completed successfully.

Field  2 -- Number of reads merged, field 6 -- Number of writes merged
Reads and writes which are adjacent to each other may be merged for efficiency.  Thus two 4K reads may become one 8K read before it is ultimately handed to the disk, and so it will be counted (and queued) as only one I/O.  This field lets you know how often this was done.

Field  3 -- Numberof sectors read - This is the total number of sectors read successfully.

Field  4 -- Number of milliseconds spent reading

Field  5 -- Total Number of writes completed 

Field  7 --  Total number of sectors written 

Field  8 -- # of milliseconds spent writing 

Field  9 -- # of I/Os currently in progress
The only field that should go to zero. Incremented as requests are given to appropriate request_queue_t and decremented as they finish.

Field 10 -- # of milliseconds spent doing I/Os
This field is increases so long as field 9 is nonzero.

Field 11 -- weighted # of milliseconds spent doing I/Os

This field is incremented at each I/O start, I/O completion, I/O merge, or read of these stats by the number of I/Os in progress (field 9) times the number of milliseconds spent doing I/O since the last update of this field.  This can provide an easy measure of both I/O completion time and the backlog that may be accumulating.

Name            units         description
----            -----         -----------
read I/Os       requests      number of read I/Os processed
read merges     requests      number of read I/Os merged with in-queue I/O
read sectors    sectors       number of sectors read
read ticks      milliseconds  total wait time for read requests
write I/Os      requests      number of write I/Os processed
write merges    requests      number of write I/Os merged with in-queue I/O
write sectors   sectors       number of sectors written
write ticks     milliseconds  total wait time for write requests
in_flight       requests      number of I/Os currently in flight
io_ticks        milliseconds  total time this block device has been active
time_in_queue   milliseconds  total wait time for all requests

Below command will print size in bytes to get the actual size in GB then divide  by 1024 thrice. This will give the total size of the particular partition or disk in bytes

cat /proc/diskstats


http://lxr.linux.no/linux+v3.5/Documentation/iostats.txt

Read more: http://linuxpoison.blogspot.nl/2009/02/how-to-measure-and-read-disk-activity.html#ixzz4c1GPbAPb

Date Command With all options

rpm -qa --last | grep -w "$(date --date='1 month ago' +'%b %Y')"

date -d "yesterday 13:00 " '+%Y-%m-%d'

SSH config files and commands


ssh_config - OpenSSH SSH client configuration files

List of Configuration files

For Key purposes


~/.ssh/id_dsa          Contains the DSA private key of the user.
~/.ssh/id_dsa.pub   The DSA public key of the user.
~/.ssh/id_rsa           The RSA private key used by ssh for version 2 of the SSH protocol.
~/.ssh/id_rsa.pub    The RSA public key used by ssh for version 2 of the SSH protocol.
~/.ssh/identity          The RSA private key used by ssh for version 1 of the SSH protocol.
~/.ssh/identity.pub   The RSA public key used by ssh for version 1 of the SSH protocol.
~/.ssh/known_hosts Contains DSA host keys of SSH servers accessed by the user. 

For Configuration purposes

a) ssh_config  - SSH client configuration file (i.e. is used by the ssh program itself). 
b) sshd_config - SSH daemon configuration file (i.e. is used by sshd)
c) /etc/pam.d/sshd- The PAM configuration file for the sshd daemon.
d) /etc/sysconfig/sshd  - Configuration file for the sshd service.

User-specific configuration files


a) ~/.ssh/authorized_keys      Holds a list of authorized public keys for servers. When the client connects to a server, the server authenticates the client by checking its signed public key stored within this file.

More info About configuration files

ssh_config: configuration file for the ssh client on the host machine you are running. For example, if you want to ssh to another remote host machine, you use a SSH client. Every settings for this SSH client will be using ssh_config, such as port number, protocol version and encryption/MAC algorithms.

sshd_config: configuration file for the sshd daemon (the program that listens to any incoming connection request to the ssh port) on the host machine. That is to say, if someone wants to connect to your host machine via SSH, their SSH client settings must match your sshd_config settings in order to communicate with you, such as port number, version and so on.

For example, most times we change the SSH port number for security reason. Take an example as follows:

For the SSH port number, ssh_config is manually set to be 1000 (decided by the remote host), sshd_config is set to be 5555. If anyone wants to connect to your host, they MUST set their ssh client port always to 1000. However, you will always use port 5555 as the default port (instead of the 22) to connect to a remote machine. If the remote machine uses another port or the standard port, you need to specify the port number in the command line,

ssh is the command you use to connect to remote machines - the client.
sshd is the daemon that is running and allows others to connect to the machine - the server.

Table 14.2. User-specific configuration files

You can edit the /etc/sysconfig/sshd file and change the AUTOCREATE_SERVER_KEYS variable:
AUTOCREATE_SERVER_KEYS=NO
or
AUTOCREATE_SERVER_KEYS=RSAONLY


Table 14.2. User-specific configuration files

1) ssh-copy-id

ssh-copy-id - install your public key in a remote machine’s authorized_keys
ssh-copy-id [-i [identity_file]] [user@]machine

ssh-copy-id  is  a  script that uses ssh to log into a remote machine (presumably using a login password, so password authentication should be enabled, unless you’ve done some clever use of multiple identiies) It also changes the permissions of the remote user’s home, ~/.ssh, and ~/.ssh/authorized_keys to remove group writability (which would otherwise prevent you from logging in.


1) sshYou can edit the /etc/sysconfig/sshd file and change the AUTOCREATE_SERVER_KEYS variable:
AUTOCREATE_SERVER_KEYS=NO
or

AUTOCREATE_SERVER_KEYS=RSAONLY