Linux Interview Questions

what is load average ?
Number of jobs in run queue are waiting for disk IO averaged over 1, 5 and 15 mins.
mpstat -P ALL 1 (produces all CPU's utilization for every 1 second)
while true; do true; done

debug cpu performance issue 

----------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------

How to get a UUID of san devices from file?
 ll /dev/disk/by-uuid/
 
 how to block particular lun using multipath.conf file
 Edit /etc/multipath/multipath.conf file and add your device in blacklist section
 blacklist {
      wwid 26353900f02796769
      devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
      devnode "^hd[a-z]"
}



 How to scan scsi and fc devices for new disks?

FC :  
# echo "1" > /sys/class/fc_host/host0/issue_lip
# echo "1" > /sys/class/fc_host/host1/issue_lip

SCSI
# echo "- - -" > /sys/class/scsi_host/host0/scan
# echo "- - -" > /sys/class/scsi_host/host1/scan

to get the list of fc adapters : ls /sys/class/fc_host

to get the wwn number :
cat /sys/class/fc_host/host0/node_name

to get the port number
cat /sys/class/fc_host/host0/port_name0x10000000c9538d83
cat /sys/class/fc_host/host1/port_name0x10000000c9538dac


scan the newly added lun
echo "1" > /sys/class/fc_host/host0/issue_lip
echo "- - -" > /sys/class/scsi_host/host0/scan

----------------------------------------------------------------------------------------------------------

 tool you use to check the system performance?
# top
# sar
# vmstat
# iostat

-----------------------------------------------------------------------------------------------------------

------------------------------
zombie process :
------------------------------
poorly written parent process might not call the wait () function when the child process is created.
SIGCHLD signal will be ignored or another application is affecting this process.

# top (to check the no of zombies)
# ps aux | egrep "Z|defunct"

# ps -o ppid= -p 7641

# ps -e | grep 7636

-------------------------------------------------------------------------------------------------------------

File descriptors :
--------------------
File descriptor 1 is the standard output (stdout).
File descriptor 2 is the standard error (stderr).
at first, 2>1 may look like a good way to redirect stderr to stdout.
However, it will actually be interpreted as "redirect stderr to a file named 1".
& indicates that what follows and precedes is a file descriptor and not a filename. So the construct becomes: 2>&1.

Consider >& as redirect merger operator.

--------------------------------------------------------------------------------------------------------------

sed command
---------------------

To extract lines one to four, we type this command:
sed -n '6,9p' coleridge.txt

We can use the -e (expression) option to make multiple selections. With two expressions, we can select two verses, like so:

sed -n -e '1,4p' -e '31,34p' coleridge.txt


occurrences of “day” to “week,” and give the mariner and albatross more time to bond:

sed -n 's/day/week/p' coleridge.txt

We have to add a “g” at the end of the expression, as shown below, to perform a global search so all matches in each line are processed:

sed -n 's/day/week/gp' coleridge.txt


We type the following, adding an i to the command at the end of the expression to indicate case-insensitivity:

sed -n 's/day/week/gip' coleridge.txt


We can achieve the same result if we use a semicolon (;)  to separate the two expressions, like so:

sed -n 's/motion/flutter/gip;s/ocean/gutter/gip' coleridge.txt  - two substitutes in single command.
sed -e '/swap/s/^/#/' - inserts # at the beginning of the line after matching the pattern.

 ------------------------------------------------------------------------------------------------------------------------

 CPU Softlockup and hardlockup :
----------------

A soft lockup is the symptom of a task or kernel thread using and not releasing a CPU for a longer period of time than allowed.

A hard lockup is when a CPU has interrupts disabled for a long time, which might block important communication between processes or hardware.
The default hard lockup threshold is 30 second on RHEL5/6 and 10 seconds on RHEL7.

---------------------------------------------------------------------------------------------------------------------------

 lvreduce:

-Umount the filesystem using umount command,
-use resize2fs command , e.g resiz2fs /dev/mapper/myvg-mylv 10G
-Now use the lvreduce command , e.g lvreduce -L 10G /dev/mapper/myvg-mylv

-----------------------------------------------------------------------------------------------------------

grub config file explained :

GRUB2 configuration file /boot/grub2/grub.cfg

Do not edit this file directly.
Use the grub2-mkconfig command to generate grub.cfg
This command uses the template scripts in /etc/grub.d and menu-configuration settings taken from /etc/default/grub

The /etc/grub2.cfg file is a symbolic link to /boot/grub2/grub.cfg

/etc/default/grub File :

# cat /etc/default/grub
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=cl/root rd.lvm.lv=cl/swap rhgb quiet"
GRUB_DISABLE_RECOVERY="true"

If changes are made to any of these parameters, you need to run grub2-mkconfig to re-generate the /boot/grub2/grub.cfg file.

# grub2-mkconfig –o /boot/grub2/grub.cfg

-------------------------------------------------------------------------------------------------------------------------

Rescue mode :
-------------------

Rescue mode : Three  ways
1) There is a GRUB2 menu option when you boot up the system which can be selected to directly boot into rescue mode.

2)During bootup, when the GRUB2 menu shows up, press the e key for edit.

 Add the following parameter at the end of the linux16 line:

systemd.unit=rescue.target

Press Ctrl+x to boot the system with the parameter.

To switch to Emergency target, simply run following command as root:

3) By booting the system from an installation boot CD-ROM.
- By booting the system from other installation boot media, such as USB flash devices or .iso files for a virtual machine.

Troubleshooting ---> Rescue a Linux machine

press 1 to continue.

chroot /mnt/sysimage



Bootup into Emergency mode(target) :

Emergency mode provides the most minimal environment possible and allows you to
repair your system even in situations when the system is unable to enter rescue mode.

In emergency mode, the system mounts the root file system as read-only,
does not attempt to mount any other local file systems, does not activate network interfaces.


1. During bootup, when the GRUB2 menu shows up, press the e key for edit.

Boot up screen grub2

2. Add the following parameter at the end of the linux16 line :

systemd.unit=emergency.target

Press Ctrl+x to boot the system with the parameter.

-------------------------------------------------------------------------------------------------------------------------

Re-install grub : 

boot into rescue mode.

change root directory using cmd  # chroot /mnt/sysimage

grub-install /dev/sda (majority of the scenarios, grub is installed on /dev/sda which is a boot device)

(you can check grub.conf to verify for the boot device from /dev/grub2/grub.conf file

grep boot=/dev /dev/grub2/grub.conf file.)

execute exit command twice to reboot.

 

-------------------------------------------------------------------------------------------------------------------------

rpmdb corrupted what would you do ?
 

There are a number of factors that can lead to the RPM database corruption, such as incomplete previous transactions,
installation of certain third-party software, removing specific packages, and many others.
backing up your current RPM database before proceeding (you might need it in the future), using the following commands.

# mkdir /backups/
# tar -zcvf /backups/rpmdb-$(date +"%d%m%Y").tar.gz  /var/lib/rpm
# rm -f /var/lib/rpm/__db*       
# /usr/lib/rpm/rpmdb_verify /var/lib/rpm/Packages
In case the above operation fails, meaning you still encounter errors, then you should dump and load a new database. Also verify the integrity of the freshly loaded Packages file as follows.

# cd /var/lib/rpm/
# mv Packages Packages.back
# /usr/lib/rpm/rpmdb_dump Packages.back | /usr/lib/rpm/rpmdb_load Packages

# rpm -qa
rebuild the RPM database using the following command
# rpm -vv --rebuilddb

-----------------------------------------------------------------------------------------------
/etc/resolv.conf entries getting erased what would you do ?
 

set peerdns=no in /etc/sysconfig/network-scripts/ifcfg-eth0  and stop the NetworkManager service.

------------------------------------------------------------------------------------------------
no nfs mounts in the server, df -h command getting hang what would you do ?
 

/etc/mtab file should be symbolic link to /proc/self/mounts.
Create the symbolic link again using ln -s.

You can even reinstall util-linux package as /etc/mtab is provided by it.
It will restore the link and permissions.
-------------------------------------------------------------------------------------------------

In db system, disk utilization is full but there are less number of files in the partition what would you do
how do you kill the files (cont from the above scenario)
 

lsof | egrep "deleted|COMMAND"
As it is a db system, we can not kill the process.
We need to truncate the file size (de-allocate the space used by that file)
echo > /proc/pid/fd/fd_number
eg:

$ file /proc/25575/fd/33
/proc/25575/fd/33: broken symbolic link to `/oradata/DATAPRE/file.dbf (deleted)'
$ echo > /proc/25575/fd/33
------------------------------------------------------------------------------------------------------

what is dracut ?

The tool named dracut is used to create a Linux boot image (initramfs) by copying tools and files from an installed system
and combining it with the Dracut framework, which is usually found in /usr/lib/dracut/modules.

How to resolve kernel panic error ?
This happens mostly because of initramfs.img file missing or corrupted

1.Boot the system in rescue mode
2.Log in using the root account
3.Navigate to /boot
4. See if initramfs.img file is available (If it is available, it must be corrupted
5.find kernel version (uname -r)
6.mkinitrd initramfs-kernel_version.img kernel_version
7.Boot the machine

-------------------------------------------------------------------------------------------------------

disk IO utilization is high, how do you trace and troubleshoot ?

Using iostat command we can trace on which  experiencing the high IO
using lsblk we can trace the exact partition.
using ps -ef and lsof we can trace which files are in usage.

----------------------------------------------------------------------------------------------------------

Pre-req to check before performing the vmotion :

The hosts must be licensed for vMotion.
The hosts must be running ESXi 5.1 or later.
The hosts must meet the networking requirement for vMotion. See vSphere vMotion Networking Requirements.
The virtual machines must be properly configured for vMotion. See Virtual Machine Conditions and Limitations for vMotion
Virtual machine disks must be in persistent mode or be raw device mappings (RDMs). See Storage vMotion Requirements and Limitations.
The destination host must have access to the destination storage.
When you move a virtual machine with RDMs and do not convert those RDMs to VMDKs, the destination host must have access to the RDM LUNs.
Consider the limits for simultaneous migrations when you perform a vMotion migration without shared storage. This type of vMotion counts against the limits for both vMotion and Storage vMotion, so it consumes both a network resource and 16 datastore resources.
----------------------------------------------------------------------------------------------------------
how do you enter into rescue mode ?
1) boot into rescue mode from installation DVD/ISO
boot with iso -- select troubleshooting -- rescue a centos linux system -- continue -- chroot /mnt/sysimage

2) There is a GRUB2 menu option when you boot up the system which can be selected to directly boot into rescue mode.

3) Edit the grub menu and append the following to linux16 line systemd.unit=rescue.target

4) systemctl rescue (it directly switches to rescue mode)
----------------------------------------------------------------------------------------------------------\
How all the users are able to change their respective passwords ?

The "/usr/bin/passwd" file is owned by user "root" and has the SETUID bit set. The process executes "/usr/bin/passwd" with effective user ID 0 (root) and so can change the contents of "/etc/shadow".
----------------------------------------------------------------------------------------------------------\
Can we reduce the size of xfs filesystem ? if yes, how ?

using xfs_reduce we can reduce. we need xfsdump to take backup of the partition before reducing.
 incase if its a LVM
 yum install xfsdump -y
 xfsdump -f /tmp/<partition_name.dmp> /data (backup the data)
 umount /<partition>
 lvreduce -L 400M /dev/vg00/lv00
 mkfs.xfs -f /dev/vg00/lv00
 mount /dev/vg00/lv00 /data
 xfsrestore -f /tmp/<partition_name> /data
 ls -l /<partition_name>
 
In some cases, the following workarounds may be acceptable:

    Use some backup/restore solution, such as xfsdump and xfsrestore. This may be useful especially if the filesystem is mostly empty.
-----------------------------------------------------------------------------------------------------------


how do you repair the xfs filesystem ?
# xfs_repair /mount/point
# xfs_repair /dev/mapper/vg_test-lv_test
# xfs_check /dev/mapper/vg_test-lv_test  (check for filesystem problems, but not fix any problems)
xfs_repair -n” command to do a dry run for xfs_repair.

---------------------------------------------------------------------------------------------------------------

how do you edit the grub in Linux explain procedure ?
1) boot into rescue mode from installation DVD/ISO
boot with iso -- select troubleshooting -- rescue a centos linux system -- continue -- chroot /mnt/sysimage

2) There is a GRUB2 menu option when you boot up the system which can be selected to directly boot into rescue mode.

3) Edit the grub menu and append the following to linux16 line systemd.unit=rescue.target

4) systemctl rescue (it directly switches to rescue mode)
---------------------------------------------------------------------------------------------------------------

How many ways are there to boot system into rescue mode ?
explained above
---------------------------------------------------------------------------------------------------------------
How do you activate the deactivated LVM ?
To deactivate the logical volume/ volume group

lvchange -an /dev/vg_name/lv_name
vgchange -an vg_name

to activate

lvchange -ay /dev/vg_name/lv_name
vgchange -ay vg_name
-----------------------------------------------------------------------------------------------------------------
/bin/mount file got deleted, after reboot now no mounts are getting mounted because of mount command not found, how do you restore ?
Anyway, we are going to fix the issue and make /bin's contents as close as is possible to where it was. The only difference would be some symbolic links which we will fix too.
boot the system into rescue mode
we can get a list of packages which have installed files in /bin using:
dpkg --search /bin | cut -f1 -d: | tr ',' '\n'
dpkg --listfiles PACKAGE-NAME | grep "^/bin/" # or awk '$0 ~ "^/bin/ (we can also use this to list files under /bin)
we simply create a list of all packages that are necessary to us, then download them and extract them to /bin with something like:
xargs apt download < list-packages
dpkg-deb -x PACKAGE .
mv ./bin/* /bin

we can make use of this script to restore multiple files
https://github.com/ravexina/restore-bin.git
-----------------------------------------------------------------------------------------------------------------

Using ping command how do we come to know it is windows or Linux machine ?
TTL value for Linux/Unix is 64, and TTL value for Windows is 128.

-----------------------------------------------------------------------------------------------------------------

ping uses which protocol ?
UDP
-----------------------------------------------------------------------------------------------------------------

boot procedure of linux role of explain initramfs in it ?
https://www.thegeekdiary.com/centos-rhel-7-booting-process/

-----------------------------------------------------------------------------------------------------------------

I want to boot the server into previous kernel, how do you do it and explain the ways ?
https://www.thegeekdiary.com/centos-rhel-7-change-default-kernel-boot-with-old-kernel/
-----------------------------------------------------------------------------------------------------------------

what is semaphore in Linux ?
a semaphore is a variable or abstract data type used to control access to a common resource by multiple threads and avoid critical section problems in a concurrent system such as a multitasking operating system.

Advantages of Semaphores

There is no resource wastage because of busy waiting in semaphores as processor time is not wasted unnecessarily to check if a condition is fulfilled to allow a process to access the critical section. Semaphores are implemented in the machine independent code of the microkernel.
----------------------------------------------------------------------------------------------------------------
what is a process and thread ?

A process is a program under execution
Processes require more time for context switching as they are more heavy.

A thread is a lightweight process that can be managed independently by a scheduler.
Threads require less time for context switching as they are lighter than processes.

-----------------------------------------------------------------------------------------------------------------