Kill Linux Processes with pkill

One of the best features in Linux is the way you can control processes from the command line, so if you have an application that locks up your GUI, you can always SSH over from another machine and just kill the offending process.

The problem is that if you are killing the same process repeatedly, it’s very tedious to have to figure out the process ID every single time so that you can kill it… so here’s the easier way to do it.

The Old Way

The classic way of killing processes meant you’d first need to use the ps command piped through grep to find the process you are trying to kill:

$ ps -ef | grep swiftfox
geek 7206 22694 0 Dec04 ? 00:00:00 /bin/sh /opt/swiftfox/swiftfox
geek 7209 7206 0 Dec04 ? 00:00:00 /bin/sh /opt/swiftfox/run-mozilla.sh /opt/swiftfox/swiftfox-bin
geek 7213 7209 0 Dec04 ? 00:04:29 /opt/swiftfox/swiftfox-bin
geek 14863 14224 0 18:19 pts/4 00:00:00 grep swiftfox

Then to kill the process, you’d have to use the kill command:

$ kill 7206

The New Way

Instead of going through all of that, you can simply use the pkill command if you already know the process name or part of it.

$ pkill swiftfox

It’s as simple as that. You should note that pkill will kill all processes matching the search text, in this case swiftfox

If you want to see what process names are matched before using the pkill command, you can use the pgrep command. Passing the -l switch tells pgrep to show the process name as well.

$ pgrep -l swiftfox
7206 swiftfox
7213 swiftfox-bin

Delete Files Older Than x Days

The find utility on linux allows you to pass in a bunch of interesting arguments, including one to execute another command on each file. We’ll use this in order to figure out what files are older than a certain number of days, and then use the rm command to delete them.

find /path/to/files* -mtime +5 -exec rm {} \;

Note that there are spaces between rm, {}, and \;

The first argument is the path to the files. This can be a path, a directory, or a wildcard as in the example above. I would recommend using the full path, and make sure that you run the command without the exec rm to make sure you are getting the right results.
The second argument, -mtime, is used to specify the number of days old that the file is. If you enter +5, it will find files older than 5 days.
The third argument, -exec, allows you to pass in a command such as rm. The {} \; at the end is required to end the command.

How to use the most popular command in Unix - Grep

grep is by far the most popular command that exists in Unix. Though some may argue about that, but once you begin using grep, it would always be present in all your complex commands that you think of executing at the shell prompt. grep stands for 'global regular expression printer' . Which makes no sense to most.. In sensible words grep extracts those lines from a given text that match the conditions set by the user.

Basically grep lets you enter a pattern of text and then it searches for this pattern within then text that you provide it. It would return all the lines that have the given pattern in them. grep can be used in 2 ways - Either on its own or along with pipes

Using grep on its own

$ grep '12.00' /home/david/backup/log.txt

This command basically shows how you can use grep to extract lines containing a particular string from a text file. (Text files need not necessarily end in .txt) The above command searches for the string 12.00 in the text file specified in the command, and displays all the lines that have this string in them.
The above command could be used to find out all the backups that took place at 12.00 (In case you have a log.txt file in that directory with a list of all the timings for the backups that you have made).

$ grep -v '12.00' /home/david/backup/log.txt

The above command would now show you all the lines in the text file except those that have the string 12.00 in them.

$ grep -l 'delay' /code/*.c

The above command searches for those files that end with a '.c' (within the /code directory) and in which the text 'delay' is present. It only returns the names of these files and not the lines where it found the string.

$ grep -w '\' *

The above commands search for text in a more refined way. The first command searches for those lines where any word in that line begins with the letters 'bay' and the second command searches for those lines where any word in that line ends with the letter 'watch'
-

Using grep with pipes

$ ls -l | grep rwxrwxrwx

As you must be knowing ls -l displays the directory listing for any directory. The grep rwxrwxrwx part of the command extracts only those lines which display the files having their read,write,execute permissions set for user, group and others also. Thus instead of getting a listing of all the files in the directory, you would only see those files that have their r,w,x permissions set for all everybody.

The output of grep can also be piped to another program as follows

$ du | grep 'mp3' | more

You should be able to figure out what the above command does..

$ grep '^#' /home/david/script1 | more

The above command would display those lines (from the file /home/david/script1) that begin with a '#'. The term '^#' means that # should be present as the first character on a line. The more part of the command should be known to you. If not, more basically displays the output a page at a time incase the output exceeds one page.

$ grep -v '^[0-9]' /home/david/backup/log.txt | more

This command searches for lines having any of the numbers from 0-9 in them as the first character on the line. It then prints all the lines except the ones it found initially.

Important : It's necessary to enclose patterns (as used in the above 2 commands) in single quotes so that the shell understands it correctly.Otherwise, the shell may interpret it in another method.

Some extra options for grep

-v
Reverses the normal behaviour of the grep command - Instead of selecting lines, it rejects the lines that match the given criteria.

-c
It supresses the normal output and only prints the total count of matching lines instead of the actual lines.

-i
Ignores the case of the text when matching the given pattern.

-w
Checks if the given pattern is a word by itself and not a part of another word. Thus if you search for 'bay' and the word 'baywatch' is present in a file, the particular line conatining that word would not be returned in the result.

-l
Only gives the names of the files in which the given pattern was found.

-r
Checks for the given pattern , recursively within the directory that you specify after the -r option

I hope this tutorial helps you get started with grep. grep is defintely one of the tools that gives Linux the advantage over other Operating Systems. Using grep effectively along with other tools gives the user a lot of power in Unix.

Setting the PATH in linux

This article explains how set your PATH variable under Linux. This has the same use as that of setting the PATH variable under DOS. Under Linux too, modifying the PATH would add these new directories to your default search path.
So in case you have a particular executable in a particular directory, then if you add that directory to your PATH, then you would only have to type the name of the executable at the prompt rather than then absolute path for that executable. Got it?? Read the example below to figure out what exactly I am speaking about..


Assumption

Suppose you have a program by the name 'tetris' in a folder called /usr/local/games . So in order to run this program you would have to type the following at the prompt

$/usr/local/games/tetris

The above command would execute your program. But typing this every time you want to play this wonderful game makes it slightly cumbersome. It would be much better if you could only type 'tetris'.


Solution

A solution would be to add the /usr/local/games directory to your PATH, so that next time onwards you would only have to type 'tetris' at the prompt rather than the absolute path.

To add this directory to your PATH you have to edit a file called 'bash_profile' that would be present in your Home directory (in Redhat Linux 6.2). So if there is a user by the name David then this file would mostly be found at /home/David/.bash_profile

Note : The period (.) before the name of the file. This period make this file a hidden file. So remember to view hidden files also while seeing a directory listing (This option would be in some menu in X , at the prompt simply use 'ls -a' to see hidden files).

This file would be having a particular line starting with the string PATH. For e.g. the file that I have on my machine has a line such as

PATH=/optional/bin:$PATH:$HOME/bin

To add the directory /usr/local/games to this I would have to modify this line as follows

PATH=/usr/local/games:optional/bin:$PATH:$HOME/bin

Once you have modified this file, save it and then execute it as follows

. $HOME/.bash_profile

Note : To execute this script basically at the $ prompt type a period ' . ' leave a space and then type $HOME/ Once this is done press key. Doing so would replace what ever you have typed with the path to your home directory. Once this happens all you have to do is append a .bash_profile to what is already present at the prompt and finally press

On executing the script you wouldn't see any messages at the output, but then onwards you could simply type 'tetris' at the prompt to execute the program /usr/local/games/tetris
So now you are on the right PATH

Unmount busy drives

You are probably all too familiar with the situation - you are trying to unmount a drive, but keep getting told by your system that it's busy. But what application is tying it up? A quick one-liner will tell you:

lsof +D /mnt/windows

This will return the command and process ID of any tasks currently accessing the /mnt/windows directory. You can then locate them, or use the kill command to finish them off.

Grabbing a screenshot without X

There are plenty of screen-capture tools, but a lot of them are based on X. This leads to a problem when running an X application would interfere with the application you wanted to grab - perhaps a game or even a Linux installer. If you use the venerable ImageMagick import command though, you can grab from an X session via the console. Simply go to a virtual terminal (Ctrl+Alt+F1 for example) and enter the following:

chvt 7; sleep 2; import -display :0.0 -window root sshot1.png; chvt 1;

The chvt command changes the virtual terminal, and the sleep command gives it a while to redraw the screen. The import command then captures the whole display and saves it to a file before the final chvt command sticks you back in the virtual terminal again. Make sure you type the whole command on one line.

This can even work on Linux installers, many of which leave a console running in the background - just load up a floppy/CD with import and the few libraries it requires for a first-rate run-anywhere screen grabber.

Batch resize images using the command line in Linux

If you have a ton of images that need resizing, you can do it all with the imagemagick package:

cd /home/user/images
mkdir resized_images
cp /home/user/images/* /home/user/images/resized_images

Now that you have a copy of the files in resized_images, time to resize them all:

mogrify -resize 800 *.jpg

This will resize them all to a width of 800px while keeping the aspect ratio. If you want a fixed image size, you can specify it like this:

mogrify -resize 800×600! *.jpg

block consecutive IP address using scripts

How to block long list of consecutive IP address?
How to call linux route command inside a script?
How to block consecutive IP address using bash script or perl script?
How to block local IP address permanently?

The are times that a server does not need to listen and process any TCP/UDP request for a long list of consecutive local IP addresses.

This blog entry provides a starting point of creating server scripts to block a long list of consecutive IP address from the server for permanent blocking.

To start, launch your fave editor and create a IPblock.sh bash script like with contents similar to the next few lines. This blog entry assumes that you have bash shell and perl currently installed from the machine.
From below example, we are permanently blocking IP address from

192.168.0.10 to 192.168.0.254.

Here's a simple sample script that does the job.

#!/bin/bash
echo Blocking started ...

for ((i=10;i<=254;i=i+1)); do /sbin/route add -host 192.168.0.$i reject done echo Done

This can also be accomplished using perl script which does the same function. Create a separate IPblock.pl perl script like so.

#!/usr/bin/perl -w

my $i;
for ($i=10; $i<=254; $i++ ) { system ("/sbin/route del -host 192.168.0.$i reject"); } }

Make sure these scripts are root executable like so

# chmod 700 IPblock.sh
# chmod 700 IPblock.pl

Now, to execute individually

# ./IPblock.sh
# ./IPblock.pl

Additionally, the above scripts can be scheduled for regular execution if you need them so by using crontab utility.

du - the disk usage linux command

du stands for disk usage. This simple linux command provides a summary of harddisk or storage space disk usage. It has many parameter arguments that can provide results in many screen format. du command can also summarize files and directories in a recursive manner.

Here are several usage of to use the du (disk usage) command.

# cd /home/vertito

To list the files and directories from there

# ls -la
-rw-r--r-- 1 root root 29 2007-08-11 11:57 file.txt
drwxr-xr-x 2 root root 4096 2007-08-11 11:57 folder1

Show summary in bytes

# du -b
4096 ./folder1
8221 .
# du -a
4 ./file.txt
4 ./folder1
12 .

Now, let us get a more human readable results

# du -ah
4.0K ./file.txt
4.0K ./folder1
12K .

The above shows that my file.txt has about 4K of filesize rounded to nearest power of 1024K including . an ..

Now, let us it in bytes
# du -ab
29 ./file.txt
4096 ./folder1
8221 .

The above is the same results you get from issuing ls -la command. 8221 is . and ..

Now let us do it once again in human readable form

# du -abh
29 ./file.txt
4.0K ./folder1
8.1K .

You can also exclude file glob pattern or shell expression for files like so

# du -abh --exclude='file.txt'
# du -abh --exclude='*.txt'
4.0K ./folder1
8.0K .

Recursive directory disk usage summary can also be achieved by doing the default usage without any parameters

# cd /home
# du

You can also limit the recursive search dept like so

# du --max-depth=2

which search on the 2nd level of directory only and ignores any folder found above the 2nd level folders.

Getting the summarized return in a human readable form

# du -sh

Alternatively if you wish to get the last time modification

# du -ah --time
4.0K 2007-08-11 11:57 ./file.txt
4.0K 2007-08-11 11:57 ./folder1
12K 2007-08-11 11:57 .

If you are using mbox type of mail storage handling, these commands can be handy checking and reporting partition and/or folder disk usage when incorporated inside a shell scripts. Furthermore, you can create and generate your TOP 10 users with largest mails on monthly or weekly basis that could give you more detailed email report and alerts from it..

At regular interval and again using a script, you can also watch and monitor folder/partition usage changes and alerts you for certain specified thresholds like for /home or /var/ftp or /tmp.

Insert a file at a specific line and column

How to insert the complete contents of a text file at a specific row and column of another text file.

If we are merely concerned with inserting after a specific line, it can be readily achieved with a number of Linux tools. For example, to insert file1.txt after the second line of file2.txt, any of the following commands will do:

$ sed -i '2r file1.txt' file2.txt
$ awk '{print} NR==2 {while (getline < "file1.txt") print}' file2.txt

Unlike the previous sed command which modifies file2 in-line, the above awk command writes the desired output to standard output only.

$ emacs -batch +3 file2.txt --insert file1.txt -f save-buffer -kill

This uses the batch capability of the emacs text editor. The command opens file2 at line 3, inserts file1, saves and then exits.

$ vi +2 file2.txt << DELIM > :r file1.txt
> :wq
> DELIM
Vim: Warning: Input is not from a terminal

Note that after you enter the first line ("vi +2 ..."), you will be prompted for more input. At that time, you enter the next three lines (:r, :wq, DELIM)
While Linux has many utilities to manipulate text (sed, perl, awk, cut, python), the easiest way I can think of to accomplish my objective to insert at a target line and column is using emacs.

$ emacs -batch -Q +2:3 file2.txt --insert file1.txt -f save-buffer -kill 2>/dev/null

The key is +2:3 which directs the editor to open the file at line 2 column 3.

Two other components in the above emacs command warrant some explanation. First, -Q means quick startup. Quick, because emacs won't load any init file, or any splash file. Second, the last part of the command pipes any standard error output from the emacs editor to the null device.

I don't doubt that sed, awk or perl can do the job. If you have a simple solution, please share with us via the comment feature of this web page. Many thanks.

Keeping Command History across Multiple Sessions

The bash shell maintains a history of the commands you entered. You can re-execute a command by recalling it from the history, without having to re-type it.

The command history is stored in a file, specified by an environment variable.

$ echo $HISTFILE
/home/peter/.bash_history

The maximum number of commands that it will save depends on the value of this environment variable:

$ echo $HISTSIZE
500

Life is simple if we operate on a single shell session at any given time. If you have 2 simultaneous sessions, you may be surprised that the history does not have the commands you expect.

The default behavior is that the command history is saved ONLY on session exit. Moreover, the existing contents in the history are overwritten by the new contents.
Assuming you already have a session running, opening a second session will not have the latest commands from the first session. This is because the first shell hasn’t exited yet. Ultimately, the contents of the history file depends on which session exits last, hence overwriting the commands from the session that finishes earlier.

This can become quite confusing.
The remedy is simple. Change the history behavior as follows:
Append commands to the history file, rather than overwrite it.

$ shopt -s histappend

Save each command right after it has been executed, not at the end of the session.

$ PROMPT_COMMAND='history -a'

Insert the 2 statements into ~/.bashrc

shopt -s histappend
PROMPT_COMMAND='history -a'

Note: If the PROMPT_COMMAND is already initialized, then you probably want to concatenate the

history -a part to what is already there.

To determine if PROMPT_COMMAND has a value:

echo $PROMPT_COMMAND

To concenate:

PROMPT_COMMAND="history -a;$PROMPT_COMMAND"

source: http://linuxcommando.blogspot.com

Display the top ten running processes sorted by memory usage

$ ps aux | sort -nk +4 | tail

This is certainly not the best way to display the top ten processes that consume the most memory, but, hey, it works.

It takes the output of ps aux, sorts it by 4th column numerically and then uses tail to output the last then lines which happen to be the processes with the biggest memory consumption.

If I was to find out who consumes the most memory, I'd simply use htop or top and not ps.

Top for files

$ watch -d -n 1 'df; ls -FlAt /path'

This one-liner watches for file changes in directory /path. It uses the watch command that executes the given command periodically. The -d flag tells watch to display differences between the command calls (so you saw what files get added or removed in /path). The -n 1 flag tells it to execute the command every second.

The command to execute is df; ls -FlAt /path that is actually two commands, executed one after other. First, df outputs the filesystem disk space usage, and then ls -FlAt lists the files in /path. The -F argument to ls tells it to classify files, appending */=>@| to the filenames to indicate whether they are executables *, directories /, sockets =, doors >, symlinks @, or named pipes |. The -l argument lists all files, -A hides . and .., and -t sorts the files by time.

Special note about doors - they are Solaris thing that act like pipes, except they launch the program that is supposed to be the receiving party. A plain pipe would block until the other party opens it, but a door launches the other party itself.

Actually the output is nicer if you specify -h argument to df so it was human readable. You can also join the arguments to watch together, making them -dn1. Here is the final version:

$ watch -dn1 'df -h; ls -FlAt /path'

Convert UTF-8 to WIN-1251

Function to convert UTF-8 to WIN-1251 charset.

function iconv-win1251 {

    if [ ${1%\.*} == ${1##*\.} ]; then

newfile="$1-win1251"

    else

newfile="${1%\.*}-win1251.${1##*\.}"

    fi

    iconv $1 --from-code UTF-8 --to-code WINDOWS-1251  > $newfile

    ls -l $newfile $1

}

Remote network restart

I guess anyone who's administered several remote boxes has had the unfortunate problem of (when not thinking straight) taking down the network card on a machine you have no physical access to. The result being that the ssh session you used to connect dies. The typical mistake is to do something like (as root):

ifconfig eth0 down; ifconfig eth0 inet 123.4.5.6; ifconfig eth0 up

The unfortunate result being that the first statement disconnects your session and hangs up the chain resulting in the network not coming back up. A nice way around this is to use the bash "disown" builtin command, ie:

(sleep 5; ifconfig eth0 inet 123.4.5.6; ifconfig eth0 up)& disown -h $! ; ifconfig eth0 down

In this case you launch a backgrounded task that is disconneced from the session (meaning the ssh session dying won't kill the process) which sleeps for 5 seconds (to give the down a chance to happen) then configures the network card as appropriate and brings it back up. As soon as this launches and is disowned, then immediately takes the network card down. If the configuration change keeps the IP address the same, you'll find that after 5 seconds your bash prompt just comes back and the session resumes.

Debug errors in configure procedures

There are a lot of parameters to adjust the installation. Unfortunately the configure doesn't end with success. A dubios error message appears and I don't find out what's wrong.
Also the modern oracle 'google' doesen't give a hint. But after hours I find a quick and easy solution:

sh -x ./configure ... configure_options ...

First there is a long rolling of letters, but afterwards, when the error appears, you can easy find out what the mistake is, because the debug modus tells you.

Import ssh host keys without verification

Automatically import host keys for cluster of machines named 'all'

Using the 'dsh' command from the clusterit tools - http://sourceforge.net/projects/clusterit

RCMD_CMD_ARGS='-o VerifyHostKeyDNS=yes -o StrictHostKeyChecking=no' dsh -g all -e true

Disk imaging with netcat and dd

Want to create a disk image of a system but write it on another hard disk? This can easily be done with the help of netcat and dd.

For this example you will need two computers connected on the same network, and enough room on one machine to hold your disk image

Destination Machine
So we’ll start off this example by preparing our destination machine to listen on tcp port 4444 via netcat. The port is arbitrary so you can really pick any port that is not being used. Just have to make sure that its the same on both ends.

root@tree:~# netcat -l -p 4444 | dd of=remote-machine.img

Source Machine
Next we’ll start a dd on the source machine and pipe it to netcat on port 4444

root@leaf:~# dd if=/dev/sda1 | netcat destination-machine-ip 4444

Now sit back and wait for your image to be done, when it’s finished dd will print out its status something like

NOTE: you will have to push CTRL+C to cancel out after this is completed, as the netcat session will still be active.

root@leaf:~#
30820468+71926 records in
30867456+0 records out
15804137472 bytes (16 GB) copied, 739.395 s, 21.4 MB/s
^C

If you want to find out the status of dd during the copy theres a couple of ways to do this, open up the system monitor in Ubuntu Linux, and it should tell you the transfer rate. Launch iostat or ifstat through a terminal. Invoke a command from terminal to get dd to display the current progress .

Viola, we’ll now have a dd image of our disk or partition. I like to verify the exact size of the file matches the size output from fdisk.

Destination Machine

root@root:~# ls -la remote-machine.img
-rw-r--r-- 1 root root 15804137472 2010-02-04 10:53 remote-machine.img

Source Machine

root@leaf:~# fdisk -l /dev/sda
Disk /dev/sda: 15.8 GB, 15804137472 bytes
255 heads, 63 sectors/track, 1921 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

IPTABLES - blocking IPs simplified

You can block an IP from IPTABLES by using

iptables -I INPUT -s 192.168.0.88 -j DROP

You can simplify things a little bit by using a shell script and a predefined text file containing the IPs we want to be blocked. Either create a new file or add the following at the end of the script which activates the firewall:

#!/bin/sh
for i in $(< bad_hosts.lst) ; do
iptables -I INPUT -i eth1 -s “$i” -j DROP
done

Now create a new file in the same directory and name it bad_hosts.lst and add a new IP to be blocked on every single line, like in the example below:
192.168.2.99
192.168.2.67
86.138.2.7

Check laptop battery in Command Line

If you run Linux off a laptop, you might want to check your battery status alt least once a year:

grep -F capacity: /proc/acpi/battery/BAT0/info

This will output something like

design capacity: 7800 mAh
last full capacity: 6414 mAh