block consecutive IP address using scripts

How to block long list of consecutive IP address?
How to call linux route command inside a script?
How to block consecutive IP address using bash script or perl script?
How to block local IP address permanently?

The are times that a server does not need to listen and process any TCP/UDP request for a long list of consecutive local IP addresses.

This blog entry provides a starting point of creating server scripts to block a long list of consecutive IP address from the server for permanent blocking.

To start, launch your fave editor and create a IPblock.sh bash script like with contents similar to the next few lines. This blog entry assumes that you have bash shell and perl currently installed from the machine.
From below example, we are permanently blocking IP address from

192.168.0.10 to 192.168.0.254.

Here's a simple sample script that does the job.

#!/bin/bash
echo Blocking started ...

for ((i=10;i<=254;i=i+1)); do /sbin/route add -host 192.168.0.$i reject done echo Done

This can also be accomplished using perl script which does the same function. Create a separate IPblock.pl perl script like so.

#!/usr/bin/perl -w

my $i;
for ($i=10; $i<=254; $i++ ) { system ("/sbin/route del -host 192.168.0.$i reject"); } }

Make sure these scripts are root executable like so

# chmod 700 IPblock.sh
# chmod 700 IPblock.pl

Now, to execute individually

# ./IPblock.sh
# ./IPblock.pl

Additionally, the above scripts can be scheduled for regular execution if you need them so by using crontab utility.

du - the disk usage linux command

du stands for disk usage. This simple linux command provides a summary of harddisk or storage space disk usage. It has many parameter arguments that can provide results in many screen format. du command can also summarize files and directories in a recursive manner.

Here are several usage of to use the du (disk usage) command.

# cd /home/vertito

To list the files and directories from there

# ls -la
-rw-r--r-- 1 root root 29 2007-08-11 11:57 file.txt
drwxr-xr-x 2 root root 4096 2007-08-11 11:57 folder1

Show summary in bytes

# du -b
4096 ./folder1
8221 .
# du -a
4 ./file.txt
4 ./folder1
12 .

Now, let us get a more human readable results

# du -ah
4.0K ./file.txt
4.0K ./folder1
12K .

The above shows that my file.txt has about 4K of filesize rounded to nearest power of 1024K including . an ..

Now, let us it in bytes
# du -ab
29 ./file.txt
4096 ./folder1
8221 .

The above is the same results you get from issuing ls -la command. 8221 is . and ..

Now let us do it once again in human readable form

# du -abh
29 ./file.txt
4.0K ./folder1
8.1K .

You can also exclude file glob pattern or shell expression for files like so

# du -abh --exclude='file.txt'
# du -abh --exclude='*.txt'
4.0K ./folder1
8.0K .

Recursive directory disk usage summary can also be achieved by doing the default usage without any parameters

# cd /home
# du

You can also limit the recursive search dept like so

# du --max-depth=2

which search on the 2nd level of directory only and ignores any folder found above the 2nd level folders.

Getting the summarized return in a human readable form

# du -sh

Alternatively if you wish to get the last time modification

# du -ah --time
4.0K 2007-08-11 11:57 ./file.txt
4.0K 2007-08-11 11:57 ./folder1
12K 2007-08-11 11:57 .

If you are using mbox type of mail storage handling, these commands can be handy checking and reporting partition and/or folder disk usage when incorporated inside a shell scripts. Furthermore, you can create and generate your TOP 10 users with largest mails on monthly or weekly basis that could give you more detailed email report and alerts from it..

At regular interval and again using a script, you can also watch and monitor folder/partition usage changes and alerts you for certain specified thresholds like for /home or /var/ftp or /tmp.

Insert a file at a specific line and column

How to insert the complete contents of a text file at a specific row and column of another text file.

If we are merely concerned with inserting after a specific line, it can be readily achieved with a number of Linux tools. For example, to insert file1.txt after the second line of file2.txt, any of the following commands will do:

$ sed -i '2r file1.txt' file2.txt
$ awk '{print} NR==2 {while (getline < "file1.txt") print}' file2.txt

Unlike the previous sed command which modifies file2 in-line, the above awk command writes the desired output to standard output only.

$ emacs -batch +3 file2.txt --insert file1.txt -f save-buffer -kill

This uses the batch capability of the emacs text editor. The command opens file2 at line 3, inserts file1, saves and then exits.

$ vi +2 file2.txt << DELIM > :r file1.txt
> :wq
> DELIM
Vim: Warning: Input is not from a terminal

Note that after you enter the first line ("vi +2 ..."), you will be prompted for more input. At that time, you enter the next three lines (:r, :wq, DELIM)
While Linux has many utilities to manipulate text (sed, perl, awk, cut, python), the easiest way I can think of to accomplish my objective to insert at a target line and column is using emacs.

$ emacs -batch -Q +2:3 file2.txt --insert file1.txt -f save-buffer -kill 2>/dev/null

The key is +2:3 which directs the editor to open the file at line 2 column 3.

Two other components in the above emacs command warrant some explanation. First, -Q means quick startup. Quick, because emacs won't load any init file, or any splash file. Second, the last part of the command pipes any standard error output from the emacs editor to the null device.

I don't doubt that sed, awk or perl can do the job. If you have a simple solution, please share with us via the comment feature of this web page. Many thanks.

Keeping Command History across Multiple Sessions

The bash shell maintains a history of the commands you entered. You can re-execute a command by recalling it from the history, without having to re-type it.

The command history is stored in a file, specified by an environment variable.

$ echo $HISTFILE
/home/peter/.bash_history

The maximum number of commands that it will save depends on the value of this environment variable:

$ echo $HISTSIZE
500

Life is simple if we operate on a single shell session at any given time. If you have 2 simultaneous sessions, you may be surprised that the history does not have the commands you expect.

The default behavior is that the command history is saved ONLY on session exit. Moreover, the existing contents in the history are overwritten by the new contents.
Assuming you already have a session running, opening a second session will not have the latest commands from the first session. This is because the first shell hasn’t exited yet. Ultimately, the contents of the history file depends on which session exits last, hence overwriting the commands from the session that finishes earlier.

This can become quite confusing.
The remedy is simple. Change the history behavior as follows:
Append commands to the history file, rather than overwrite it.

$ shopt -s histappend

Save each command right after it has been executed, not at the end of the session.

$ PROMPT_COMMAND='history -a'

Insert the 2 statements into ~/.bashrc

shopt -s histappend
PROMPT_COMMAND='history -a'

Note: If the PROMPT_COMMAND is already initialized, then you probably want to concatenate the

history -a part to what is already there.

To determine if PROMPT_COMMAND has a value:

echo $PROMPT_COMMAND

To concenate:

PROMPT_COMMAND="history -a;$PROMPT_COMMAND"

source: http://linuxcommando.blogspot.com

Run commands on logout

If a file named $HOME/.logout (a file named .logout in your home directory) exists, and the following trap statement is in your .profile, .logout is executed when you logout.

Add this to .profile:

trap "$HOME/.logout" 0

Top for files

$ watch -d -n 1 'df; ls -FlAt /path'

This one-liner watches for file changes in directory /path. It uses the watch command that executes the given command periodically. The -d flag tells watch to display differences between the command calls (so you saw what files get added or removed in /path). The -n 1 flag tells it to execute the command every second.

The command to execute is df; ls -FlAt /path that is actually two commands, executed one after other. First, df outputs the filesystem disk space usage, and then ls -FlAt lists the files in /path. The -F argument to ls tells it to classify files, appending */=>@| to the filenames to indicate whether they are executables *, directories /, sockets =, doors >, symlinks @, or named pipes |. The -l argument lists all files, -A hides . and .., and -t sorts the files by time.

Special note about doors - they are Solaris thing that act like pipes, except they launch the program that is supposed to be the receiving party. A plain pipe would block until the other party opens it, but a door launches the other party itself.

Actually the output is nicer if you specify -h argument to df so it was human readable. You can also join the arguments to watch together, making them -dn1. Here is the final version:

$ watch -dn1 'df -h; ls -FlAt /path'

Copy and paste from the command line

Add the following alias and function to your profile to be able to copy and paste files at the command line:

ccopy(){ cp $1 /tmp/ccopy.$1; }
alias cpaste="ls /tmp/ccopy* | sed 's|[^\.]*.\.||' | xargs -I % mv /tmp/ccopy.% ./%"

You can see below how this can be used:

blackbird:~/tst tks1$ ls
1.txt 2.txt t1.tss t2.tss t3.tss
blackbird:~/tst tks1$ ccopy 1.txt
blackbird:~/tst tks1$ ccopy 2.txt
blackbird:~/tst tks1$ cd ../tst2
blackbird:~/tst2 tks1$ ls
blackbird:~/tst2 tks1$ cpaste
blackbird:~/tst2 tks1$ ls
1.txt 2.txt

Convert UTF-8 to WIN-1251

Function to convert UTF-8 to WIN-1251 charset.

function iconv-win1251 {

    if [ ${1%\.*} == ${1##*\.} ]; then

newfile="$1-win1251"

    else

newfile="${1%\.*}-win1251.${1##*\.}"

    fi

    iconv $1 --from-code UTF-8 --to-code WINDOWS-1251  > $newfile

    ls -l $newfile $1

}

Revert files with changed mode

Sometimes (due to mismanagement!) end up with files in a git repo which have had their modes changed, but not their content. This one-liner lets me revert the mode changes, while leaving changed-content files be, so I can commit just the actual changes made.

git diff --numstat | awk '{if ($1 == "0" && $1 == "0") print $3}'  | xargs git checkout HEAD

Remote network restart

I guess anyone who's administered several remote boxes has had the unfortunate problem of (when not thinking straight) taking down the network card on a machine you have no physical access to. The result being that the ssh session you used to connect dies. The typical mistake is to do something like (as root):

ifconfig eth0 down; ifconfig eth0 inet 123.4.5.6; ifconfig eth0 up

The unfortunate result being that the first statement disconnects your session and hangs up the chain resulting in the network not coming back up. A nice way around this is to use the bash "disown" builtin command, ie:

(sleep 5; ifconfig eth0 inet 123.4.5.6; ifconfig eth0 up)& disown -h $! ; ifconfig eth0 down

In this case you launch a backgrounded task that is disconneced from the session (meaning the ssh session dying won't kill the process) which sleeps for 5 seconds (to give the down a chance to happen) then configures the network card as appropriate and brings it back up. As soon as this launches and is disowned, then immediately takes the network card down. If the configuration change keeps the IP address the same, you'll find that after 5 seconds your bash prompt just comes back and the session resumes.

Debug errors in configure procedures

There are a lot of parameters to adjust the installation. Unfortunately the configure doesn't end with success. A dubios error message appears and I don't find out what's wrong.
Also the modern oracle 'google' doesen't give a hint. But after hours I find a quick and easy solution:

sh -x ./configure ... configure_options ...

First there is a long rolling of letters, but afterwards, when the error appears, you can easy find out what the mistake is, because the debug modus tells you.

Import ssh host keys without verification

Automatically import host keys for cluster of machines named 'all'

Using the 'dsh' command from the clusterit tools - http://sourceforge.net/projects/clusterit

RCMD_CMD_ARGS='-o VerifyHostKeyDNS=yes -o StrictHostKeyChecking=no' dsh -g all -e true

Disk imaging with netcat and dd

Want to create a disk image of a system but write it on another hard disk? This can easily be done with the help of netcat and dd.

For this example you will need two computers connected on the same network, and enough room on one machine to hold your disk image

Destination Machine
So we’ll start off this example by preparing our destination machine to listen on tcp port 4444 via netcat. The port is arbitrary so you can really pick any port that is not being used. Just have to make sure that its the same on both ends.

root@tree:~# netcat -l -p 4444 | dd of=remote-machine.img

Source Machine
Next we’ll start a dd on the source machine and pipe it to netcat on port 4444

root@leaf:~# dd if=/dev/sda1 | netcat destination-machine-ip 4444

Now sit back and wait for your image to be done, when it’s finished dd will print out its status something like

NOTE: you will have to push CTRL+C to cancel out after this is completed, as the netcat session will still be active.

root@leaf:~#
30820468+71926 records in
30867456+0 records out
15804137472 bytes (16 GB) copied, 739.395 s, 21.4 MB/s
^C

If you want to find out the status of dd during the copy theres a couple of ways to do this, open up the system monitor in Ubuntu Linux, and it should tell you the transfer rate. Launch iostat or ifstat through a terminal. Invoke a command from terminal to get dd to display the current progress .

Viola, we’ll now have a dd image of our disk or partition. I like to verify the exact size of the file matches the size output from fdisk.

Destination Machine

root@root:~# ls -la remote-machine.img
-rw-r--r-- 1 root root 15804137472 2010-02-04 10:53 remote-machine.img

Source Machine

root@leaf:~# fdisk -l /dev/sda
Disk /dev/sda: 15.8 GB, 15804137472 bytes
255 heads, 63 sectors/track, 1921 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

What RAM type you have in Linux

To check what RAM memory type yo have installed (and also see other useful information about your system), do a

#dmidecode

Depending on the version of dmidecode you have installed and the hardware configuration you have, each hardware device will have a certain type number assigned t it. On my machine, the RAM has type 6. So to see what RAM type and speed you have, do a

#dmidecode --type 6

and the output will be something like

# dmidecode 2.9
SMBIOS 2.3 present.

Handle 0×0008, DMI type 6, 12 bytes
Memory Module Information
Socket Designation: ROW-0
Bank Connections: 1 0
Current Speed: 800
Type: DIMM SDRAM
Installed Size: 256 MB (Double-bank Connection)
Enabled Size: 256 MB (Double-bank Connection)
Error Status: OK

Change MAC address in Linux

If you wish to change your MAC address in Linux, all you have to do is bring the interface down then use the hw ether switch:

#ifconfig eth0 down
#ifconfig eth0 hw ether 02:01:02:03:04:08
#ifconfig eth0 up

but if you want your pc to change its MAC address on boot add that to a script in /etc/init.d/ folder, and also add symbolic link(ln) to /etc/rc2.d, /etc/rc3.d, /etc/rc4.d, /etc/rc5.d which refers to the script in /init.d/

#!/bin/bash

ifconfig eth0 down
ifconfig eth0 hw ether 02:01:02:03:04:08
ifconfig eth0 up
/etc/init.d/networking restart

Display dd progress during dd

Started a dd but wondering what the progress is? I haven’t found a way to do a verbose mode for dd, but this command seems to do the trick.

Lets start off by creating a dd of /dev/sda1

mnk0@tree:~# dd if=/dev/sda1 of=my-dd.img

We’ll need to find the process number of our dd which can easily be done with the following command.

ps -ef | grep dd

we’ll get something like this

root 31733 31268 54 10:44 pts/0 00:01:55 dd of my-dd.img

Now we can run our command to find the status of this dd. Open another terminal session.

kill -SIGUSR1 31733

and looking back at our dd page we should see dd dump out a status of its current progress.

mnk0@tree:~# dd if=/dev/sda1 of=my-dd.img
12574781+40555 records in
12601304+0 records out
6451867648 bytes (6.5 GB) copied, 224.634 s, 28.7 MB/s

Validating an IP Address in a Bash Script

Here’s a more useful example of using them to test IP addresses for validity.

To belabor the obvious: IP addresses are 32 bit values written as four numbers (the individual bytes of the IP address) separated by dots (periods). Each of the four numbers has a valid range of 0 to 255.

The following bash script contains a bash function which returns true if it is passed a valid IP address and false otherwise. In bash speak true means it exits with a zero status, anything else is false. The status of a command/function is stored in the bash variable “$?”.

#!/bin/bash

# Test an IP address for validity:
# Usage:
# valid_ip IP_ADDRESS
# if [[ $? -eq 0 ]]; then echo good; else echo bad; fi
# OR
# if valid_ip IP_ADDRESS; then echo good; else echo bad; fi
#
function valid_ip()
{
local ip=$1
local stat=1

if [[ $ip =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]]; then
OIFS=$IFS
IFS=’.’
ip=($ip)
IFS=$OIFS
[[ ${ip[0]} -le 255 && ${ip[1]} -le 255 \
&& ${ip[2]} -le 255 && ${ip[3]} -le 255 ]]
stat=$?
fi
return $stat
}

# If run directly, execute some tests.
if [[ "$(basename $0 .sh)" == 'valid_ip' ]]; then
ips=’
4.2.2.2
a.b.c.d
192.168.1.1
0.0.0.0
255.255.255.255
255.255.255.256
192.168.0.1
192.168.0
1234.123.123.123
‘
for ip in $ips
do
if valid_ip $ip; then stat=’good’; else stat=’bad’; fi
printf “%-20s: %s\n” “$ip” “$stat”
done
fi

If you save this script as “valid_ip.sh” and then run it directly it will run some tests and prints the results:

# sh valid_ip.sh
4.2.2.2 : good
a.b.c.d : bad
192.168.1.1 : good
0.0.0.0 : good
255.255.255.255 : good
255.255.255.256 : bad
192.168.0.1 : good
192.168.0 : bad
1234.123.123.123 : bad

In the function valid_ip, the if statement uses a regular expression to make sure the subject IP address consists of four dot separated numbers:

if [[ $ip =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]]; then

If that test passes then the code inside the if statement separates the subject IP address into four parts at the dots and places the parts in an array:

OIFS=$IFS
IFS=’.’
ip=($ip)
IFS=$OIFS

It does this by momentarily changing bash’s Internal Field Separator variable so that rather than parsing words as whitespace separated items, bash parses them as dot separated. Putting the value of the subject IP address inside parenthesis and assigning it to itself thereby turns it into an array where each dot separated number is assigned to an array slot. Now the individual pieces are tested to make sure they’re all less than or equal to 255 and the status of the test is saved so that it can be returned to the caller:

[[ ${ip[0]} -le 255 && ${ip[1]} -le 255 \
&& ${ip[2]} -le 255 && ${ip[3]} -le 255 ]]
stat=$?

Note that there’s no need to test that the numbers are greater than or equal to zero because the regular expression test has already eliminated any thing that doesn’t consist of only dots and digits.