Thursday, 31 October 2013

Howto: Calculate exponent or power in Bash

I have been looking on how to calculate exponent or power in Bash, but most website or blog will show the calculation by using bc (a command line calculator)

I prefer not to use bc in my script, as it is a dependency to another application.

Luckily, I found this blog after searching the Internet for few hours. http://blog.sanctum.geek.nz/calculating-with-bash/

In essence, if you want to calculate exponent or power in Bash, just use this notation:

**

Example:
Gigabyte is 1024^3. In bash notation:

gigabytes=$((bytes / (1024**3)))

[user@server]$
echo $((10737418240/(1024**3)))
10


Bear in mind that bash calculation only return round number.

Happy scripting :)

Thursday, 22 August 2013

Howto: Extract all email address from Google Contacts

Let's say you want to extract all email address from your contacts in Google Contacts, and export it somewhere else. Yes you can just export the CSV file, and then upload. The other service will do the import and cleanup for you. But, if you just want to share the email address, why give them other info that they are not suppose to have?

OK. Here are the steps to only extract email address using command line. I'm using Cygwin, but it should be similar if you are using Linux or other Unix-based operating system.

Follow this steps:

1. Select Contacts in Gmail


2. Your contacts will be shown. Click More > Export


3. Select All Contacts and Google CSV format, then click Export


4. Save the file somewhere. I save it in C:\google.csv . If you are using Cygwin, the file is accessible using the path \cygdrive\c\google.csv

5. Now come the interesting part, to extract the email address. The data is separated by comma, and we don't really know which column holds the email address. So we must iterate all column, and extract anything that resembles an email address. Here is the command:

$ grep @ google.csv | awk -F, '{for(i=1;i<=NF;i++) if ($i ~ /@.*\./) {printf "%s\n", $i};}' | awk -F" ::: " '{for (i=1;i<=NF;i++) {print $i};}' |sort |uniq

6. Let's break the command apart

7. grep @ google.csv

This command will get line that contain "@" character. The result is lines that contain "@", with multiple column, separated by comma ","

8. awk -F, '{for(i=1;i<=NF;i++) if ($i ~ /@.*\./) {printf "%s\n", $i};}'

This command let awk know that the field separator is comma (-F,). It will loop through all the field (for(i=1;i<=NF;i++)). If that field match an email pattern ($i ~ /@.*\./), it will print that field.

9. awk -F" ::: " '{for (i=1;i<=NF;i++) {print $i};}'

Some of the field will have multiple email address separated by " ::: ", because it groups the email address together. This command will split the field using " ::: " separator (-F" ::: ") then loop through each field, and print each of them

10. sort

This command will sort the output

11. uniq

This command will remove any duplicates.

12. In the end you will get a list of emails. But, you must understand that the output might not be 100% clean. Some of your contact might put their email with their name, or the note area of your contact might contain additional information that resembles email. You need to clean up your output, but the effort will be small.


Thursday, 4 April 2013

How To: Boot from USB drive even if your BIOS won't let you

This post is a continuation from my post about How To: Easy CentOS 6.3 installation using USB thumb drive

After I finished setting up the USB thumb drive, one of the machine that I'm going to install did not allow me to boot from USB, because the BIOS did not support that capability.

As I'm not going to burn the CentOS 6.3 Installer ISO into a CD-RW, just for this machine, I searched the Internet to find solution to this problem. And I found it.

The solution is called plopKexec. This software, when you boot from the CD drive, will search if there are any USB drive attached to the system, and will try to load the Linux bootloader from that drive.

How to use it:

  1. Visit the URL http://www.plop.at/en/plopkexec.html
  2. Click on Download
  3. Dowload the plopkexec.iso file
  4. Burn the ISO file into CD-R or CD-RW
 On the computer that you want to install using USB drive
  1. Plug in your USB drive into any USB port
  2. Put the plopkexec CD into the CD drive
  3. Power on your computer
  4. The CD will boot, an the menu from your USB thumb drive will shown
  5. Select "Install or upgrade an existing system" and continue installing CentOS 6.3 as normal
Here's the screenshot of plopkexec loading the GRUB menu from my USB thumb drive


That's it.

Simple way to solve your USB booting issue :)

Please leave comment if this solution have helped you.

Thanks.

Thursday, 3 January 2013

How To: Easy CentOS 6.3 installation using USB thumb drive

Recently, I need to do CentOS 6.3 installation on two machine that could be 32-bit or 64-bit processor. I have downloaded the minimal install ISO for both architecture, but I thought it could be a waste to burn multiple CD-RW, just to use it maybe only once.

Then, I search the Internet on the simplest and easiest way to install CentOS 6.3 by using USB thumb drive. After learning from few website, and facing some trouble, this post will teach you the simplest and easiest way.

What you will require:
  1. Minimal install ISO for CentOS 6.3. Get it from your local mirror.
  2. USB thumb drive - must be bigger that the size of the ISO. For minimal install, 1GB is good enough.
  3. UNetbootin software - I download the Windows version, because my work laptop is Windows
The instruction given here is by using Windows. If you use Linux, you need to find out how to format the USB thumb drive in FAT32

How to do it:



Format the USB thumb drive
  1. Plug your USB thumb drive into the USB port on your Windows laptop/PC
  2. Your USB thumb drive should be detected, or a pop-up will say that your USB thumb drive is not formatted.
  3. If detected, say drive F:, right click and select Format
  4. If you got pop-up in step 2, proceed with Format
  5. Select FAT32 as file system.
  6. Tick Quick Format as format option
  7. Click Start
  8. Your USB thumb drive will be formatted with FAT32

Transfer ISO content to USB thumb drive using UNetbootin
  1. Download and launch UNetbootin
  2. Select Diskimage and click the ". . ." button. Find the ISO and click Open
  3. Make sure Type is USB drive and the Drive letter is what your Windows detect.
  4. Click OK
  5. Your ISO content will be copied to USB thumb drive
It actually did not end here. If you proceed with installation by using the USB thumb drive, the installer will say that it cannot find the ISO image.

Follow this instruction:
  1. Copy the ISO file that you use with UNetbootin to the root of the USB thumb drive, eg. the ISO file should be F:\CentOS-6.3-i386-minimal.iso


Now, proceed to boot up the machine that you want to install with CentOS 6.3 with your USB thumb drive. Make sure the machine BIOS support booting up from USB.


During installation, you will be asked where to find the installation image, select from hard disk.


When asked for partition that hold the image, just select OK.

That's it. If you have any issue, please leave a comment.

Thanks :)

Tuesday, 30 October 2012

Putty, Xming, CentOS X Forwarding not working

If you are using Putty to forward your X session on CentOS Linux, make sure you have this requirement.

Putty

Make sure Enable X11 Forwarding is enabled



CentOS

Make sure package xorg-x11-xauth is installed. This is very important

Xming

No special setting required.

Monday, 3 October 2011

How to to list all packages by size for RPM and DEB

You might be experiencing an issue where you don't have enough space for "/" in your server. Usually, the biggest thing that consume the space is your software installed in the server.

So, for sure you want to remove the biggest software package installed on the server to recover back the space.

The question is, how do you list all packages installed and sort it by size?

I found the answer from this website: http://www.pixelbeat.org/docs/packaging.html

The command that you should use is:

For RPM:
rpm -q -a --qf '%10{SIZE}\t%{NAME}\n' | sort -k1,1n

For DEB:
dpkg-query -W --showformat='${Installed-Size;10}\t${Package}\n' | sort -k1,1n

That's all. Remove the package by using Yum or Aptitude so that they can automatically find the correct dependencies for the removed package.

Friday, 26 August 2011

Check machine firewall if you are configuring network firewall

I have wasted 2 days of my time trying to figure out what is really happening with my firewall config. The story is like this. I tried to configure a 3 NIC firewall machine for my office using Shorewall. For the DMZ zone, I put a test machine with Apache HTTPD installed to test if I can connect to it. With example from this site: http://www.shorewall.net/three-interface.htm . I can have my laptop in the local area to surf the Internet by configuring NAT/MASQ. I can SSH to the test web server, I can ping the server, but what really bugging me is that I cannot access the test web page hosted on the server.

When I tried to connect from firewall to the web server in DMZ by using links/elinks, the application returned error "No route to host". Weird. I tried to check my routing table, and search the Internet for clue. Tried to play around with default gateway for the DMZ, but in the end, I still cannot connect to the web server.

I even scrapped the whole thing and start again from scratch by following the example from this website: http://wiki.debian.org/HowTo/shorewall . Still cannot access.

Then, while searching for solution on the Internet again, I found a forum that ask a poster whether the firewall on the machine itself is turned on. That struck me like lightning. I straight away SSH to the web server in the DMZ and issue the command "service iptables stop".

Going back to my laptop and hit refresh in the web browser, voila!! The page is there!!

I slapped my forehead 3 times for this silly mistake :D

So, moral of the story, if you are configuring firewall for your network, make sure you turn off firewall on the machine so that you are not being fooled into thinking that your firewall configuration is problematic.

2 day wasted, but really priceless experienced learned :)

By the way, Shorewall really rocks :)