Basic and fundamental knowledge

I was recently asked with some basic and fundamental question, directly related to Linux system administration job. When you are already a senior engineer, most of the time you will forget about the fundamental, or don't really care about it. But truth is, you still need to know about it, because it is so fundamental. People will judge you with this basic information to know that if you are really knowledgeable and your level is what you really say you are at. I'm not able to answer all the question satisfactorily, and feel somewhat ashamed about it.

Let's learn it together.


1. What is the difference between a network hub and a network switch? 

When I search Google for the answer, I believe this site put it very nicely. Take a look at http://www.duxcw.com/faq/network/hubsw.htm . In essence:

• Hub repeats the packet it receive on one port to the other port available
• The bandwidth is shared across all the ports. If the hub is 10Mbps, with 5 ports, then each port can only transfer at max 2Mbps
• Switch divides the network into multiple segment thus a pair of ports can communicate without affecting other pair of ports
• Switch maintains a table of destination address and its port, so when a packet arrives, it will send the packet to the correct port
• The bandwidth of the port is dedicated. If the switch is 10Mbps, with 5 ports, when port 1 connect to port 2, the bandwidth is 10Mbps for that instance, and when port 3 connect to port 4, the bandwidth is also 10Mbps for that instance

2. How many bit are there in a MAC address?

I cannot answer this question correctly. The answer is 48 bit. This site provides the information: http://compnetworking.about.com/od/networkprotocolsip/l/aa062202a.htm .  In essence:

• MAC address have 12 digit hexadecimal number
• 1 hex = 4 bit, thus 12 hex = 48 bit
• Hex symbol is 0123456789ABCDEF
• The first 6 hex digit represent the manufacturer
  

3. What is the difference between TCP and UDP?

Wikipedia have the answer: http://en.wikipedia.org/wiki/User_Datagram_Protocol#Comparison_of_UDP_and_TCP

• TCP is Transmission Control Protocol
• TCP is connection oriented link
• When a machine send a TCP packet, the receiving machine have to send back acknowledgment packet when it arrive
• If sending machine fail to get the acknowledgment after certain time period, the packet will be resend again.
• Example of TCP usage is between web server and web browser

• UDP is User Datagram Protocol
• UDP never guarantees that a packet will arrive at destination
• When a machine send a UDP packet, it is not expecting acknowledgement from the receiving machine.
• Example of UDP usage is audio streaming, and DNS

3. What information are available in TCP packet? Name some of them.

• Source address
• Destination address
• Checksum

4. What flag are available in TCP packet? Name some of them.

I also cannot answer this question correctly. The answer are:

• SYN
• ACK
  

5. When you execute "uptime" command, there are 3 numbers at the end of the line. What are they?

Answer here: http://linux.die.net/man/1/uptime . The 3 numbers are load averages for the past 1, 5, and 15 minutes.


6. What is the meaning of the load average number?

Here is the answer: http://www.lifeaftercoffee.com/2006/03/13/unix-load-averages-explained/ . It means "the average sum of the number of processes waiting in the run-queue plus the number currently executing over 1, 5, and 15 minute time periods."


7. How do you know that the server is busy from the number?

The best answer probably from this site: http://www.teamquest.com/resources/gunther/display/5/index.htm . For this question, I answered if the number is 2 or bigger, then the server is busy or under heavy load. This is relative, and you will know from experience handling Linux or Unix machines.


8. What happen when the machine is busy?

I answered, the most noticeable clue is that you have trouble accessing the server remotely. When you SSH to the server, it will take a while before you are able to login. This is because the SSH connection is encrypted and the server will need to decrypt the data before able to give you access. Encryption and decryption takes big amount of CPU cycle, and if the machine is already busy, you will see it will take some time before you are able to login.

Other than that, if the machine have small amount of memory, you will see a lot of disk activity, because the OS is swapping the application that resides in memory, but is not executed, to the disk to make way for application that have higher priority.


9. How do you list processes running in the machine?

Use the command "ps"



10. How do you terminate a misbehaving application?

Use the command "kill -9 <appname>". The number 9 is sending the SIGKILL signal. To terminate application with the same name, use "pkill <appname>"


11. What other signal available?

This site summarize it: http://linux.about.com/od/commands/l/blcmdl7_signal.htm . Other signal available is SIGHUP and SIGTERM. I answered SIGHUP is to restart an application, but most information in Internet said that SIGHUP is to re-read configuration file or to stop an application. You might need to search for more concrete answer.

SIGTERM is terminate signal sent to application to stop it gracefully. When application receive a SIGTERM, it will do the necessary process to make sure it is stopped cleanly.


12. What is the difference between SIGKILL and SIGTERM?

SIGTERM is a graceful termination signal. The application that receive the signal will try its best to stop or notify any dependency, and then terminating itself. For example, if a parent process got SIGTERM, and it has few child process, the parent process will notify the child process that the parent is being terminated, and the parent might also send SIGTERM to the child to terminate them before terminating itself. SIGTERM can be ignored if the application was programmed to do so.

For SIGKILL, the application will be directly terminated by the OS. No information will be sent to any dependencies of the program. If a program have a child process, that child process might become orphan or zombie because its parent has been killed and it has no clue on what to do next. This kind of issue might cause further instability to the server if the server is already have some issue.


13. Have you experienced application that will not terminate even after you send SIGKILL signal? How do you terminate such application?

More info here: http://en.wikipedia.org/wiki/Zombie_process . That application is called zombie application. To find zombie application, use command "ps aux | grep Z", where the zombie application will have Z as its status. You cannot killed a zombie application because it is already dead. What I usually do is, if the zombie process have a parent process, I will terminate the parent process, where most of the time the zombie process will terminate because its dependency to its parent has been terminated.

But, there are cases that, when you terminate the parent process of the zombie, the zombie will then use process with PID 1 as its parent. Process with PID 1 is the init process, and it is the first process to run when the server starts. If this happen, you have no other way to kill the zombie other than rebooting the server.


14. What actually pkill command do?

pkill will list the PID of the process that have the name as specified, then will terminate the application one by one. The default signal sent is SIGTERM.


That's all. I have learned quite a lot after this event. Hopefully this post will be helpful to someone out there.
Got comment? Let me know. :)

Tracing bitmap in Inkscape

During BOF session in MyGOSSCON 2009, Nuhaa a.k.a cawanpink representing FOSSchix.my, was presenting on how to use Inkscape to produce graphic images. After her presentation, I inform her that Inkscape has the capability to trace bitmap into vector graphic, so that if you want to scale the image, it will not become blur (when scale down) or become boxy (when scale up). I guide her the step to produce the vector image. I believe a lot of Inkscape user also did not know much about this capability. This post will explain the step.

Step:

1. Search Google Images for the image that you want to trace. Usually you would like to trace logo to be use somewhere else. When you search for image, try to find big or medium size image. In this example, I want to search for 1Malaysia logo.








2. Download the logo to your computer. Here is the logo that I got.








3. Open the bitmap image in Inkscape.








4. Press Ctrl+A to select all the image. Then select the menu Path > Trace Bitmaps








5. Dialog option will be displayed. Select the option Colors, Stack scans and Remove background. After that, click the Update button. You will see a preview of the trace. If you are happy with the preview, click OK








6. The vector image will be available on top of the bitmap image. If you select the logo, you can move it around. In the picture below, I move the vector image to the right.








7. You can delete the bitmap image then move back the vector image to the canvas. After that you can do anything with the vector image. Here, I made a clone of the image, then scale it down. The image is still sharp because it is a vector.

That's all. You can save the vector image to SVG format if you want to edit it again in the future.

Happy drawing! :)

Programming in PostScript

I have found my new craze! It is programming in PostScript! :D

What is PostScript, you may ask? PostScript is a programming language optimized to print graphic and text. You can think it as a page description language, much like HTML is the document description language for the web.

It all started when Najmi posted to MyPenguin99 mailing list about the powerfulness of Python to generate a PDF file. It is called Simson Garfinkel's Notepaper Generator. Upon closer inspection, I found out that it is not the Python that is powerful, but it is the PostScript language that is powerful that you can program your paper document to look like what you want it to be.

The Python script is just a script that help you to either enable or disable a feature, change the owner of the paper, and generate the calendar to be put into the PostScript file. From the PostScript file, it is converted to PDF by using ps2pdf command line program.

Suddenly, a question pops in my head. How hard it is to hand-coded a Postscript file? I have never program a PostScript before and don't know the answer. But I'm going to find out.

Few years back when I'm an electronic engineering student, graph paper is one of my tools of the trade. Back then, when I'm running out of graph paper, I have to buy a graph paper pad with 10 sheet in it, even though I might only need a single sheet to complete my assignment.

So, to challenge myself for the PostScript programming, I'm planning to create a 1 mm by 1 mm grid that will occupy an A4 paper, with about 1 inch border from the edge of the paper. That should be easy, I think.

And actually, it is easy! I just learned PostScript in 2 days from information available on the Internet, and also found a PDF file from Adobe called "PostScript Language Tutorial & Cookbook". It contains all the command that you need to know about programming in PostScript. Search Google using the keyword "postscript tutorial".

PostScript is not hard. It is stack-based in the same manner as RPN calculator. And this is easy for me because I program Motorola microcontroller using stack-based instruction in assembly language during my university days :)

The result of the PostScript file can be viewed without having to print it out, by using GSview for Windows. You can also use Evince for Gnome or Okular for KDE. The PostScript file can be converted to PDF file
 by using PDFCreator in Windows, or ps2pdf command line in Unix/Linux/BSD.

Below are two screenshot of GSview, one viewing the A4 paper in whole, and the other zooming the paper to its width.

Get the source from the URL below. Now I can start to create my own GTD paper organizer. :D

Enjoy! :)

http://sharuzzaman.tripod.com/file/graph.ps

Having fun with Apache reverse proxy

Today I managed to configure an Apache reverse proxy. Why do I need a reverse proxy? Consider this scenario:

1. I got 2 public IP: 192.0.2.1 and 192.0.2.2
2. Firewall hold all the public IP via IP aliasing (eg. eth0:1 = 192.168.0.2.1, eth0:2 = 192.0.2.2)
3. I'm running authoritative DNS on my firewall
4. I initially have 2 webserver inside firewall, with IP address 172.16.0.1 (xavier) and 172.16.0.2 (magneto)
5. My domain name is example.com
6. Subdomain www.example.com is served by xavier, and webmail.example.com is served by magneto
7. DNS was configured so that www.example.com will be resolved to public IP 192.0.2.1 and webmail.example.com will be resolved to public IP 192.0.2.2
8. IPTables was used to redirect port 80 from public IP to its corresponding private IP (eg. 192.0.2.1 -> 172.16.0.1)
9. All webserver must run on port 80 and is using Apache 2.x.
   

Now, I want to add another web server with IP address 172.16.0.3 (cyclops), that should serve the subdomain dev.example.com

How can I do that? I only have 2 public IP! I cannot use other non-standard port!

The solution?

It comes in the mixture of DNS and Apache reverse proxy.

On DNS:
- add subdomain dev.example.com to resolve to public IP 192.0.2.1
- restart DNS service

On Apache 2.x web server in xavier (172.16.0.1)
- make sure mod_proxy is enabled in /etc/httpd.conf
- create a new file in /etc/httpd/conf.d/ (eg. dev.example.com.conf)
- in the file, put the following directives

<VirtualHost 172.16.0.1>
ProxyPreserveHost On
ProxyPass / http://172.16.0.3/
ProxyPassReverse / http://172.16.0.3/
ServerName dev.example.com
ProxyRequests Off

<Proxy *>
Order deny,allow
Allow from all
</Proxy>
</VirtualHost>

- restart Apache service

That's it. Try to surf the new subdomain http://dev.example.com using anonymous surfing site (eg. http://anonymouse.org/) and you should be presented with the content of cyclops web server.

How does it work?

- The subdomain dev.example.com will be resolved to public IP 192.0.2.1 by our DNS server.
- Request for port 80 to dev.example.com will be forwarded to the Apache web server running on xavier (172.16.0.1)
- On xavier, the VirtualHost directive is aware that it is serving for subdomain dev.example.com
- But that directive contains a ProxyPass instruction to pass all request (hence "/" or root directory) to the server cyclops (172.16.0.3)
- It also have the directive ProxyPassReverse to pass everything received from cyclops back to the client as if the root (/) is on the server
- Other directive is left as an exercise to the reader to find out

Other usage of Apache reverse proxy

- Reverse proxy can be used to mask/map port 80 to webserver hosted on non-standard port (eg. 8080) **I believe IPtables port manipulation can achieve the same result

- Shield not-so-secure I*S web server running on not-secure-at-all W*s host.
This will not protect against SQL injection or other web-based attacks. It can help to shield the server from OS related attacks because client will see that the webserver is Apache running on Linux, not I*S running on W*s. You might want to take a look at mod_security for added protection


Have fun :)

Using shell script to solve problem

I'm translating KDE4 to Malay language, but when I'm in the kdebase folder, I cannot determine which files that should be translated, and also have been neglected for some time. While I can check each file manually to determine its translation status, I'm really lazy to do the same thing again and again.

By using shell script, I can automate the task and become more lazier :P

Requirement:
List all po files according to date, older to newer, and only contain fuzzy or untranslated message.

Solution:

sharuzzaman@debian:/kde4-stable/kdebase$ cat -n list.sh
1  #!/bin/bash
2
3  for file in `ls -tr *.po`
4  do
5    output=`msgfmt -o /dev/null --statistics $file 2>&1`
6    fuzzyuntranslate=`echo $output | grep -e "fuzzy\|untranslated"`
7    if [ "$fuzzyuntranslate" != "" ]
8    then
9      echo $file
10    fi
11  done

Let's take a look at the solution, line by line.

Line 1: Declare the script as a Bash script

Line 2: Blank space for clarity

Line 3: This is the starting point of the "for" loop. The command "ls -tr *.po" will list all po files according to date and reversed, which means older to newer. We quote the command in backtick `` because we want the command to be executed, and the output to become the array of file name for variable "file"

Line 4: The "do" is the part in the "for" loop that we process our list of files.

Line 5: Execute the command "msgfmt -o /dev/null --statistics $file 2>&1" and put the result in variable "output". The 2>&1 redirection is required because the output for msgfmt is printed on stderr, not stdout, so we redirect it to stdout in order to capture it.

Line 6: Echo back the variable "output" and check if it contain the word "fuzzy" or "untranslated" using grep. Put the result in variable "fuzzyuntranslated". If the variable "output" did not contain the word that we search for, the variable "fuzzyuntranslated" will be blank. We use "grep -e" because we have regular expression "|" that carry the meaning "or" on the command

Line 7: Check if the variable "fuzzyuntranslated" is not blank (which means either contain fuzzy, untranslated, or both fuzzy and untranslated)

Line 8: Then

Line 9: Print out the file name that match our requirement.

Line 10: Close the if block with fi

Line 11: Close the do block with done


That's it. We have completed our requirement.

The output should be a long list of filename. I can pipe it to "head" to get only the first 10 line of the filename.

sharuzzaman@debian:/kde4-stable/kdebase$ ./list.sh |head
kdmgreet.po
nsplugin.po
kdialog.po
kdebugdialog.po
kcmkwindecoration.po
kcmusb.po
kcmstyle.po
kdmconfig.po
kcmscreensaver.po
khtmlkttsd.po

Happy scripting :)

Journey to NetBSD world - part zero

I’m always unhappy with Linux memory requirement for old and small memory PC. I have an Intel Pentium 133 MHz, with 64 MB RAM. Installing minimal Debian Linux, without any other application will eat up as much as 50 MB of RAM, leaving only about 14 MB for application. On percentage basis, Linux took about 78% of system memory.

Few people that I have asked about this issue, advise me to recompile the kernel, and remove any unnecessary thing that is available in the kernel. Well, being lazy, I want an OS that when installed out-of-the-box, will use small enough memory, while leaving a lot more for application.

I have heard about NetBSD for a long time, since I started using Linux in 2001. But, I don't have the courage to use it; maybe because when I first use Linux, I thought Linux is hard enough. BSD-based OS must be harder. But sometimes, necessity can be a good motivator for you to try something new.

In this part zero, I will explain the preparation that I have done to install NetBSD on the machine. Because the machine already got Debian Sarge, I don't want to format it because I think it might come handy later. So, I bought a used 2GB hard disk drive, just for this exercise. It's not that expensive, only RM 16

Because the PC can be considered a dinosaur, it can only boot from floppy and hard disk. There is no way to boot from CD. So, to get the floppy boot image, I browsed to http://ftp.netbsd.org/pub/NetBSD/NetBSD-3.1/i386/installation/floppy/ and then downloading boot1.fs and boot2.fs using IE. (My office is using IE, btw )

Then I download WinRawrite from http://www.chrysocome.net/rawwrite to write the image into a couple of floppy disk. The write process complete, but when I tried to boot the floppy, the machine keeps rebooting. Hmm... why this is happening? NetBSD don't like my PC? NetBSD is not good enough? (scratching my head)

Reading again the installation document, they already mentioned that the image must be downloaded in “binary” format. Downloading using web browser will result in the image downloaded as “ascii” format. They advised to download the image using FTP client, and setting the client to download in binary format.

This time, the Debian installation really comes handy. I plug again the Debian hard disk; reboot the computer, and FTP into NetBSD ftp site. Download the image using binary format, then write the image into floppy disk. I write the image using dd command available in Debian.

dd if=boot1.fs of=/dev/fd0
dd if=boot2.fs of=/dev/fd0

I re-plug the blank hard disk, then boot the computer with the NetBSD boot floppy, and few minutes later, NetBSD installation menu comes up on the screen. Wee… I manage to boot NetBSD

On next part of the journey, I will show what option I select during the installation, and how it progress after that. Till next time.