FRIHOST FORUMS SEARCH FAQ TOS BLOGS COMPETITIONS
You are invited to Log in or Register a free Frihost Account!

In Command: Who needs a GUI anyway?




In Command: Who needs a GUI anyway?

Good use of the shell can ease a lot of tasks. The shell is especially good at automating tasks, getting jobs done quickly and manipulating more files at a time. To give you an idea of what you can do from the commandline I give you some examples for both Windows and Linux.

Batch downloading

Suppose you want to download a set of files which are named 1.jpg, 2.jpg, 3.jpg, etc. upto 100.jpg. There are several ways to accomplish this. We can create a little program to do this. Or we can download some kind of webscraper and configure it to download the files for us. But for an experienced shell-user the fastest way is propably just using the shell.

How would we download these set of files from the command line? Here is one way to do it:
Code:

seq 1 100 | sed "s/$/.jpg/" | sed "s/^/http:\/\/www.somesite.com\//" | wget -i -

First we create the numbers 1 to 100 using seq. Then we apply sed to it two times. sed "s/$/.jpg/" adds the .jpg extension to the end of each line. The second use of sed adds the url in front of each line. The result is the list of URLs we want to download.

Finally we hand this list to wget, a very useful tool to automate all kinds of downloading tasks. We tell wget to get the urls to download from the standard input with the "-i -" option. Wget will download 1.jpg, 2.jpg, etc. and put them in the current directory.

Windows: To make this example work on Windows you have to install the Wget package, the Sed package and the Sh-utils package from http://gnuwin32.sourceforge.net/packages.html. See The Windows Command Prompt for instructions on how to install these packages.

Editing more files at once

You have a local copy of your website. It consists of a large amount of HTML files in several subdirectories. Finally you have decided to add a doctype header to your HTML files. Now you have to add this line to the beginning of all html files:
Code:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">

Not only to index.html in the root directory, but also to admin/main.html, user/profile/private/help.html, etc. How would you do this? Is there some program to do this? Or can you use the shell? Yes, you can:

Linux:
Code:

find . -name "*.html" -exec sed -i '1i <!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\">' {} \;

Windows:
Code:

\gnu\bin\find . -name "*.html" -exec sed -i "1i <!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\">" {} ;


Note: In this case you have to specify the directory in which you have installed the gnu tools explicitly. If you don't specify a directory, Windows will use its own find tool, which isn't able to do this kind of stuff.

Note: The biggest problem when creating the Windows equivalent of a complicated Linux shell command is getting the quotes and special characters right.

To make this example work on Windows you have to install the FindUtils package and the Sed package from http://gnuwin32.sourceforge.net/packages.html. See The Windows Command Prompt for instructions on how to install these packages.

Smart Searching

You have lost the email address of your good friend John Doe. You have reason to believe it is propably in some file in your Documents folder somewhere. But in which file? And you don't even remember a part of the email address. Yes, well, it has probably the name John in it, or Doe. Now what? Try this:

Code:

grep -Ehir "(john|doe)[.a-z]*@[a-z]+\.[a-z]+" *.*


This will find all find all (well propably almost all) email addresses containing either "John" or "Doe" in all files in the current directory and its subdirectories:

Code:

C:\xampp\htdocs>grep -Ehir "(john|doe)[.a-z]*@[a-z]+\.[a-z]+" *
J.Doe @mail.com
Johnny.D @hotmail.com
misterdoe @mail.company.com
J.Doe @mail.com
misterdoe @mail.company.com
misterdoe @mail.company.com
johnny @hotmail.com
j.doe @gmail.com

C:\xampp\htdocs>


Windows: for this to work on Windows you need to have the Grep package installed from http://gnuwin32.sourceforge.net/packages.html. See The Windows Command Prompt for instructions on how to install these packages.

Did you know you had so many email addresses?

Try this in the root directory of your haddrive:

Code:

grep -Erohi "[.a-z]+@[a-z]+\.[a-z]+" * | grep "@" | sort | uniq > mail.txt


Depening on the number of files on your drive this takes a while (think hours), but when it is finished you have a file "mail.txt" with (almost) all emails on your harddrive, nicely sorted and the duplicates removed.

Windows: for this to work on Windows you need to have the Grep package and the Sh-utils package installed. See above.

Automating FTP (Linux only)

Suppose you have a file which you change several times a day. Every time you change it, you upload it to your webspace using ftp. After a while you get tired of starting the ftp-client and uploading your file everytime you changed it. Isn't there an easier way? Meet Expect. After installing Expect (apt-get install expect or rpm -i expect or something like that) you create a simple textfile in your home directory called upload:

Code:

spawn ftp "www.johndoe.frih.net"
expect "Name"
send "johndoe\r"
expect "Password:"
send "secret\r"
expect "ftp>"
send "cd somedir\r"
expect "ftp>"
send "put some.file\r"
expect "ftp>"
send "quit\r"


Substituting "johndoe" and "secret" with your username and password. From now on, when you issue the command

Code:

expect ~/upload


in the directory in which your file sits, your file gets uploaded. Finally create a shortcut on your desktop to "expect ~/upload" and uploading your file only takes a click of the mouse. (Of course this and the other examples in this article need some tweaking to work for your specific situation.)


Manipulating Images

Before you upload a set of jpg images to your website you want to add a copyright messages to all images. This can be done using the NetPbm tools. This set of tools let you easily convert images between different formats, change images, create thumbnails and analyze images. To add a copyright message to all jpg images in the current directory you can use this line:

Windows:
Code:

for %i in (*.jpg) do jpegtopnm %i | ppmlabel -x 0 -y 15 -color #000000 -background #ffffff -text "Copyright John Doe (C) 2007" | pnmtojpeg > t.%i

Linux:
Code:

for x in *.jpg; do  jpegtopnm $x | ppmlabel -color 0,0,0 -background 1,1,1 -x 0 -y 15 -text "Copyright John Doe (C) 2007" | pnmtojpeg > bla.$x; done


Windows: for this to work on Windows you need to install the NetPbm package from http://gnuwin32.sourceforge.net/. See above for details.

The End

There are much more useful tools and tricks to use from the command line. The most important thing is to remember the command line exists and can be a useful (or even the best) tool to perform a certain task. In other articles in this blog category I will try to convince you even more of the power of the shell and show you more tools and tricks of the trade.



0 blog comments below




FRIHOST HOME | FAQ | TOS | ABOUT US | CONTACT US | SITE MAP
© 2005-2011 Frihost, forums powered by phpBB.