N.Sivashanmugam
1. OTHER SCRIPTING LANGUAGES SHELL SCRIPT
1.1 What is Shell Script?
Normally shells are interactive. It means shell accept command from you (via keyboard) and execute them. But if you use command one by one (sequence of 'n' number of commands) , the you can store this sequence of command to text file and tell the shell to execute this text file instead of entering the commands. This is know as shell script. Shell script defined as: "Shell Script is series of command written in plain text file. Shell script is just like batch file is MS-DOS but have more power than the MS-DOS batch file."
1.2 What is Linux Shell ?
- Computer understands the language of 0's and 1's called binary language. In early days of computing, instruction are provided using binary language, which is difficult for all of us, to read and write. So in Os there is special program called Shell.
- Shell accepts your instruction or commands in English (mostly) and if its a valid command, it is pass to kernel. Shell is a user program or it's environment provided for user interaction.
- Shell is an command language interpreter that executes commands read from the standard input device (keyboard) or from a file. Shell is not part of system kernel, but uses the system kernel to execute programs, create files etc.
===Several shell available with Linux including:===
Shell Name Developed by Where Remark BASH ( Bourne-Again SHell ) Brian Fox and Chet Ramey Free Software Foundation Most common shell in Linux. It's Freeware shell. CSH (C SHell) Bill Joy University of California (For BSD) The C shell's syntax and usage are very similar to the C programming language. KSH (Korn SHell) David Korn AT & T Bell Labs -- TCSH See the man page.
Type $ man tcsh -- TCSH is an enhanced but completely compatible version of the Berkeley UNIX C shell (CSH).
Tip:
To find all available shells in your system type following command: $ cat /etc/shells Note that each shell does the same job, but each understand a different command syntax and provides different built-in functions. Any of the above shell reads command from user (via Keyboard or Mouse) and tells Linux Os what users want. If we are giving commands from keyboard it is called command line interface ( Usually in-front of $ prompt, This prompt is depend upon your shell and Environment that you set or by your System Administrator, therefore you may get different prompt ).
Tip:
To find your current shell type following command $ echo $SHELL Why to Write Shell Script ? Shell script can take input from user, file and output them on screen. Useful to create our own commands. Save lots of time. To automate some task of day today life. System Administration part can be also automated. How to write shell script Following steps are required to write shell script: (1) Use any editor like vi or mcedit to write shell script. (2) After writing shell script set execute permission for your script as follows
syntax:
chmod permission your-script-name Examples: $ chmod +x your-script-name $ chmod 755 your-script-name
Note:
This will set read write execute(7) permission for owner, for group and other permission is read and execute only(5). (3) Execute your script as syntax: bash your-script-name sh your-script-name ./your-script-name Examples: $ bash bar $ sh bar $ ./bar NOTE In the last syntax ./ means current directory, But only . (dot) means execute given command file in current shell without starting the new copy of shell, The syntax for . (dot) command is as follows Syntax: . command-name Example: $ . foo Now you are ready to write first shell script that will print "Knowledge is Power" on screen. $ vi first
- My first shell script
clear echo "Knowledge is Power” After saving the above script, you can run the script as follows: $ ./first This will not run script since we have not set execute permission for our script first; to do this type command $ chmod 755 first $ ./first First screen will be clear, then Knowledge is Power is printed on screen. Script Command(s) Meaning $ vi first Start vi editor
My first shell script
- followed by any text is considered as comment. Comment gives more information about script, logical explanation about shell script.
Syntax:
- comment-text
clear clear the screen echo "Knowledge is Power" To print message or value of variables on screen, we use echo command, general form of echo command is as follows syntax: echo "Message" Tip: For shell script file try to give file extension such as .sh, which can be easily identified by you as shell script. Exercise: 1)Write following shell script, save it, execute it and note down the it's output. $ vi ginf
- Script to print user information who currently login , current date & time
- clear echo "Hello $USER" echo "Today is \c ";date echo "Number of user login : \c" ; who | wc -l echo "Calendar" cal exit 0
==AWK OVERVIEW==
The Awk text-processing language is useful for such tasks as:
Tallying information from text files and creating reports from the results. Adding additional functions to text editors like "vi". Translating files from one format to another. Creating small databases. Performing mathematical operations on files of numeric data. Awk has two faces: it is a utility for performing simple text-processing tasks, and it is a programming language for performing complex text-processing tasks. The two faces are really the same, however. Awk uses the same mechanisms for handling any text-processing task, but these mechanisms are flexible enough to allow useful Awk programs to be entered on the command line, or to implement complicated programs containing dozens of lines of Awk statements. Awk statements comprise a programming language. In fact, Awk is useful for simple, quick-and-dirty computational programming. Anybody who can write a BASIC program can use Awk, although Awk's syntax is different from that of BASIC. Anybody who can write a C program can use Awk with little difficulty, and those who would like to learn C may find Awk a useful stepping stone, with the caution that Awk and C have significant differences beyond their many similarities. There are, however, things that Awk is not. It is not really well suited for extremely large, complicated tasks. It is also an "interpreted" language -- that is, an Awk program cannot run on its own, it must be executed by the Awk utility itself. That means that it is relatively slow, though it is efficient as interpretive languages go, and that the program can only be used on systems that have Awk. There are translators available that can convert Awk programs into C code for compilation as stand-alone programs, but such translators have to be purchased separately. One last item before proceeding: What does the name "Awk" mean? Awk actually stands for the names of its authors: "Aho, Weinberger, & Kernighan". Kernighan later noted: "Naming a language after its authors ... shows a certain poverty of imagination." The name is reminiscent of that of an oceanic bird known as an "auk", and so the picture of an auk often shows up on the cover of books on Awk. AWK Command-Line Examples It is easy to use Awk from the command line to perform simple operations on text files. Suppose I have a file named "coins.txt" that describes a coin collection. Each line in the file contains the following information:
metal weight in ounces date minted country of origin description gold 1 1986 USA American Eagle gold 1 1908 Austria-Hungary Franz Josef 100 Korona silver 10 1981 USA ingot gold 1 1984 Switzerland ingot gold 1 1979 RSA Krugerrand gold 0.5 1981 RSA Krugerrand gold 0.1 1986 PRC Panda silver 1 1986 USA Liberty dollar gold 0.25 1986 USA Liberty 5-dollar piece silver 0.5 1986 USA Liberty 50-cent piece silver 1 1987 USA Constitution dollar gold 0.25 1987 USA Constitution 5-dollar piece gold 1 1988 Canada Maple Leaf
I could then invoke Awk to list all the gold pieces as follows:
awk '/gold/' coins.txt
This tells Awk to search through the file for lines of text that contain the string "gold", and print them out.
The result is:
gold 1 1986 USA American Eagle gold 1 1908 Austria-Hungary Franz Josef 100 Korona gold 1 1984 Switzerland ingot gold 1 1979 RSA Krugerrand gold 0.5 1981 RSA Krugerrand gold 0.1 1986 PRC Panda gold 0.25 1986 USA Liberty 5-dollar piece gold 0.25 1987 USA Constitution 5-dollar piece gold 1 1988 Canada Maple Leaf
This is all very nice, you say, but any "grep" or "find" utility can do the same thing. True, but Awk is capable of doing much more. For example, suppose I only want to print the description field, and leave all the other text out. I could then change my invocation of Awk to:
awk '/gold/ {print $5,$6,$7,$8}' coins.txt
This yields:
American Eagle Franz Josef 100 Korona ingot Krugerrand Krugerrand Panda Liberty 5-dollar piece Constitution 5-dollar piece Maple Leaf
This example demonstrates the simplest general form of an Awk program:
awk <search pattern> {<program actions>}
Awk searches through the input file for each line that contains the search pattern. For each of these lines found, Awk then performs the specified actions. In this example, the action is specified as:
{print $5,$6,$7,$8}
The purpose of the "print" statement is obvious. The "$5", "$6", "$7", and "$8" are "fields", or "field variables", which store the words in each line of text by their numeric sequence. "$1", for example, stores the first word in the line, "$2" has the second, and so on. By default, a "word" is defined as any string of printing characters separated by spaces.
Since "coins.txt" has the structure:
metal , weight in ounces, date minted , country of origin , description -- then the field variables are matched to each line of text in the file as follows: metal: $1 weight: $2 date: $3 country: $4 description: $5 through $8
The program action in this example prints the fields that contain the description. The description field in the file may actually include from one to four fields, but that's not a problem, since "print" simply ignores any undefined fields. The astute reader will notice that the "coins.txt" file is neatly organized so that the only piece of information that contains multiple fields is at the end of the line. This is a little contrived, but that's the way examples are. Awk's default program action is to print the entire line, which is what "print" does when invoked without parameters. This means that the first example:
awk '/gold/'
-- is the same as:
awk '/gold/ {print}'
Note that Awk recognizes the field variable $0 as representing the entire line, so this could also be written as:
awk '/gold/ {print $0}'
This is redundant, but it does have the virtue of making the action more obvious. Now suppose I want to list all the coins that were minted before 1980. I invoke Awk as follows:
awk '{if ($3 < 1980) print $3, " ",$5,$6,$7,$8}' coins.txt
This yields:
1908 Franz Josef 100 Korona 1979 Krugerrand
This new example adds a few new concepts: No search pattern is specified. Without a search pattern, Awk will match all lines in the input file, and perform the actions on each one. I can add text of my own to the "print" statement (in this case, four spaces) simply by enclosing the text in quotes and adding it to the parameter list. An "if" statement is used to check for a date field earlier than 1980, and the "print" statement is executed only if that condition is true. There's a subtle issue involved here, however. In most computer languages, strings are strings, and numbers are numbers. There are operations that unique to each, and one must be specifically converted to the other with conversion functions. You don't concatenate numbers, and you don't perform arithmetic operations on strings. Awk, on the other hand, makes no strong distinction between strings and numbers. In computer-science terms, it isn't a "strongly-typed" language. All the fields are regarded as strings, but if that string also happens to represent a number, numeric operations can be performed on it. So we can perform an arithmetic comparison on the date field. The next example prints out how many coins are in the collection:
awk 'END {print NR,"coins"}' coins.txt
This yields:
13 coins
The first new item in this example is the END statement. To explain this, I have to extend the general form of an Awk program to:
awk 'BEGIN {<initializations>}
<search pattern 1> {<program actions>}
<search pattern 2> {<program actions>}
...
END {<final actions>}'
The BEGIN clause performs any initializations required before Awk starts scanning the input file. The subsequent body of the Awk program consists of a series of search patterns, each with its own program action. Awk scans each line of the input file for each search pattern, and performs the appropriate actions for each string found. Once the file has been scanned, an END clause can be used to perform any final actions required. So, this example doesn't perform any processing on the input lines themselves. All it does is scan through the file and perform a final action: print the number of lines in the file, which is given by the "NR" variable. NR stands for "number of records". NR is one of Awk's "pre-defined" variables. There are others, for example the variable NF gives the number of fields in a line, but a detailed explanation will have to wait for later. Suppose the current price of gold is $425, and I want to figure out the approximate total value of the gold pieces in the coin collection. I invoke Awk as follows:
awk '/gold/ {ounces += $2} END {print "value = $" 425*ounces}' coins.txt
This yields:
value = $2592.5
In this example, "ounces" is a variable I defined myself, or a "user defined" variable. You can use almost any string of characters as a variable name in Awk, as long as the name doesn't conflict with some string that has a specific meaning to Awk, such as "print" or "NR" or "END". There is no need to declare the variable, or to initialize it. A variable handled as a string variable is initialized to the "null string", meaning that if you try to print it, nothing will be there. A variable handled as a numeric variable will be initialized to zero. So the program action:
{ounces += $2}
-- sums the weight of the piece on each matched line into the variable "ounces". Those who program in C should be familiar with the "+=" operator. Those who don't can be assured that this is just a shorthand way of saying:
{ounces = ounces + $2}
The final action is to compute and print the value of the gold:
END {print "value = $" 425*ounces}
The only thing here of interest is that the two print parameters, the literal '"value = $"' and the expression "425*ounces", are separated by a space, not a comma. This concatenates the two parameters together on output, without any intervening spaces.
Using AWK from the Command Line The Awk programming language was designed to be simple but powerful. It allows a user to perform relatively sophisticated text-manipulation operations through Awk programs written on the command line. For example, suppose I want to turn a document with single-spacing into a document with double-spacing. I could easily do that with the following Awk program:
awk '{print ; print ""}' infile > outfile
Notice how single-quotes (' ') are used to allow using double-quotes (" ") within the Awk expression. This "hides" special characters from the shell you are using. You could also do this as follows:
awk "{print ; print \"\"}" infile > outfile
-- but the single-quote method is simpler. This program does what it supposed to, but it also doubles every blank line in the input file, which leaves a lot of empty space in the output. That's easy to fix, just tell Awk to print an extra blank line if the current line is not blank:
awk '{print ; if (NF != 0) print ""}' infile > outfile
One of the problems with Awk is that it is ingenious enough to make a user want to tinker with it, and use it for tasks for which it isn't really appropriate. For example, you could use Awk to count the number of lines in a file:
awk 'END {print NR}' infile
-- but this is dumb, because the "wc (word count)" utility gives the same answer with less bother: "Use the right tool for the job." Awk is the right tool for slightly more complicated tasks. Once I had a file containing an email distribution list. The email addresses of various different groups were placed on consecutive lines in the file, with the different groups separated by blank lines. If I wanted to quickly and reliably determine how many people were on the distribution list, I couldn't use "wc", since, it counts blank lines, but Awk handled it easily:
awk 'NF != 0 {++count} END {print count}' list
Another problem I ran into was determining the average size of a number of files. I was creating a set of bitmaps with a scanner and storing them on a floppy disk. The disk started getting full and I was curious to know just how many more bitmaps I could store on the disk. I could obtain the file sizes in bytes using "wc -c" or the "list" utility ("ls -l" or "ll"). A few tests showed that "ll" was faster. Since "ll" lists the file size in the fifth field, all I had to do was sum up the fifth field and divide by NR. There was one slight problem, however: the first line of the output of "ll" listed the total number of sectors used, and had to be skipped. No problem. I simply entered:
ll | awk 'NR!=1 {s+=$5} END {print "Average: " s/(NR-1)}'
This gave me the average as about 40 KB per file.
- Awk is useful for performing simple iterative computations for which a more sophisticated language like C might prove overkill. Consider the Fibonacci sequence:
1 1 2 3 5 8 13 21 34 ...
Each element in the sequence is constructed by adding the two previous elements together, with the first two elements defined as both "1". It's a discrete formula for exponential growth. It is very easy to use Awk to generate this sequence:
awk 'BEGIN {a=1;b=1; while(++x<=10){print a; t=a;a=a+b;b=t}; exit}'
This generates the following output data:
1 2 3 5 8 13 21 34 55 89
2.0 INTRODUCTION TO XML
2.1 BASICS
XML was designed to describe data and to focus on what data is.HTML was designed to display data and to focus on how data looks.
What You Should Already Know
Before you continue you should have a basic understanding of the following: HTML / XHTML JavaScript or VBScript
2.2 What is XML?
XML stands for EXtensible Markup Language XML is a markup language much like HTML XML was designed to describe data XML tags are not predefined. You must define your own tags XML uses a Document Type Definition (DTD) or an XML Schema to describe the data XML with a DTD or XML Schema is designed to be self-descriptive XML is a W3C Recommendation
The Main Difference Between XML and HTML
XML was designed to carry data. XML is not a replacement for HTML.
XML and HTML were designed with different goals:
XML was designed to describe data and to focus on what data is. HTML was designed to display data and to focus on how data looks. HTML is about displaying information, while XML is about describing information.
XML Does not DO Anything
XML was not designed to DO anything. Maybe it is a little hard to understand, but XML does not DO anything. XML was created to structure, store and to send information.
The following example is a note to Tove from Jani, stored as XML: <note> <to>Tove</to> <from>Jani</from> <heading>Reminder</heading> <body>Don't forget me this weekend!</body> </note>
The note has a header and a message body. It also has sender and receiver information. But still, this XML document does not DO anything. It is just pure information wrapped in XML tags. Someone must write a piece of software to send, receive or display it.
XML is Free and Extensible
XML tags are not predefined. You must "invent" your own tags. The tags used to mark up HTML documents and the structure of HTML documents are predefined. The author of HTML documents can only use tags that are defined in the HTML standard
XML allows the author to define his own tags and his own document structure. The tags in the example above (like <to> and <from>) are not defined in any XML standard. These tags are "invented" by the author of the XML document. XML is a Complement to HTML XML is not a replacement for HTML. It is important to understand that XML is not a replacement for HTML. In future Web development it is most likely that XML will be used to describe the data, while HTML will be used to format and display the same data. My best description of XML is this: XML is a cross-platform, software and hardware independent tool for transmitting information. XML in Future Web Development XML is going to be everywhere. We have been participating in XML development since its creation. It has been amazing to see how quickly the XML standard has been developed and how quickly a large number of software vendors have adopted the standard. We strongly believe that XML will be as important to the future of the Web as HTML has been to the foundation of the Web and that XML will be the most common tool for all data manipulation and data transmission. XML SYNTAX RULES The syntax rules of XML are very simple and very strict. The rules are very easy to learn, and very easy to use. Because of this, creating software that can read and manipulate XML is very easy. An Example XML Document XML documents use a self-describing and simple syntax. <?xml version="1.0" encoding="ISO-8859-1"?> <note> <to>Tove</to> <from>Jani</from> <heading>Reminder</heading> <body>Don't forget me this weekend!</body> </note> The first line in the document - the XML declaration - defines the XML version and the character encoding used in the document. In this case the document conforms to the 1.0 specification of XML and uses the ISO-8859-1 (Latin-1/West European) character set. The next line describes the root element of the document (like it was saying: "this document is a note"): <note> The next 4 lines describe 4 child elements of the root (to, from, heading, and body): <to>Tove</to> <from>Jani</from> <heading>Reminder</heading> <body>Don't forget me this weekend!</body> And finally the last line defines the end of the root element: <note> Can you detect from this example that the XML document contains a Note to Tove from Jani? Don't you agree that XML is pretty self-descriptive? All XML Elements Must Have a Closing Tag With XML, it is illegal to omit the closing tag. In HTML some elements do not have to have a closing tag. The following code is legal in HTML: <p>This is a paragraph <p>This is another paragraph In XML all elements must have a closing tag, like this: <p>This is a paragraph</p> <p>This is another paragraph</p> Note: You might have noticed from the previous example that the XML declaration did not have a closing tag. This is not an error. The declaration is not a part of the XML document itself. It is not an XML element, and it should not have a closing tag. XML Tags are Case Sensitive Unlike HTML, XML tags are case sensitive. With XML, the tag <Letter> is different from the tag <letter>. Opening and closing tags must therefore be written with the same case: <Message>This is incorrect</message> <message>This is correct</message> XML Elements Must be Properly Nested Improper nesting of tags makes no sense to XML. In HTML some elements can be improperly nested within each other like this: This text is bold and italic</b> In XML all elements must be properly nested within each other like this: <b>This text is bold and italic XML Documents Must Have a Root Element All XML documents must contain a single tag pair to define a root element. All other elements must be within this root element. All elements can have sub elements (child elements). Sub elements must be correctly nested within their parent element: <root> <child> <subchild>.....</subchild> </child> </root> XML Attribute Values Must be Quoted With XML, it is illegal to omit quotation marks around attribute values. XML elements can have attributes in name/value pairs just like in HTML. In XML the attribute value must always be quoted. Study the two XML documents below. The first one is incorrect, the second is correct: <?xml version="1.0" encoding="ISO-8859-1"?> <note date=12/11/2002> <to>Tove</to> <from>Jani</from> </note> <?xml version="1.0" encoding="ISO-8859-1"?> <note date="12/11/2002"> <to>Tove</to> <from>Jani</from> </note> The error in the first document is that the date attribute in the note element is not quoted. This is correct: date="12/11/2002". This is incorrect: date=12/11/2002. Comments in XML The syntax for writing comments in XML is similar to that of HTML. There is nothing special about XML. It is just plain text with the addition of some XML tags enclosed in angle brackets. Software that can handle plain text can also handle XML. In a simple text editor, the XML tags will be visible and will not be handled specially. In an XML-aware application however, the XML tags can be handled specially. The tags may or may not be visible, or have a functional meaning, depending on the nature of the application.
