Those of you who follow my columns will recall that I always define the Web as a pair of killer protocols, HTML and HTTP. The former defines the structure and appearance of all Web documents while the latter provides the handshaking necessary for the client and server to communicate with one another. In it's simplest form, however, this communication is pretty one-sided: the client requests something from the server, the server acknowledges the request and complies, whereupon the client requests for more information, and so forth. While this arrangement works well for locating and viewing cybermedia (CYBERspace multiMEDIA), it's pretty primitive. The interactivity between the user, the client and the server is pretty much restricted to downloads and links.
The need for additional interactivity became obvious early on in the evolution of the Web. The second iteration of Web specifications, version 1.0 of the Web standard promulgated by the Internet Engineering Task Force, was built around the CGI extension. This was called the so-called "forms" extension. The <FORM...> tag is an HTML extension which accommodates CGI programming.
We will illustrate CGI capability with a simple example. Suppose that we decide to add our office business hours along with the local time to our business homepage so that people will know if we're currently open for business. That's all, we just want to add the time and business hours.
While providing our business hours is a no-brainer, providing the local time would be a major hassle under HTML 1.0 specifications. In fact, it would require preparing as many different HTML documents as there are minutes in the day. The server would then determine the local time, translate that into a filename, and report the corresponding document. Just imagine a directory of almost identical files <homepage_12-00am.html>, <homepage_12- 01am.html>,..., <homepage_11-59pm.html> - 1,440 HTML documents in all - just so the user could see the current time on our server.
The problem was that prior to HTML 2.0 (circa 1993), the HTML documents were static - there was no way for the server to modify them on-the-fly. The server's internal clock kept track of the time, but there was no mechanism by means of which the server could insert that into the static HTML document. A serious deficiency, indeed.
HTML 2.0 fixed all that with the Common Gateway Interface - CGI for short. It is the mechanism by means of which the Web comes alive with interactivity. Interactive forms, continuously updated access counters, visitor's logs, sensitized imagery, and search engines all use the CGI.
From the user's perspective, CGI may be thought of as a collection of programs which run on a server. These programs may be written in any language which is capable of interpreting the standard input, output and environmental variables which are used to communicate between clients and servers. For ease of use, it is helpful if they are well-suited for string manipulation and interface with a wide variety of available libraries and utilities, particularly in the database and multimedia areas.Such CGI programs fall into three basic categories:
Any of these environments will work for simple CGI programming. Perl, however, is the clear choice of the professionals for robust CGI programming. The shell scripts, especially C-shell, are considered too impoverished for sophisticated applications.
While CGI programs may reside in any subdirectory which a system administrator allows for CGI binary files (hence the nickname "cgi-bin"), there are certain generally accepted conventions followed. For example, Perl script is usually run from #!/usr/local/bin/perl subdirectories while Bourne shell programs run from #!/bin/sh. This assumes that your server is set up to support the #! syntax for specifying translators. If this is not the case, you will need a work-around from your system administrator. CGI-bin rules-of-the-road are "pretty much" standard. Check your SYS_AD for details.
The invocation of CGI scripts is accomplished from within an HTML document. Let's use our local time problem as an example. Under HTML Version 2.0 we could include in an HTML document an instruction for the server to run a program which will report the current time at the exact place in the HTML document where the directive occurred. The HTML document would become dynamic. Only one document would be created, with a small window for the current time left in it, and the time would be updated only when it was requested. A much more satisfactory way of reporting the local time than having 1,440 tokens of the same homepage.
The current time example is really a special case of CGI programming. The more general case would be where the client would request a CGI executable file as a URL (Uniform Resource Locator). To illustrate, suppose that we requested the PERL program, <howdy.pl>, in the URL http://www.my_company.com/howdy.pl. In this case, the message from the client to the server would specify (a) the protocol which is to be used to exchange information (e.g., HTTP/1.0), (b) the data formats which the client can render (e.g., text/html, image/jpeg), and (c) the program, <howdy.pl> in this example, which the server is asked to execute. Since the system administrator has (we assume) set up the server to execute all programs in the CGI-bin subdirectory on access (rather than transferring them to the client), the server executes <howdy.pl> on the server on behalf of the client. The server then reports the result to the client in a format which the client can render (it wouldn't do to send the client graphics if it doesn't have a graphics monitor, would it).
In broad strokes, this is what CGI is all about. Providing additional interactivity between user, client and server with the goal of creating dynamic HTML documents.
Environment variables are the primary information-sharing resources shared by the client, the server and the CGI program. The information in the variables may come from the server itself, the client via the packet headers of the message, or user-supplied data in the form of path names or query variables. We will loosely divide these data sources into three categories: server-side, client-side and user-specified. We note that the latter two categories of information are only available upon the receipt of a client request.
We'll take up the server-side and client-side environment variables first. Here are just a few.
Modern scripting languages are designed with easy access to these operating system environment variables in mind. In Perl, access is made through the %ENV array. Shell languages have their own individual provisions.
Here is a very basic Bourne shell CGI script called <env_report.sh> which returns a few environment variable values. It is based upon a vanilla NCSA test-cgi shell script.
Note that the script prepared the output for HTML rendering (vs. plain text). The "Content-type" tag which specifies this is called a "partial HTTP header". The server adds the HTML information necessary for the response automatically. One may also specify the header completely without server involvement. This is called a "non-parsed header". We will defer its use to a later forum.
It should be mentioned that for simplicity we have removed one essential ingredient from this script - the ability to handle client input. Again, we defer I/O for awhile. Other than that, the CGI script will run satisfactorily when triggered by a link like <A HREF=http://www.host_serv.com/env_report>Here is pretty much the same program in Perl:
There are about two dozen environment variables for our use. Some of them can produce interesting Web enhancements. A cautionary note is in order, though, as the use of some environment variables return little of value when used in the wrong context.
As an example, consider the environment variable "USER_AGENT". This returns the browser type which was in use by the client making the HTTP request of the server. In an idea world, there would only be a few of these defined for all time and each browser name would be fully descriptive. Such is not the case, and as a result this variable is capable of misuse.
One possible misuse involves the routing of browsers to different HTML documents based on their level of compliance. Support of graphics provides a time-worn example. Older, text-only browsers (e.g., Lynx) cannot render graphics, so some cybersphere developers only show these browsers text documents. Here's an overview of how this is accomplished.
First, we create two different documents: text_only.html and graphics.html, with content befitting their names. Both documents will be referenced by the CGI program which makes the selection, say the Perl program <browser_select.pl>. This program will then use the USER_AGENT environment variable to identify the browser type and compare it to a list of text-only browsers. If the browser type is on the list, text.html is accessed; else, text_with_graphics.html. In other words, every client gets an HTML document that is just right for it.The business part might look something like this. First, our link from the outside might be <A HREF=http://www.host_serv.com/browser_select.pl>
The effect is pretty much as expected except for the fact that we left off the I/O. If someone using either browser, web_cheapskate or ancient_software, accesses our page, the CGI Perl program, <browser_select.pl>, will assign the ASCII page "text.html" to $html_document. For all other browsers the active document will be "graphics.html".
Therein lies the rub. Browser variations are abundant, and likely to remain so. The notion that any list (e.g., $text_only_browsers in this case) could be kept current without continuous review and revision is naive. Further, even within a family of browsers, versions n and n+1 may differ considerably in terms of compliance. Where this technique of routing users based on the contents of the USER_AGENT variable made some sense in 1993, the complexities of the modern Web experience today diminish its utility. The cartesian product of today's important features (e.g., support of frames and plug-ins, internal vs. external rendering of inline media, support of Java scripts and applets) with all the type and varieties of browsers is too large of a set to work with and still produce meaningful results.
The example of offering two differing formats of HTML documents according to the type of browser which made the request illustrates a CGI front-end which involves comparing the contents of an environment variable against a list. The decision is built around either a match or a failure-to-match.
We might also base the decision on partial contents of the environment variable. We may illustrate this technique with a very primitive firewall which regulates access to documents based upon the analysis of the IP domains.Internet IP addresses are represented in 32 bits. Using a standard 8-bit byte, the IP addresses can be (and usually are) represented as four integers in the range 0-256. Thus, the IP
Such IP numbers are separated into ranges which define classes, and some IP numbers are reserved for special use, but for most of us an IP number is just our node on the Internet. There is a low-level TCP/IP protocol, the address resolution protocol, which maps each of these IP numbers onto a particular physical address which represents the wire into our computer.
The environment variable which records this IP number, which is provided to the server by our client-side software, is REMOTE_ADDR. Our goal is to branch according to whether or not the first two numbers (in decimal) match against our "mask".
We begin with the specification of our IP mask - in this case the domain sequence 130.184 and the reading of the appropriate environment variable, REMOTE_ADDR.
One of most common uses of CGI programs is the CGI welcome which returns client-side information with the HTML document. Some use this to "personalize" the document; some to try to impress people that they know how to use environment variables; others because it is a by-product of the logging procedures which they have in place to monitor traffic. We'll outline this technique, here. Many modifications are trivial.For this example, we'll use four environment variables
A slight caveat is in order here, because not all client software set the REFERRER and FROM variables, hence they may not always work and even when they work they may be incorrect. Further, the referrer may be a bookmark on the client's hard disk - and hence meaningless. Still in all, this is a useful CGI exercise.The following CGI code produces the plaintext we want via the stand-alone CGI program specified in our URL. We modify slightly in order to incorporate within a HTML document.
There is another category of environment variables which we've postponed. This consists of user-supplied information which is available to a CGI program. By stretching our literary license, we will refer to these variables as "user-side", to attach lexical symmetry to the three types of environment variables.
The programs above illustrate one important part of communication between client and server - where the client, by linking to a URL, causes a CGI program to be executed. In the most basic form, user-supplied information is simply appended to the program name.Recall that the basic structure of an HTML link to a URL which contains a CGI program, say <update.pl> is
We note that the use of the environment variable QUERY_STRING is the same as for other environment variables. The only difference is the source of the information contained therein. This simple change, however, has major significance for the Web. Now, CGI programs are directly responsive to user input.
Now, let's connect this with HTML documents. Fortunately, the connectivity and the corresponding interface is made by the FORMS capability added in HTML version 2.0. Although there are differences between browsers in the way that they render forms, the effect is predictable.The FORM tag provides us with a convenient way to enter data as if the user typed the information onto the URL after the "?". Consider the following HTML
The example, above, is interesting for its simplicity. However, it is not the preferred way of handling user-side input. For one thing, the GET or QUERY_STRING method appends the input to the URL, so the information must conform to URL specifications (e.g., no spaces, no special characters, restricted length, etc.). To get around these limitations, we use standard input.
For many if not most applications, some search engines being the exception, the preferred way to pass user-side data to CGI programs is via the standard input, STDIN. The only real downside is that the input must now be encoded at the client side which, in turn, requires decoding at the server-side. When using FORMs, the STDIN option is designated by METHOD=POST.For the example in the previous section, the STDIN strategy would require that we rewrite our HTML as follows:
FORMS are the interface backbone of the Web. Everything that can be handled with forms can be done with query strings or standard input to be sure. But FORMS make the job infinitely easier.
At this moment, support of forms is quite limited: the I/O can only be text, no error-checking is possible on the client side, and FORMS cannot be nested (although one may have multiple forms in a single HTML document). Despite these limitations, FORMS provide an important, convenient interface tool for interactivity.The basic structure of the <FORM> HTML tag is
The <FORM> tag defines the nature of the interactivity between the client, server and CGI program. There remains to determine exactly what sort of information the form is to pass on. This is defined by the <INPUT> tag which produces a name/value pair. The name is the symbolic name given to the value and the value itself is the input provided to the form by the user. This pair is returned to the server, and hence to the CGI program, as a string of the form "name&value".The general form of <INPUT> is
Name, is the symbolic name attached to the user-entered value in the name-value pair. Value sets default value which appears before keyboard entry.
In addition, there are SELECT, OPTION and TEXTAREA form tags which extend the input flexibility. These are covered in any standard reference on HTML.
The client-side form output is the server-side form input which has to be unraveled by the CGI program after it is passed in as either an environment variable (METHOD="GET") or standard input (METHOD="POST"). Suppose we use the standard, default ENCTYPE. The client browser sends a stream of name-value pairs with sundry extraneous junk in it to the server, and hence to the CGI program which has to parse it. We'll deal with only the more difficult case of standard input, assuming that the CGI program has already read the environment variable REQUEST_METHOD which tells whether input is via GET or POST.
Next, we read the size of the client request with the environment variable CONTENT_LENGTH, and read that much standard input. We then split the query string on the value-pair delimiter, &, which separates each consecutively. Then we decode the special characters which were inserted for special characters, spaces, and the like. Then we can write the pair to a vector, which will produce indexed vector of all of the name-value pairs.Here's the idea. We'll create a Perl program, <form_input.pl> which will be launched by the URL specified as the form ACTION. Our first activity will be to read in the standard input
The "splits" break up the string around the delimiter in the first argument position. "&" is the delimiter which separates the pairs from one another, "=" separates the name and the value. The result is the creation of the associative array @name- value_pairs with the name-value pairs, indexed by name. "tr" is the translate function which returns the spaces. Finally, the "s" operator substitutes the ASCII form for any hex digits (% followed by two hex digits) that are found.At this point you can directly reference the name-value pairs by name.
Sensitized imagery in the limited sense of mouse-sensitive images included within anchors is as old as the Web itself. Iconized images of arrows substitute for textual navigation aids throughout the Web. Little pictures of houses are surrogates for "go to homepage," graphics of chains replace the word "link", "digitized portraits" serve as links to biographies and resumes, and so forth. The Web has been a pot pourrie of image-oriented anchors. Some of us refer to these links as sensitized icons.
Sensitized imagery is a logical extension of the concept of sensitized icons. Only in this case, the image is subdivided into geometrical partitions, each one of which represents or denotes a different URL. While conceptually simple, it is much harder to implement than sensitized icons. The coordination between the browser's registration of the current cursor location over the image and the URL to be navigated requires server-side interactivity - a CGI program.
Here's how it works. We begin with a standard HTML image in some format which is HTML compliant - e.g., .GIF, .JPEG. Next , we define for that image an "image map" which consists of a list of sensitized geometrical regions defined over the standard image. The image map is actually an ASCII file where each line defines one of the geometrical regions. We may think of these regions as superimposed over the original graphic like lines on tracing paper.To illustrate, let's suppose that our image file is GLOBE.GIF. Our image map, GLOBE.MAP, might look like this.
In operation, the client Web browser reports two things to the server. First, it gives the current cursor location relative to the original image's coordinates at the time of the mouse click. In addition, the client provides the corresponding .MAP filename and path for interpretation. In our case, if the X,Y coordinates 3,5 were reported to the server by the browser, the server would select the link http://www.your_business.com/ since it falls within the polygon.
The advantage of server-side image sensitizing is at the same time its weakness: the server must be involved. Having the server interpret the X,Y location of the cursor with respect to the sensitized regions requires client-server communication. In an attempt to overcome this drawback, Netscape and Internet Explorer support client-side image maps.
From the user's point of view, image sensitizing on the client- side work essentially the same way as on the server-side. With respect to the HTML, the same <img> tag is used to identify both the server and client image-map pairs. For the client-side, however, the image map is identified by the "usemap=" attribute. As with the server-side case, the client-side specifies both the image and the corresponding map. However, the client-side approach allows the user to place the image/map pair in the same directory.This is the HTML to specify the client-side image/map pair for our globe example:
We observe that the format for the coordinates is slightly different. The circle is of the form X,Y,radius and none of the coordinate sequences employ spacing. In addition, beware that browsers may differ in the shapes that they support.
SSI's are directives which you can place in HTML documents which substitute for CGI programs. While SSI can be both costly in terms of machine cycles and may present a security risk to the server, most servers no include SSI support as a standard Web resource. You should understand that HTML documents which contain SSI directives must be parsed which is where the time hit comes from. The parsable files are identified by extent and username, so the least server effort would involve segregating the HTML files with SSIs from those without by creating a special extent (.shtml is the popular form). A better way of handling this would be to put all parsable HTML documents in identifiable directories and let the OS parse or not based upon path. However, I don't know whether this is actually done.
In one way, SSI's are simple work-arounds for CGI. Instead of writing an CGI program to report the contents of an environment variable, for example, one may use an SSI directive to do the same thing.To illustrate, compare this Perl program
However, SSI directives extend beyond the ECHOing of environment and other variable assignments within HTML documents. In addition, one may INCLUDE text files into HTML documents, EXECute external programs and then insert the output into HTML documents. One may also report the filesize, FSIZE, of a document so that users may determine whether they want to download it.
We will illustrate a few basic applications of SSI's below. For further detailed information, consult any good reference on advanced HTML editing or CGI programming.
Counters, or "hit counters" as they are commonly called, are among the simplest SSI execs to demonstrate. The EXEC SSI command is used within a document to trigger a server-side program which (a) increments the counter value in a file, and (b) inserts the result into the HTML document. The CGI and CMD parameters of the EXEC directive distinguish between CGI and non- CGI executables on the server.
NOTE: Servers must enable EXEC in the options line of <access.conf> in order to support SSI executables. This poses a security risk to the server that the system administrator is unwilling to take.The general form of an SSI directive with the EXEC command is illustrated by
Using FILE as the filehandle, we attempt to open the file (designated by the scalar $count_val) with the current count value. The "<" and ">" preceding the scalar which designates the filename are the Perl prefixes which defines the OPEN as input or output, respectively. If the file OPEN is successful, the scalar $count is assigned the first line of the file - which, is simply the integer count value; else, a read error is reported. FILE is then closed so that it may be opened as a write file, the value augmented by 1 ($count++), closed again and the current count printed. If the write fails, it is reported. NOTE: remember to set your counter file initially and check the permissions so that the program may read and write to it!
The discussion above provides a basic overview of counters. An important extension of this Perl program would be one which outputs a graphical counter (say, like a car odometer), to the HTML document. Regrettably, current HTML standards only support text I/O for SSI's, hence the graphical interface involves a lot of busywork. The general theme is to output the count value, parse it into individual digits, and then output the digits as sequenced images. It is far easier to simply reuse (with credit given, of course) graphical counters from other sites than develop it from scratch, so a little surfing will reveal considerable commonality between counter programs.
A far more detailed analysis is provided in the December/Ginsburg book HTML and CGI Unleashed, particularly in chapter 32. Here, a pot pourrie of graphical counters are described. A word to the wise is in order here - a counter is just a counter. Overkill may be misunderstood or misinterpreted.
Much like with the counter program, we keep track of the previously-viewed slide in the file "last_slide.cnt". When <slideshow.pl> executes, it advances the counter and opens the corresponding file. That is, if the current count is "n", the document "ncount.html" is opened and returned to the parent HTML document.
A variation on the wrap-around slideshow is the random slideshow. This is even simpler than the case above, because we avoid opening and closing files and the concomitant error handling.We begin with the same SSI exec
In this case, we define a URL_LIST to be the vector of URL's to choose from. "srand" sets the random seed. $total_URLs is set to the index of the last URL in the vector and then used by the random number generator to produce an integer within the range 0 to $total_URLs. The active_URL is then defined to be the URL in this integer's (i.e., $randnum's) position. The link to this URL is then accomplished. The "qq" is syntactic sugar which elimates the need to escaped quotations within the string in the output pipe that follows.
We observe that this process may be extended beyond slideshow applications. Many splash pages now have "random links" on them. Be forewarned that "random links" are equated by many with "useless links". Use them purposefully (awarding free gifts and prizes seems to be a good use, if you can afford it).
The latter two directives define the formats for other SSI commands. "sizefmt" is used with the FSIZE command; "timefmt" is used with the DATE_LOCAL, DATE_GMT, and LAST_MODIFIED SSI environment variables. In either case, the CONFIG command defines the structure of the reported information.The most diverse range of formats is to be found in the SSI time formats, a brief list of which appear below:
At this writing, Java is currently in vogue. I'm really not all that confident that Java will stand the test of time. I'll give you my reasons. You be the judge.
BACKGROUND: Java is one programming environment among many. Like C and C++, it consists of a language kernel and a set of class libraries (which are called packages in Javaese). Java with packages; Microsoft Visual C++ with foundation classes; Borland C++ with Object Windows Library; etc. The parallel is unmistakable.
What makes Java "revolutionary" in some views is its focus on network connectivity and its portability.
In terms of network compliance, it includes the complete interface to the Internet, from the low-level TCP/IP socket to the use of canned, library routines. Java does for networking what C did for bit-wise manipulation.
However, at the highest level, Java lacks strength. There is, for example, no way to render imagery in Java. For obvious reasons, Sun did not want to compete with the navigator/browser development community, and so left this feature out of the product. Thus, the only way that you can render media in Java is by means of applets which augment the capability of client-side Web resources.
Java achieves its portability with the Java Virtual Machine language - a pseudo code for abstract machines like the Warren Abstract Machine for Prolog. Thus the Java source code written in the intermediate, abstract machine language produces what is called a bytecode file, which is then interpreted by machine- specific translators on the fly for execution on the host. This triumvirate of source code, byte code and host-resident interpreter, is the key to successful Java operation.
Suppose that security is added to Java. What form would that security take? It would have to protect against both the implementation of malicious code within Java as well as implementing the malicious code via system resources accessible to Java. The former case might be easier to manage by crippling the interpreter. However, the latter case would be fairly sticky. How can one prevent the execution of an operating system function call or interrupt from within a program? If you take away that, you couldn't create a directory or recursively delete files.
The problem, it seems to me, lies within the concept of executing unproven code from strangers. Launching such code from within a Web client is just an inherently bad idea it seems to me. Note that there is an important difference here from using tested, commercial, shrink-wrapped code to render downloaded media on the one hand, and using downloaded executables on the other.
One author refers to the activity of downloading and running Java executables as the cyberspace "leap of faith". He likens this to the Pilgrims who took off for America. In my view it is more like the travelers who deferred passage on the Titanic only to book the Lusitania. Traveling the North Atlantic during icy seas or wartime imperils.