CGI Introduction

If you've done any Perl scripting, you're probably familiar with the acronym CGI (Common Gateway Interface). Each word in the acronym Common Gateway Interface helps you to understand the interface:

• Common - interacts with many different operating systems (ie, UNIX, Windows, or MacOS).
• Gateway - provides users with a way to gain access to different programs, such as scripts, executables, databases, or picture generators.
• Interface - uses a well-defined method to interact with a Webserver (API).

It's simply a portal for communication between the Webserver and any other program or script that it can access. But CGI is not a programming language! In essence, CGI is a server-side process that serves as a go-between for the Webserver and other applications programs, information resources, and databases. These information resources and databases can reside on the same physical machine as the Webserver or on a machine at some other geographical location.

CGI gives you a way to make Websites dynamic and interactive. CGI just makes it possible for a Webserver to pass information (such as what host the user is connecting from, or input the user has supplied through an HTML form) to another program. The program then processes that data and the server then "captures" the resulting output from that program — typically a web page — and passes the program's response back to the Web browser. With CGI, you can write scripts that create interactive, user-driven applications.

CGI Overview

Rather than limiting the Web to documents written ahead of time, CGI enables Web pages to be created on the fly, based upon the input of users. You can use CGI scripts to create a wide range of applications, from surveys to search tools, from Internet service gateways to quizzes and games. You can count the number of users who access a document or let them sign an electronic guest book. You can provide users with all types of information, collect their comments, and respond to them.

CGI's strength is its flexibility. CGI can be used to develop applications on nearly every Webserver in existence. Another important advantage of CGI is that it doesn't really care what script or programming language you use for your applications. It simply passes the message between the Webserver and the host computer.

CGI "scripts" are commonly written in Perl, but they can also be written in C, C++, COBOL, TCL/Tk, Python, Visual Basic, AppleScript, UNIX Shell scripts (Bourne, csh, etc.), or any other language that the host computer can understand. Since most of these programming and scripting languages are available on all the popular operating systems, running CGI scripts is an option for just about anyone.

There are significant downsides to CGI scripts as well. The main one is that CGI scripts act like any other program or script running on a computer — they need to be owned and executed by someone. Given the fundamental openness of the Web, this creates a number of problems.

The first is that, in most cases, there's a single user account associated with the Webserver—often called something like WEBUSER. Every time a web page accesses an application through CGI, a new instance of that user is created to run the requested program. This means that if you have a large, complicated shopping cart program on your site and 100 users are visiting, 100 copies of that program are being simultaneously executed by WEBUSER. This creates a definite performance problem.

NOTE: Just about all the new Webservers today have solved the problem of having to create a new instance of a CGI process for each requested program. The CGI requests all run in one process with each request running as a separate thread.

A further problem for CGI applications is security. Since every CGI application is executed by the same user account (e.g. WEBUSER), every application accessible through CGI needs to have permissions set that allow WEBUSER to execute it, but not necessarily to read or modify it. There are ways to handle these security issues; the point is that it is easy to leave files compromised, especially if you are a novice to CGI and/or permission management.

Regardless of the performance and security issues, CGI is by far the most common way for Webservers to access other applications. Its success may be attributable to the fact that it's straightforward, cross-platform, and compatible with traditional languages such as C and Perl.

There are a number of ways to access databases through CGI, particularly using Perl libraries or custom C code. Many more sophisticated web database application servers can provide some portion of their capabilities as a CGI application. Most of these sophisticated platforms, however, use API access instead.

Gateway Program, CGI Program, or CGI Script?

Gateway program, CGI program, and CGI script are essentially three names for the same thing. CGI programs contain the code that accepts data from the Webserver (most often passed on from its Web clients) and does something with that data. Tasks can range from various forms of processing, accessing information resources, creating output, or all of these. CGI programs can be developed in virtually any language supported on the Webserver host machine, including compiled and interpreted languages such as C, C++, Java, JavaScript, Visual Basic, FORTRAN, Perl, Awk, Expect, and Tcl, as well as some of the newer Perl extensions for client/server database access — for example, Sybperl and Oraperl.

How Does CGI Work?

CGI programs are always placed on a disk that the Webserver has access to. This means that if you are using a dial-up account to maintain your Web site, you need to upload your CGI programs to the server before they can be run.

Webservers are generally configured so that all CGI applications are placed into a cgi-bin directory. However, the Webserver may have aliases so that "virtual directories" exist. Each user might have his or her own cgi-bin directory. The directory location is totally under the control of your Web site administrator.

API (Application Programming Interface) - not required

You probably treat most of the applications you use like a black box — put information into the program and get out the results the creator of the application intended. But many power users and developers need slightly different implementations of existing features. That's when an Application Programming Interface (API) comes in handy.

An API helps you "program the program" by accessing the internal functions used by the application for its own tasks. If you ever created a macro in Word or Excel, or designed behaviors in Dreamweaver, you were probably using the built-in interface to program the application itself. The form that an API takes can vary, but the underlying idea is that some portion of the guts of the application are exposed for your manipulation. You're limited in the kinds of changes you can make to the fundamental actions inside the application, but otherwise your imagination is the only limit.

Most modern Webservers have APIs that allow custom functionality to be built right into them. In terms of web database connectivity, the API is typically used to create a line of communication between the Webserver and the web database application server that bypasses CGI. You could build your own custom code that interfaces directly with the Webserver and the database, or you could use a commercial product (such as Active Server Pages - ASP, ColdFusion, PHP, and so on) that does the same thing.

The biggest advantage of API-based Webserver access is speed. Instead of a new process being required for each instance of an application called by the Webserver, a single multi-threaded application can handle all the traffic. This arrangement can drastically improve the performance of web applications.

Since the API doesn't have to be compatible with other Webserver APIs (unlike CGI, which is the same on all platforms), a great deal of optimization can be done to improve communication between the server and other applications. This optimization not only improves performance, but also allows more sophisticated security since the application accessed through the server API runs pretty much like it actually is the server.

The tradeoff for all the advantages of API access methods is that each Webserver has a different API and any application must be customized for a given API. This means that custom developers tend to stay on a single platform (Netscape Webservers, or Internet Information Servers (IIS) for example) to reduce their workload. It also means that commercial products that work with multiple Webserver APIs are more complex (and thus more expensive) to develop.

Summary

CGI programs were shown to be invoked by a URL. The URL could be entered directly into a Web browser or stored in a Web page as a hypertext link or the destination for HTML form information.

Before CGI program can be run under the UNIX operating systems, their file permissions need to be set correctly. Files have three types of permissions: read, write, and execute. And there are three types of users that access files: user, group, and others. CGI programs must be both readable and executable by others.

The first line of output of any CGI program must be some type of HTTP header. The most common header is Content-type:, which basically tells the Web browser what to expect (plain text, perhaps? Or maybe some HTML). The Location: header redirects the Web browser to another URL. The Set-cookie: header stores a small bit of information on the visitor's local disk. The last header is Status:, which tells the Web browser that an error has arisen.

By placing a / or ? at the end of a URL, information can be passed to the CGI program. Information after a / is placed into the PATH_INFO environment variable. Information after a ? is placed into the QUERY_STRING Environment Variable.

Environment Variables play a big role in CGI programs. They are the principal means that Webservers use to provide information. For example, you can find out the client's IP address using the REMOTE_ADDR variable. And the SCRIPT_NAME variable contains the name of the current program.

URL encoding is used to prevent characters from being misinterpreted. For example:

•  < is encoded as %3C
• " " is converted into +

One of the biggest security risks happens when a user's data (form input or extra path information) is exposed to operating system commands such as mail or grep. Never trust user input! Always suspect the worst. Most hackers spend many hours looking at manuals and source code to find software weaknesses. You need to read about Web security in order to protect your site.

Cookies are used to store information on the user's hard drive. They are a way to create persistent information that lasts from one visit to the next.