Creating 'Encoded' Name & Value Pairs

This page is a slight variation of last week's URL Encoding/Decoding. If you understood last week's example then you can bypass this page.

GET and POST

Last week we said there are two basic methods of passing data to the server, using either the GET or POST Method, as specified in the Form's METHOD Attribute:

<FORM METHOD="GET" ACTION="/cgi-bin/feedback.pl">

Or:

<FORM METHOD="POST" ACTION="/cgi-bin/feedback.pl">

The program on the server can be written to handle both method types. Before we do this, let's examine the differences between GET and POST.

GET

GET encodes the data and appends it to the end of the URL. For example the following form:

<FORM METHOD="GET" ACTION="feedback.pl">
    <INPUT TYPE="text" NAME="title"
      VALUE="
Creating 'Encoded' Name & Value Pairs">
    <INPUT TYPE="text" NAME="author" VALUE="Frank Peter">
    <INPUT TYPE="submit" VALUE="Submit">
</FORM>

Once submitted, it requests the following document from the server:

feedback.pl?title=For+creating+%27Encoded%27+Name+%26+Value+Pairs&
author=Frank+Peter

The request is to GET the document, and at the same time pass the encoded NAME=VALUE pairs of the form to the document. It is then up to the document to retrieve the NAME=VALUE pairs passed to itself.

Note that the encoding of the data replaces any spaces (" ") with a plus ("+"), and any punctuation, or any characters outside of the standard ANSI character set with the two digit hexadecimal number of the characters' Latin-1 (ISO-8859-1) encoding value. The "%" character indicates that the following number is hexadecimal.

The following table shows the Latin-1 and hexadecimal values of the more common non-standard ANSI characters:

ANSI Latin-1 Hex
! 33 %21
" 34 %22
# 35 %23
$ 36 %24
% 37 %25
& 38 %26
' 39 %27
( 40 %28
) 41 %29
* 42 %2A
+ 43 %2B
, 44 %2B
- 45 %2D
. 46 %2E
/ 47 %2F
: 58 %3A
; 59 %3B
ANSI Latin-1 Hex
< 60 %3C
= 61 %3D
> 62 %3E
? 63 %3F
@ 64 %40
[ 91 %5B
\ 92 %5C
] 93 %5D
^ 94 %5E
_ 95 %5F
` 96 %60
{ 123 %7B
| 124 %7C
} 125 %7D
~ 126 %7E
&127; 127 %7F
     

POST

With POST, the form NAME=VALUE pairs are sent after the URL request has been sent. Generally, POST is only used when posting data to the server. The process on the server uses the standard input to retrieve the NAME=VALUE pairs.

As the NAME=VALUE pairs are not part of the URL, it is impossible to simulate a form submission using the POST method without using a Form. The rest of this page will concentrate on the GET Method.

Passing data in a link

Suppose we wanted to capture feedback for this current page. Instead of placing a Feedback Form at the bottom of the article, which might only get filled in and returned by a few visitors, we could link to a generic feedback form.

This reduces the file size of the article, the bandwidth necessary to send it, and the time it takes to display. What's more, we only have to maintain one Feedback Form.

The disadvantage is that, unless we pass the details of the article to the Feedback Form, we wouldn't know which article the feedback was for.

We could use a Form with a single Submit button and several Hidden Form fields to pass the data from the article to the Feedback Form, but this involves the submission of two Forms, with two warnings to the user that they are submitting form data over the Internet.

Using a text link to the Feedback Form with the data already encoded in the URL will overcome this problem.

Each time a new page is created, you will have to hand-craft the encoded URL. The encoding ensures that the data arrives at its destination, regardless of the character encodings used by the many computers and networks that the data may pass through.

The manual approach will require you to translate each non-standard ANSI character to the Latin-1 hexadecimal character. So:

Creating 'Encoded' Name & Value Pairs

is manually edited, step by step:

Creating+'Encoded' Name & Value Pairs
Creating+%27Encoded' Name & Value Pairs
Creating+%27Encoded%27 Name & Value Pairs
Creating+%27Encoded%27+Name & Value Pairs
Creating+%27Encoded%27+Name+& Value Pairs
Creating+%27Encoded%27+Name+%26 Value Pairs
Creating+%27Encoded%27+Name+%26+Value Pairs
Creating+%27Encoded%27+Name+%26+Value+Pairs

Are you sure you've converted all the characters to the correct value? Have you missed any? Why not automate the process?

The following text link, when clicked, will alter the URL to include an encoded title NAME=VALUE pair:

<A HREF="feedback.pl"
   onClick="this.href='feedback.pl?title=' +
   escape('Creating \'Encoded\' Name & Value Pairs')
">
Feedback
</A>

Encoding Data

The final part of this page will demonstrate a simple tool that you can adapt and extend for your own purposes. You'll be able to use it to automate the production of encoded NAME=VALUE URLs, which you can then simply cut and paste into the page before you publish it.

The following form shows how to encode a single NAME=VALUE pair, where the name of the data is "title" and the value is whatever is entered in the Title text field:

Title:

Encoded URL:

There is, however, one subtle difference between the data encoded by escape() Method, and URL Encoded when a form is submitted. The escape() Method replaces the space character, not with the "+" character, but with the Latin-1 hexidecimal %20. This shouldn't normally cause any problems, as most CGI scripts can decode either correctly.

The HTML and JavaScript code to perform the encoding is extremely simple:

<FORM>
    Title:
    <INPUT TYPE="text" NAME="title"
      VALUE="
Creating 'Encoded' Name & Value Pairs">
    <INPUT TYPE="button"
     
onClick="this.form.output.value='title=' + escape(this.form.title.value)" VALUE="Create">
    Encoded URL:
    <INPUT TYPE="text" NAME="output"
</FORM>

In fact, it's too simple. It doesn't allow for any additional name/value pairs.

The following example demonstrates a form that allows you to select which NAME=VALUE pairs are added to the encoded URL. It shows how to select values from a select form field, and allows you to add an additional name and value:

Title: Value:

Author: Value:

Unknown: Value:

Encoded URL:

The HTML and JavaScript code to create the form looks like this:

<SCRIPT>

function createEncodedPairs(form) {
    var output = '';

    if (form.titleChecked.checked) {
        output += escape(form.titleName.value) + '=' + escape(form.titleValue.value);
    }

    if (form.authorChecked.checked) {
        if (output != '') output += '&';
        AV = form.authorValue;
        output += escape(form.authorName.value) + '=' +
                        escape(AV.options[AV.selectedIndex].text);
    }

    if (form.unknownChecked.checked) {
        if (output != '') output += '&';
        output += escape(form.unknownName.value) + '=' +
                        escape(form.unknownValue.value);
    }

    return output;
}

</SCRIPT>

<FORM>
     <INPUT TYPE="reset">
     <INPUT TYPE="checkbox" NAME="titleChecked" CHECKED>
     Title:
     <INPUT TYPE="text" NAME="titleName" SIZE="7" VALUE="title">
     Value:
     <INPUT TYPE="text" NAME="titleValue" SIZE="40" VALUE="">
     <INPUT TYPE="checkbox" NAME="authorChecked" CHECKED>
     Author:
     <INPUT TYPE="text" NAME="authorName" SIZE="7" VALUE="author">
     Value:
     <SELECT NAME="authorValue">
          <OPTION>Frank Peter</OPTION>
          <OPTION>Jon Allen</OPTION>
     </SELECT><P>
     <INPUT TYPE="checkbox" NAME="unknownChecked">
     Unknown:
     <INPUT TYPE="text" NAME="unknownName" SIZE="7" VALUE="">
     Value:
     <INPUT TYPE="text" NAME="unknownValue" SIZE="40" VALUE="">
     <INPUT TYPE="button" VALUE="Create"
      
onClick="this.form.output.value= createEncodedPairs(this.form)" >
     Encoded URL:
     <INPUT TYPE="text" NAME="output">
</FORM>

As you can see, this small but effective tool is extremely useful. It will save time when pre-encoding data for use on text links.

Here is a simple CGI script that can accept the encoded title and author URL data from the previous example and then display it on a feedback page:

#!/usr/local/bin/perl

$input = $ENV{'QUERY_STRING'};
@pairs = split(/&/, $input);

foreach $pair (@pairs) {
    ($name, $value) = split(/=/, $pair);
    $value =~ tr/+/ /;
    $value =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C", hex($1))/eg;
    $FORM{$name} = $value;
}

print "Content-type: text/html\n\n";

print << "(HTML DOC)";
    <HTML>
    <HEAD><TITLE>Feedback: $FORM{title}</TITLE></HEAD>

    <BODY>

    <FORM METHOD="POST" ACTION="mailto:frank\@sislands.com"><P>
    Title:<BR>
    <INPUT TYPE="TEXT" NAME="title" SIZE="60" VALUE="$FORM{title}"><P>
    Author:<BR>
    <INPUT TYPE="TEXT" NAME="author" SIZE="60" VALUE="$FORM{author}"><P>
    Your Name:<BR>
    <INPUT TYPE="TEXT" NAME="name" SIZE="60" VALUE=""><P>
    Your Email Address:<BR>
    <INPUT TYPE="TEXT" NAME="email" SIZE="60" VALUE=""><P>
    Comments:<BR>
    <TEXTAREA NAME="COMMENTS" COLS="60" ROWS="10" WRAP="VIRTUAL">
    </TEXTAREA><P>
    <INPUT TYPE="RESET">&nbsp;&nbsp;<INPUT TYPE="SUBMIT">
    </FORM>

    </BODY>
    </HTML>
(HTML DOC)
exit;

The above CGI script will present a form with the title and author fields pre-filled, which will, when submitted, send an e-mail with the feedback comments.