url.pl -- Analysing and constructing URL

This library deals with the analysis and construction of a URL, Universal Resource Locator. URL is the basis for communicating locations of resources (data) on the web. A URL consists of a protocol identifier (e.g. HTTP, FTP, and a protocol-specific syntax further defining the location. URLs are standardized in RFC-1738.

The implementation in this library covers only a small portion of the defined protocols. Though the initial implementation followed RFC-1738 strictly, the current is more relaxed to deal with frequent violations of the standard encountered in practical use.

author

- Jan Wielemaker

- Lukas Faulstich

deprecated

- New code should use library(uri), provided by the clib package.

 global_url(+URL, +Base, -Global) is det

Translate a possibly relative URL into an absolute one.

Errors

- syntax_error(illegal_url) if URL is not legal.

 is_absolute_url(+URL)

True if URL is an absolute URL. That is, a URL that starts with a protocol identifier.

 http_location(?Parts, ?Location)

Construct or analyze an HTTP location. This is similar to parse_url/2, but only deals with the location part of an HTTP URL. That is, the path, search and fragment specifiers. In the HTTP protocol, the first line of a message is

<Action> <Location> HTTP/<version>

Arguments:

Location

- Atom or list of character codes.

 csearch(+Attributes)//[private]

 cvalue(+Value)// is det[private]

Construct a string from Value. Value is either atomic or a code-list.

 cfragment(+Attributes)//[private]

 parse_url(?URL, ?Attributes) is det

Construct or analyse a URL. URL is an atom holding a URL or a variable. Attributes is a list of components. Each component is of the format Name(Value). Defined components are:

protocol(Protocol)

The used protocol. This is, after the optional url:, an identifier separated from the remainder of the URL using :. parse_url/2 assumes the http protocol if no protocol is specified and the URL can be parsed as a valid HTTP url. In addition to the RFC-1738 specified protocols, the file protocol is supported as well.

host(Host)

Host-name or IP-address on which the resource is located. Supported by all network-based protocols.

port(Port)

Integer port-number to access on the \arg{Host}. This only appears if the port is explicitly specified in the URL. Implicit default ports (e.g., 80 for HTTP) do not appear in the part-list.

path(Path)

(File-) path addressed by the URL. This is supported for the ftp, http and file protocols. If no path appears, the library generates the path /.

search(ListOfNameValue)

Search-specification of HTTP URL. This is the part after the ?, normally used to transfer data from HTML forms that use the GET protocol. In the URL it consists of a www-form-encoded list of Name=Value pairs. This is mapped to a list of Prolog Name=Value terms with decoded names and values.

fragment(Fragment)

Fragment specification of HTTP URL. This is the part after the # character.

The example below illustrates all of this for an HTTP URL.

?- parse_url('http://www.xyz.org/hello?msg=Hello+World%21#x',
       P).

P = [ protocol(http),
      host('www.xyz.org'),
      fragment(x),
      search([ msg = 'Hello World!'
             ]),
      path('/hello')
    ]

By instantiating the parts-list this predicate can be used to create a URL.

 parse_url(+URL, +BaseURL, -Attributes) is det

Similar to parse_url/2 for relative URLs. If URL is relative, it is resolved using the absolute URL BaseURL.

 globalise_path(+LocalPath, +RelativeTo, -FullPath) is det[private]

The first clause deals with the standard URL /... global paths. The second with file://drive:path on MS-Windows. This is a bit of a cludge, but unfortunately common practice is -especially on Windows- not always following the standard

 absolute_url//[private]

True if the input describes an absolute URL. This means it starts with a URL schema. We demand a schema of length > 1 to avoid confusion with Windows drive letters.

 uri(-Parts)//[private]

 schema(-Atom)//[private]

Schema is case-insensitive and the canonical version is lowercase.

Schema ::= ALPHA *(ALPHA|DIGIT|"+"|"-"|".")

 hier_part(+Schema, -Parts, ?Tail)//[private]

 query(-Parts, ?Tail)// is det[private]

Extract &Name=Value, ...

 search_sep// is semidet[private]

Matches a search-parameter separator. Traditionally, this is the &-char, but these days there are `newstyle' ;-char separators.

See also

- http://perldoc.perl.org/CGI.html

To be done

- This should be configurable

 fragment(-Fragment, ?Tail)//[private]

Extract the fragment (after the =#=)

 fragment_char(-Char)[private]

Find a fragment character.

 pchar(-Code)//[private]

unreserved|pct_encoded|sub_delim|":"|"@"

Performs UTF-8 decoding of percent encoded strings.

 lwalpha(-C)//[private]

Demand alpha, return as lowercase

 sub_delim(?Code)[private]

Sub-delimiters

 unreserved(+C)[private]

Characters that can be represented without percent escaping RFC 3986, section 2.3

 www_form_encode(+Value, -XWWWFormEncoded) is det

www_form_encode(-Value, +XWWWFormEncoded) is det

En/decode to/from application/x-www-form-encoded. Encoding encodes all characters except RFC 3986 unreserved (ASCII alnum (see code_type/2)), and one of "-._~" using percent encoding. Newline is mapped to %OD%OA. When decoding, newlines appear as a single newline (10) character.

Note that a space is encoded as %20 instead of +. Decoding decodes both to a space.

deprecated

- Use uri_encoded/3 for new code.

 www_encode(+Codes, +ExtraUnescaped)//[private]

 www_decode(-Codes)//[private]

 set_url_encoding(?Old, +New) is semidet

Query and set the encoding for URLs. The default is utf8. The only other defined value is iso_latin_1.

To be done

- Having a global flag is highly inconvenient, but a work-around for old sites using ISO Latin 1 encoding.

 url_iri(+Encoded, -Decoded) is det

url_iri(-Encoded, +Decoded) is det

Convert between a URL, encoding in US-ASCII and an IRI. An IRI is a fully expanded Unicode string. Unicode strings are first encoded into UTF-8, after which %-encoding takes place.

 parse_url_search(?Spec, ?Fields:list(Name=Value)) is det

Construct or analyze an HTTP search specification. This deals with form data using the MIME-type application/x-www-form-urlencoded as used in HTTP GET requests.

 file_name_to_url(+File, -URL) is det

file_name_to_url(-File, +URL) is semidet

Translate between a filename and a file:// URL.

To be done

- Current implementation does not deal with paths that need special encoding.