MiNTnet OVerLay version 1.86 for CAB

Introduction

This is a network interface module for Alexander Clauss's CAB web browser for
Atari STs. But you probably knew that already, or else you wouldn't have
bothered to retrieve these files... Anyway, this module uses the socket
library provided with Kay Roemer's MiNTnet network drivers for MiNT. Currently
the module fully supports the HTTP/1.1 protocol, as well as using SMTP for
sending outgoing email.

This module evolved from code written by Andrew Vesperman, but it's been
rewritten pretty heaviliy since he last touched it. It runs as a child process
of the main CAB process. I wrote it this way because I was frustrated by the
original module's behavior, which didn't allow you to interrupt slow
transactions. (I.e., the STOP button in CAB usually was ineffective.) Along
the way, I cleaned up a lot of the original code, introduced my own brand of
messiness, and generally extended it.

Features

You probably already know this too, but anyway:
	Full support of HTTP/1.1 including
		persistent/reusable connections
		1.1 transfer encodings
		both 1.0 and 1.1 cookies
		basic authentication/authorization

	Supports ftp/gopher/wais/news via http-based proxy
	Netscape SSL (Secure Sockets Layer) for secure connections
		(must be obtained outside the USA - you can get the
		SSL library from http://psych.psy.uq.oz.au/~ftp/Crypto/.
		You will need to copy the "certs" directory from the
		library source code to /usr/local/ssl/certs on your
		machine to get server verification.)
	
See the readme.hyc file for more up to date information.

Configuration

CAB-provided configuration

There's not a lot to it. For proxy setup, it uses the information provided
in CAB's configuration menu. A proxy entry should be specified in URL format,
e.g.
	FTP proxy: http://my-proxy.mydomain.com:8080

The "No proxy" entry is a comma-separated list of host or domain names. (You
can also use semicolons if you want.) A name may also optionally be followed
by a colon and a port number, in case you want to filter by individual service
type. A domain name is distinguished from host names with a leading '.'. e.g.

	 No proxy: some.host.com,.local.domain,secure.host:443

Specifying a domain name will bypass proxies for every host within that domain.

The SMTP server entry is used for sending outbound email. The NNTP server
entry is currently unused, but would otherwise be used for news:xxx URLs.

CABOVL.CNF file

This plain text file should be stored in the same directory as the OVL itself.
(The OVL should go in the same folder as CAB itself, unless you're using CAB
2.5 or newer. Then you should put it in the cab/modules folder, so that the
AutoSurf and Download modules can also access it.) Here are the keywords for
the config file:

	gzip=yes

	The OVL now uses libz and can decompress gzip'd data on the fly.
Set this variable to "yes" if you want to enable this decompression. When set,
the OVL will include a line in its requests that tells the server it can
decompress files on the fly. If the server is configured to take advantage of
this feature, then you can experience some speedup in your transfers. Only
files of type "text/*" will be decompressed, anything else will be left in
compressed form. (E.g., files of type "text/html" or "text/plain" will be
decompressed, but files of type "application/tar" etc. will be left alone.)

	cookies=reject

	Set this variable to control the OVL's treatment of HTTP cookies. The
default is "reject" which means that all cookies are silently ignored. You can
also set it to "prompt" in which case you are prompted every time a server
tries to send you a cookie. You can also choose "server" which will silently
accept cookies intended for the same host that sent them; it will prompt if
the cookie is set for a domain instead of a host. (It is illegal for a host
to set a cookie that is intended for a different, unrelated host.) Finally,
you can use "accept" which will silently accept all legal cookies. Illegal
cookies are always rejected silently.

	Note that as of CAB 2.7, CAB itself can also manage cookies. This OVL
will handle cookies for any version of CAB noo matter how old or how new. When
used with CAB 2.7 or newer, the OVL will use CAB's cookie manager in addition
to its own. Specifically, any long-lived cookies will be given to CAB to deal
with. Single-session cookies will be maintained by the OVL itself.

	mailfrom=joe.user@hotmail.com

	Set this to the email address you want to use in From headers for your
outgoing email.

	maxhosts=16

	The OVL caches information about every host and every URL you access.
This setting controls the maximum number of hosts that will be remembered.

	maxnodes=32

	This setting controls how many object paths will be remembered for a
particular host in the info cache.

	postredir=manual

	Normally when a POST gets redirected, the OVL asks you what to do
next. The HTTP spec says posts should never be automatically redirected, due
to various security vulnerabilities this can open up. Also, the spec says that
when redirecting, you must exactly repeat the original request that caused the
redirection. However, most servers violate the spec and expect the client to
change the POST into a GET when it is redirected. If you're tired of being
asked what to do with the redirects, set this to "postredir=auto" and most
of those cases will be taken care of silently.

Environment

Setting the CMDEBUG environment variable before invoking CAB will cause the
OVL to log its activity into a "cmdebug.log" file. This file will appear in
the same directory as CAB (or the module, if the OVL is invoked by a module).

The OVL will use the gethostname call to get the name of the current machine.
This name is only used on SMTP connections, in the initial "HELO" greeting.

That pretty much wraps up all there is to know about configuration.

Miscellaneous Information

The OVL does its best to eliminate redundant network transactions, caching as
much relevant info in memory as possible. Note that the info cache is only
kept in memory, nothing is ever saved to disk. So, when you exit CAB, it all
goes away...

The info cache is based on a list of hosts. The host list is maintained in
most-recently used order. Each entry in the list stores a hostname, up to 4
IP addresses for this host, a list of "domains" for this host, and a list of
paths referenced on this host. Each path entry holds the pathname of the
object, its size, modification time, MIME type, and Entity Tag. (The entity
tag is a unique identifier for a URL, introduced in HTTP/1.1. It is used in
Conditional Gets instead of or in addition to modification time, to determine
if an object needs to be re-downloaded or not.) The MIME type describes the
type of the object, e.g. "text/html" or "image/gif" or whatever...

CAB 1.5 uses all of this (except Entity Tags) via the get_url_info call.
CAB 2.5 doesn't use this call any more, so in fact CAB 2.5 never knows the
size or type of the URLs that it accesses. Instead it uses the
get_url_if_modified call, which uses the modification time only.

The OVL still tracks the modification time if given by the server, and sets
the timestamp on the cachefile appropriately. Note that on regular GEMDOS
filesystems timestamps can only be stored down to a 2-second resolution. As
such, it's best to put the CAB cache onto a Minix filesystem or some other
filesystem that actually preserves timestamps with 1-second resolution.

A domain object holds a domain name and a list of paths which have additional
cookie or authentication information. Any web server can set cookies that will
apply to itself or any other host in its domain. Likewise, a server can demand
authentication info, which will apply to any host or domain it specifies.

Any questions? Email me. mailto:hyc@highlandsun.com?subject=MiNTnetOVL