Commit 93aed1e6 authored by Martin Kraemer's avatar Martin Kraemer
Browse files

Add a preliminary EBCDIC porting paper (derived from README.EBCDIC) 


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@80689 13f79535-47bb-0310-9956-ffa450edef68
parent c2f0d3f6

docs/manual/ebcdic.html

0 → 100644
+201 −0
Changes for docs/manual/ebcdic.html: 201 added lines, 0 removed lines.
Original line number Diff line number Diff line 
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">

<HTML>

<HEAD>

<TITLE>The Apache EBCDIC Port</TITLE>

</HEAD>



<!-- Background white, links blue (unvisited), navy (visited), red (active) -->

<BODY

 BGCOLOR="#FFFFFF"

 TEXT="#000000"

 LINK="#0000FF"

 VLINK="#000080"

 ALINK="#FF0000"

>

<!--#include virtual="header.html" -->

<H1 ALIGN="CENTER">Overview of Apache EBCDIC Port</H1>



 <P>

  Version 1.3 of the Apache HTTP Server is the first version which

  includes a port to a (non-ASCII) mainframe machine which uses

  the EBCDIC character set as its native codeset.<BR>

  (It is the SIEMENS NIXDORF family of mainframes running the

  <A HREF="http://www.sni.de/servers/bs2osd/osdbc_us.htm">BS2000/OSD

  operating system</A>. This mainframe OS nowadays features a

  SVR4-derived POSIX subsystem).

 </P>



 <P>

 The port was started initially to

  <UL>

  <LI> prove the feasibility of porting

       <A HREF="http://dev.apache.org/">the Apache HTTP server</A>

       to this platform

  <LI> find a "worthy and capable" successor for the venerable

       <A HREF="http://www.w3.org/Daemon/">CERN-3.0</A> daemon

       (which was ported a couple of years ago), and to

  <LI> prove that Apache's preforking process model can on this platform

       easily outperform the accept-fork-serve model used by CERN by a

       factor of 5 or more.

  </UL>

 </P>



 <P>

  This document serves as a rationale to describe some of the design

  decisions of the port to this machine.

 </P>



 <P>

  One objective of the EBCDIC port was to maintain enough backwards

  compatibility with the (EBCDIC) CERN server to make the transition to

  the new server attractive and easy. This required the addition of

  a configurable method to define whether a HTML document was stored

  in ASCII (the only format accepted by the old server) or in EBCDIC

  (the native document format in the POSIX subsystem, and therefore

  the only realistic format in which the other POSIX tools like grep

  or sed could operate on the documents). The current solution to

  this is a "pseudo-MIME-format" which is intercepted and

  interpreted by the Apache server (see below). Future versions

  might solve the problem by defining an "ebcdic-handler" for all

  documents which must be converted.

 </P>



 <P>

  Since all Apache input and output is based upon the BUFF data type

  and its methods, the easiest solution was to add the conversion to

  the BUFF handling routines. The conversion must be settable at any

  time, so a BUFF flag was added which defines whether a BUFF object

  has currently enabled conversion or not. This flag is modified at

  several points in the HTTP protocol:

  <UL>

   <LI><STRONG>set</STRONG> before a request is received (because the

       request and the request header lines are always in ASCII

       format)



   <LI><STRONG>set/unset</STRONG> when the request body is

       received - depending on the content type of the request body

       (because the request body may contain ASCII text or a binary file)



   <LI><STRONG>set</STRONG> before a reply header is sent (because the

       response header lines are always in ASCII format)



   <LI><STRONG>set/unset</STRONG> when the response body is

       sent - depending on the content type of the response body

       (because the response body may contain text or a binary file)

  </UL>

 </P>



<H1 ALIGN="CENTER">Porting Notes</H1>

 <P>

  <OL>

   <LI>

   The relevant changes in the source are #ifdef'ed into two

   categories:

   <DL>

    <DT><CODE><STRONG>#ifdef CHARSET_EBCDIC</STRONG></CODE>

    <DD>Code which is needed for any EBCDIC based machine. This

	includes character translations, differences in in

	contiguity of the two character sets, flags which

	indicate which part of the HTTP protocol has to be

	converted and which part doesn't etc.

    <DT><CODE><STRONG>#ifdef _OSD_POSIX</STRONG></CODE>

    <DD>Code which is needed for the BS2000 SIEMENS NIXDORF

	mainframe platform only. This deals with include file

	differences and socket implementations topics which are

	only required on the BS2000/OSD platform.

   </DL>

   </LI><BR>



   <LI>

    The possibility to translate between ASCII and EBCDIC at the

    socket level (on BS2000 POSIX, there is a socket option which

    supports this) was intentionally <EM>not</EM> chosen, because

    the byte stream at the HTTP protocol level consists of a

    mixture of protocol related strings and non-protocol related

    raw file data. HTTP protocol strings are always encoded in

    ASCII (the GET request, any Header: lines, the chunking

    information etc.) whereas the file transfer parts (i.e., GIF

    images, CGI output etc.) should usually be just "passed thru"

    by the server. This separation between "protocol string" and

    "raw data" is reflected in the server code by functions like

    bgets() or rvputs() for strings, and functions like bwrite()

    for binary data. A global translation of everything would

    therefore be inadequate.<BR>

    (In the case of text files of course, provisions must be made so

    that the documents are always served in ASCII format)

   </LI><BR>



   <LI>

    This port therefore features a built-in protocol level conversion

    for the server-internal strings (which the compiler translated to

    EBCDIC strings) and thus for all server-generated documents.

    The hard coded ASCII escapes \012 and \015 which are

    ubiquitous in the server code are an exception: they are

    already the binary encoding of the ASCII \n and \r and must

    not be converted to ASCII a second time. This exception is

    only relevant for server-generated strings; and <EM>external</EM>

    EBCDIC documents always go through a bijective EBCDIC &lt;-&gt; ASCII

    translation table.

   </LI><BR>



   <LI>

    By examining the call hierarchy for the BUFF management

    routines, I added an "ebcdic/ascii conversion layer" which

    would be crossed on every puts/write/get/gets, and a

    conversion flag which allowed enabling/disabling the

    conversions on-the-fly. It is now possible to read the header

    lines of a CGI-script output in EBCDIC format, and then find

    out that the remainder of the script's output is in ASCII

    (like in the case of the output of a WWW Counter program: the

    document body contains a GIF image). Likewise, the server

    always generates its header lines in EBCDIC (and with ASCII

    conversion enabled) and determines, based on the type of

    document being served, whether the document body (except for

    the chunking information, of course) is in ASCII already or

    is converted from EBCDIC.

   </LI><BR>



   <LI>

    For Text documents (MIME types text/plain, text/html etc.),

    an implicit translation to ASCII can be used, or (if the

    users prefer to store some documents in raw ASCII form for

    faster serving, or because the files reside on a NFS-mounted

    directory tree) can be served without conversion.

    <BR>

    <STRONG>Example:</STRONG><BLOCKQUOTE>

	to serve files with the suffix .ahtml as a raw ASCII text/html

	document without implicit conversion (and suffix .ascii

	as ASCII text/plain), use the directives:<PRE>

      AddType  text/x-ascii-html  .ahtml

      AddType  text/x-ascii-plain .ascii

      </PRE></BLOCKQUOTE>

    Similarly, any text/XXXX MIME type can be served as "raw ASCII" by

    configuring a MIME type "text/x-ascii-XXXX" for it using AddType.

   </LI><BR>



   <LI>

    Non-text documents are always served "binary" without conversion.

    This seems to be the most sensible choice for, .e.g., GIF/ZIP/AU

    file types. This of course requires the user to copy them to the

    mainframe host using the "rcp -b" binary switch.

   </LI><BR>



   <LI>

    Server parsed files are always assumed to be in native (i.e.,

    EBCDIC) format as used on the machine, and are converted after

    processing.

   </LI><BR>



   <LI>

    For CGI output, the CGI script determines whether a conversion is

    needed or not: by setting the appropriate Content-Type, text files

    can be converted, or GIF output can be passed through unmodified.

    An example for the latter case is the wwwcount program which we ported

    as well.

   </LI><BR>

  </OL>

 </P>



<!--#include virtual="footer.html" -->

</BODY>

</HTML>