Further explain the custom bucket types idea for mod_file_cache

APACHE 2.0 STATUS:						-*-text-*-

Last modified at [$Date: 2001/06/04 14:20:20 $]

Last modified at [$Date: 2001/06/04 15:29:53 $]

Release:

      for use on a sendfile call (because the filepointer is not used, 

      in sendfile) it is NOT okay for threads to share a file handle

      for reads or writes (because the filepointer is used in reads

      and writes).

      and writes).  This may also be broken on a few quirky platforms

      whose sendfile() actually uses the file pointer.  Jeff was looking

      into that.

      You can share a file handle across threads in Windows if you

      Potential Solutions:

      1. We either need to ensure that a cached file handle is only

         used on a sendfile call (Bill prefers this solution. I think.).

      2. Cliff Woolley has some ideas about a custom bucket type

         that would recognise when an apr_file_t is cached (thus shared

         across threads) and do the right thing to ensure that the

         filepointer does not get trashed.  The idea here is that the

         "cacheable" file bucket would never allow its file handle to be

         read by any method OTHER than sendfile.  Attempts to do otherwise

         would result in the file being re-opened for sync io into the

         request pool, and the new file handle would get put into a regular

         file bucket which would then be read.  This method thus incorporates

         the spirit of solution #1 while having a fall-back protection

         scheme.  Note that mod_file_cache can still attempt to

         be discriminatory and DECLINE requests that it believes will

         result in an apr_bucket_read() call; this solution just allows

         it to not worry TOO much about missing some cases.  Note that

         no locking is involved, and in the worst case (ie, when the cached

         file bucket is read) we are no worse off performance-wise than

         if we'd just DECLINED in the first place.  Patch forthcoming.

      3. You can share a file handle across threads in Windows if you

         open the file for OVERLAPPED io. A file opened for overlapped

         io does not have a file pointer; each thread must manage

         tracking its location in the file explicity. This is a cool

         async io events internal to APR). This doesn't fix the problem

         under *ix though. 

      Potential Solutions:

      1. We either need to ensure that a cached file handle is only

      used on a sendfile call (Bill prefers this solution. I think.).

      2. Cliff Wooley has some ideas about special purpose buckets

      that would recognise when an apr_file_t is cached (thus shared)

      and do the right thing (locking, whatever) to ensure that the

      filepointer does not get trashed.

      3.?

    * There is a big leak of MMAPs that occurs in modules such as

      mod_file_cache that needs to be taken care of.  Several

      potential solutions were tossed about on new-httpd and apr-dev

      in late April/early May.  Perhaps the cleanest proposed

      approaches were the following:

          - dup the cached apr_file_t into the request pool on each

            request so that the MMAP is created in the request pool

          - just cache the FD, not the whole apr_file_t.  Build the apr_file_t

            and its apr_bucket_file in the request pool.

          - add an extra parameter to apr_bucket_file_create() which is the

            pool that an MMAP (if any) for that file should be created in

      [Bill says that this memory leak has been fixed in

      mod_file_cache. We now create a new apr_file_t in the request pool that 

      we send on the ap_send_fd in mod_file_cache. There are other significant

      problems (see above)]

    * There is a bug in how we sort some hooks, at least the pre-config

      hook.  The first time we call the hooks, they are in the correct 

      order, but the second time, we don't sort them correctly.  Currently,