[edit] mod_zip

mod_zip (formerly ngx_zip) is an HTTP module for nginx that assembles ZIP archives dynamically. In simple configurations, mod_zip will take a list of files on the local file system and serve them as a single ZIP archive. In more complex setups, mod_zip can stream component files from upstream servers with nginx's native proxying code. Unlike many ZIP creation scripts, the process never takes up more than a few KB of RAM at time, even while assembling archives that are (potentially) hundreds of megabytes.

[edit] Installation

To install, download the source tarball, expand it, and then compile nginx with the following option:

--add-module=/path/to/mod_zip-1.x

nginx 0.7.25 or greater is required.

[edit] Usage

The module has no configuration directives. It is activated when the original response (presumably from an upstream) includes the following HTTP header:

X-Archive-Files: zip

It then scans the response body for a list of files. The syntax is a space-separated list of the file checksum (CRC-32), size (in bytes), location (properly URL-encoded), and file name. One file per line. The file location corresponds to a location in your nginx.conf; the file can be on disk, from an upstream, or from another module. The file name can include a directory path, and is what will be extracted from the ZIP file. Example:

1034ab38 428    /foo.txt   My Document1.txt
83e8110b 100339 /bar.txt   My Other Document1.txt

Files are retrieved and encoded in order. If a file cannot be found or the file request returns any sort of error, the download is aborted.

If all files in the list have a CRC-32 value, mod_zip will support the "Range" header for the download. Unknown CRC-32's should be indicated with a dash ("-"), e.g.

- 428    /foo.txt   My Document1.txt
- 100339 /bar.txt   My Other Document1.txt

[edit] Remote Upstreams

You can use the following setup to compose archives from multiple remote servers:

1034ab38 428    /server1/foo.txt   My Document1.txt
83e8110b 100339 /server2/bar.txt   My Other Document1.txt

location ~ "^/(?<srv>server[12])/(?[http://wiki.nginx.org/index.php?title=NgxZip&action=raw .*txt)" {
  proxy_pass http://$srv.domain.com/$file
}

[edit] Tips

• add a header "Content-Disposition: attachment; filename=foobar.zip" in the upstream response if you would like the client to name the file "foobar.zip"
• provide the CRC-32 value for files if you'd like to have maximum compatibility with BOMArchiveHelper ('Archiver Utility') and Stuffit Expander.
• Each line should have no whitespace before the CRC-32 field. At the end of the line, \r\n. No empty or extra lines.
• To save bandwidth, add a "Last-Modified" header in the upstream response; mod_zip will then honor the "If-Range" header from clients.
• Make sure the upstream file list is not gzip compressed (e.g. by Apache).
• Make sure the upstream file list has a final newline.

[edit] Changelog

[edit] 1.1

• 1.1.6: Features: Zip64 for large archives, serial subrequests, UTF-8 filenames. Bugfix: Allow spaces in URLs.
• 1.1.5: Features: "If-Range" support, Range support with local files. Bugfix: nginx 0.7.25 compatibility.
• 1.1.4: Feature: Range end is optional (e.g. "bytes=0-"). Bugfix: compilation error on FreeBSD.
• 1.1.3: Feature: optional CRC-32's. Bugfix: support BOMArchiveHelper.app on Mac OS X. Bugfix: occasional crash when file returned 404.
• 1.1.2: Bugfix: Compilation error with nginx 0.6 series.
• 1.1.1: Bugfix: Compilation error on certain platforms.
• 1.1.0: Feature: Full byte-range support. Change: New file list syntax.

[edit] 1.0

• 1.0.1: Initial public release
• 1.0.2: Bugfix: Fixed compilation with no --with-http-debug flag
• 1.0.3: Feature: support for Content-Length when X-Archive-Files-* headers are provided
• 1.0.4: Bugfix: support empty files in an archive
• 1.0.5: Bugfix: support archives larger than 2GB
• 1.0.7: Bugfix: clear outgoing "Accept-Ranges" header
• 1.0.8: Bugfix: strip "Range" header from subrequests

[edit] Bugs

Send bug reports to Evan Miller .

mod_zip at Google Code.

[edit] Thanks

Thanks to box.net for sponsoring the initial development of mod_zip and to vtunnel.com for sponsoring byte-range support.]