FAQ

Page Discussion Edit History

NginxHttpUploadProgressModule

Contents

Edit section: Nginx Upload Progress Module Nginx Upload Progress Module

Note: this module is not distributed with the Nginx source. Installation instructions are below .

nginx_uploadprogress_module is an implementation of an upload progress system, that monitors RFC1867 POST upload as they are transmitted to upstream servers.

It works by tracking the uploads proxied by Nginx to upstream servers without analysing the uploaded content and offers a web API to report upload progress in Javascript, JSON or configurable format. It works because Nginx acts as an accelerator of an upstream server, storing uploaded POST content on disk, before transmitting it to the upstream server. Each individual POST upload request should contain a progress unique identifier.

The JSON idea and the mechanism idea are based on Lighttpd mod_uploadprogress: http://blog.lighttpd.net/articles/2006/08/01/mod_uploadprogress-is-back

Note:

  • Beside the archive linked to this page, if you want to have access to the very last version, please see the development git repository available at:

http://github.com/masterzen/nginx-upload-progress-module/tree/master


WARNING:

  • when compiled with --with-debug, this module will produce high number of log messages.


Edit section: Directives Directives


Edit section: upload_progress upload_progress

syntax: upload_progress zone_name zone_size

default: n/a

context: server

This directive enables the upload progress module and reserve zone_size bytes to the zone_name which will be used to store the per-connection tracking information.


Edit section: track_uploads track_uploads

syntax: track_uploads zone_name timeout

default: n/a

context: location

This directive enables tracking uploads for the current location. Each POST landing in this location will register the request in the zone_name upload progress tracker.
Since Nginx doesn't support yet RFC 1867 upload, the location must be a proxy_pass or fastcgi location.
Some minial testing indicates that it does work with the phusion pasenger plugin.
The POST _must_ have a query parameter called X-Progress-ID (or an HTTP header of the same name) whose value is the unique identifier used to get progress information. If the POST has no such information, the upload will not be tracked.
The timeout parameter controls the time upload information are kept after the upload request has finished. This allows the upload progress probe to be able to get the "upload done" information or when an error as occurred. Usual value is 30 seconds.
Warning: since version 0.4, the track_uploads must be the last directive of the location (i.e. it should be placed after any proxy_pass or fasctgi_pass).


Edit section: report_uploads report_uploads

syntax: report_uploads zone_name

default: n/a

context: location

This directive allows a location to report the upload progress that is tracked by track_uploads for zone_name. The returned document is a JSON text with the possible 4 results:

  • the upload request hasn't been registered yet or is unknown: 
	
new Object({ 'state' : 'starting' })
  • the upload request has ended: 
new Object({ 'state' : 'done' })
  • the upload request generated an HTTP error: 
new Object({ 'state' : 'error', 'status' : <error code> })

One error code that is interesting to track for clients is HTTP error 413 (Request entity too large)

  • the upload request is in progress: 
new Object({ 'state' : 'uploading', 'received' : <size_received>, 'size' : <total_size>})

It is possible to return pure json instead of this javascript (see upload_progress_json_output). It is also possible to configure completely the response format with the directive: upload_progress_template

The HTTP request to this location must have either an X-Progress-ID parameter or X-Progress-ID HTTP header containing the unique identifier as specified in your upload/POST request to the relevant tracked zone. If you are using the X-Progress-ID as a query-string parameter, ensure it is the LAST argument in the URL.

Edit section: upload_progress_content_type upload_progress_content_type

syntax: upload_progress_content_type <content_type>

default: test/javascript

context: main,sever,location

This directive allows to change the upload progress probe response content-type.

Edit section: upload_progress_header upload_progress_header

syntax: upload_progress_header <progress-id>

default: X-Progress-ID

context: main,sever,location

This directive allows to change the header name of the progress ID.

Edit section: upload_progress_json_output upload_progress_json_output

syntax: upload_progress_json_output

default: n/a

context: main,sever,location

This directive sets everything to output as pure json.

Edit section: upload_progress_template upload_progress_template

syntax: upload_progress_template <state> <template> default: n/a

context: main,sever,location

This directive can be used to install a progress response template. The available list of state is:

  • starting
  • uploading
  • error
  • done

Nginx will replace the value of the following variables with their respective value for the upload:

  • $uploadprogress_length: total size of the upload
  • $uploadprogress_received: what the server has received so far
  • $uploadprogress_status: error code in case of HTTP error

For instance to return XML (instead of the default Javascript or json): 

    upload_progress_content_type 'text/xml';
    upload_progress_template starting '<upload><state>starting</state></upload>';
    upload_progress_template uploading '<upload><state>uploading</state><size>$uploadprogress_length</size><uploaded>$uploadprogress_received</uploaded></upload>';
    upload_progress_template done '<upload><state>done</state></upload>';
    upload_progress_template error '<upload><state>error</state><code>$uploadprogress_status</code></upload>';

Edit section: Configuration Example Configuration Example 

http {
	
  # reserve 1MB under the name 'proxied' to track uploads
  upload_progress proxied 1m;

  server {
    listen       127.0.0.1 default;
    server_name  _ *;
		
    root /path/to/root;

    location / {
      # proxy to upstream server
      proxy_pass http://127.0.0.1;
      proxy_redirect default;
	
      # track uploads in the 'proxied' zone
      # uploads expires 30s after they finish.
      track_uploads proxied 30s;
    }
		
    location ^~ /progress {
      # report uploads tracked in the 'proxied' zone
      report_uploads proxied;
    }
  }

Edit section: Usage Usage

This usage example is taken from Lighttd mod_uploadprogress module example:

First we need a upload form: 

  <form id="upload" enctype="multipart/form-data" 
  action="/upload.php" target="uploadframe" method="post" 
  onsubmit="openProgressBar(); return true;">
    <input type="hidden" name="MAX_FILE_SIZE" value="30000000"  />
    <input name="userfile" type="file" label="fileupload" />
    <input type="submit" value="Send File" />
  </form>
  <iframe id="uploadframe" name="uploadframe" width="0" height="0" frameborder="0" border="0" src="about:blank"></iframe>

And a progress bar to visualize the progress: 

  <div>
   <div id="progress" style="width: 400px; border: 1px solid black">
     <div id="progressbar" 
          style="width: 1px; background-color: black; border: 1px solid white">
           
     </div>
  </div>
  <div id="tp">(progress)</div>
  </div>

Then we need to generate the Unique Identifier and launch the upload on submit action. This also will start the ajax progress report mechanism. Note: the unique identifier shouldn't be reused for the timeout specified in track_upload, otherwise the server will return an HTTP error code 500. This is by design so that probes can still see a finished upload. 

  interval = null;

  function openProgressBar() {
    /* generate random progress-id */
    uuid = "";
    for (i = 0; i < 32; i++) {
      uuid += Math.floor(Math.random() * 16).toString(16);
    }
    /* patch the form-action tag to include the progress-id */
    document.getElementById("upload").action="/upload.php?X-Progress-ID=" + uuid;

    /* call the progress-updater every 1000ms */
    interval = window.setInterval( function () {  fetch(uuid);  }, 1000 );
  }

  function fetch(uuid) {
    req = new XMLHttpRequest();
    req.open("GET", "/progress", 1);
    req.setRequestHeader("X-Progress-ID", uuid);
    req.onreadystatechange = function () {
    if (req.readyState == 4) {
      if (req.status == 200) {
        /* poor-man JSON parser */
        var upload = eval(req.responseText);

        document.getElementById('tp').innerHTML = upload.state;

        /* change the width if the inner progress-bar */
        if (upload.state == 'done' || upload.state == 'uploading') {
          bar = document.getElementById('progressbar');
          w = 400 * upload.received / upload.size;
          bar.style.width = w + 'px';
        }
        /* we are done, stop the interval */
        if (upload.state == 'done') {
          window.clearTimeout(interval);
        }
      }
    }
  }
  req.send(null);
}


Edit section: Installation Installation

This module is not distributed with the Nginx source.
You can download the ningx_uploadprogress_module here:
http://github.com/masterzen/nginx-upload-progress-module/tarball/v0.7
Or you can get the source with git directly from my github repository: http://github.com/masterzen/nginx-upload-progress-module/tree/master

After extracting, add the following option to your Nginx ./configure command: 

  --add-module=path/to/nginx_uploadprogress_module

Then "make" and "make install" as usual.

Edit section: Safari-enabling patch (uploadprogress-module &lt; 0.3) Safari-enabling patch (uploadprogress-module < 0.3)

There is an incompatibility between this version of nginx_uploadprogress_module and the Safari web browser. The problem is caused by the fact that the rules Safari uses to "normalize" case of HTTP request headers varies from the rules followed by Firefox and IE, and by the fact that nginx_uploadprogress_module looks for the X-Progress-ID HTTP request header case-sensitively. See this article for more information, or If you don't care to actually read about it just download the patch to make the lookup case-insensitive.

This particular problem has been fixed since v0.3 which now uses case-insensitive lookup to accomodate Safari and Internet Explorer (Brice Figureau).

Retrieved from "http://wiki.nginx.org/NginxHttpUploadProgressModule"