:orphan: File Upload Service =================== The **File Upload Service** allows uploading of files to a directory on the Crossbar.io node via (chunked) HTTP/POSTs. The file upload service has some issues, and may be removed in a future release. Use at your own risk! Modern browsers `support `__ the `HTML5 File API `__ which allows users to select files from their local system to be uploaded by the browser. This service can handle big files (GBs) and when combined with `Resumable.js `__ features: - upload one or multiple files - chunked uploading - select or drag & drop to upload - resuming uploads - progress indication via WAMP PubSub events Configuration ------------- To configure a File Upload Service, attach a dictionary element to a path in your :doc:`Web transport ` : +---------------------------+---------------------------------------------------------------------------------------------------------------------------------+ | attribute | description | +===========================+=================================================================================================================================+ | type | MUST be "upload" (required) | +---------------------------+---------------------------------------------------------------------------------------------------------------------------------+ | realm | The realm to which the service session associated with the resource will attach to. (required) | +---------------------------+---------------------------------------------------------------------------------------------------------------------------------+ | role | The role under which the service session associated with the resource will attach under. (required) | +---------------------------+---------------------------------------------------------------------------------------------------------------------------------+ | directory | The folder for completely uploaded files relative to the .crossbar folder in your crossbar node. (required) | +---------------------------+---------------------------------------------------------------------------------------------------------------------------------+ | temp_directory | A folder to hold incomplete uploads. Each incomplete upload will be a subfolder containing the uploaded file chunks. (required) | +---------------------------+---------------------------------------------------------------------------------------------------------------------------------+ | form_fields | Contains the form field mapping between client POST request and backend (required) | +---------------------------+---------------------------------------------------------------------------------------------------------------------------------+ | options.file_types | A JSON Array of permitted file extension strings including the dots. | +---------------------------+---------------------------------------------------------------------------------------------------------------------------------+ | options.file_permissions | The file access permissions to use for the completely uploaded files. (chmod octal code) | +---------------------------+---------------------------------------------------------------------------------------------------------------------------------+ | options.max_file_size | The maximally allowed file size in bytes to upload. Refers to the file not to the chunks of the file. | +---------------------------+---------------------------------------------------------------------------------------------------------------------------------+ The ``form_fields`` dictionary contains the form field names that the client uses to upload files. It has the following configuration parameters: .. code:: javascript "form_fields": { "file_name": "resumableFilename", "mime_type": "resumableType", "total_size": "resumableTotalSize", "chunk_number": "resumableChunkNumber", "chunk_size": "resumableChunkSize", "total_chunks": "resumableTotalChunks", "content": "file", "on_progress": "on_progress", "session": "session", "chunk_extra": "chunk_extra", "finish_extra": "finish_extra" } +---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | attribute | description | +===============+=================================================================================================================================================================================================================================+ | file_name | The name of the form field containing the file name. (The file name is not used for anything in the backend). (required) | +---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | mime_type | The name of the form field to hold the MIME type of the uploaded file. (required) | +---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | total_size | The name of the form field to hold the integer representing the size of the file in bytes. (required) | +---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | chunk_number | The name of the form field to hold the chunk number of the current file chunk. (required) | +---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | chunk_size | The name of the form field holding the chunk size. (required) | +---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | total_chunks | The name of the form field holding the total number of chunks for the file to be transfered. Needs to be POSTed with every chunk. (required) | +---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | content | The name of the form field containing the file content. (required) | +---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | on_progress | Optional name of the form field containing the URI to publish upload related events to. If an URI is provided, progress events will be published as a file is being uploaded. | +---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | session | Optional name of the form field containing the WAMP session ID of the session to which publihed progress event should be restricted. If no session ID is provided, progress events can be received by any (authorized) session. | +---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | chunk_extra | The optional name of the form field to hold a serialized JSON object with custom information that will be sent on every chunk upload completion to any listening client. | +---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | finish_extra | The optional name of the form field to hold a serialized JSON object with custom information that will be sent on file upload completion to any listening client. | +---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ In the example above the file name is passsed to the backend in a POST multipart formdata field with name="myFilename") .. code:: html -------------- File Post processing -------------------- To trigger post processing of files on the server one solution would be to create a WAMP client on the server (e.g. a python component using autobahn-python) which subscribes to the upload topic specified under the form field name given in ``on_progress``. This component then checks the progress payload for the key/value ``status="finished"`` and can also extract custom additional data sent along from the client in the propertie with name given by ``finish_extra``. Upon reception of this event the component can fire off post processing of the file. Another solution would be to use the python library `watchdog `__ to watch on the upload folder. As long as the specified upload-temp folder and the upload folder reside on the same file system, the crossbar file uploader handles files such that all files are *moved* into the upload folder which constitutes an atomic file system operation. Thereby no incompletely copied or downloaded files can be picked up by watchdog. -------------- Resumable Uploads ----------------- To implement resumable uploads crossbar file upload functionality provides a GET response on the same path. The response will either be with - ``Status 200`` which indicates that the file or chunk of file is already pressent in the backend. - A response with any other Status means the file/chunk is not yet present in the backend and should be uploaded. With this service the upload client can check for existence of the chunk in the backend prior to POSTing the chunk. This effectively implements resumable uploads. The GET response needs to have the same arguments as the POST request above. -------------- Example ------- We have a `complete example `__ in the `Crossbar.io examples repository `__ repository. Clone the repo, change to the example folder ``fileupload`` and start Crossbar.io: :: crossbar start To start Crossbar.io with debug log messages: :: crossbar start --loglevel=debug Open http://localhost:8080 in your browser. Open the JavaScript console to see file upload progress events when uploading files. Then either click **Select files to upload** or drop files to **Drop files here to upload**. The uploaded files will appear within the ``uploaded`` subdirectory in the example folder. The example uses this configuration: .. code:: javascript { "workers": [{ "type": "router", ... "transports": [{ "type": "web", ... "paths": { ... "upload": { "type": "upload", "realm": "realm1", "role": "anonymous", "directory": "../uploaded", "temp_directory": "../temp", "form_fields": { "file_name": "resumableFilename", "mime_type": "resumableType", "total_size": "resumableTotalSize", "chunk_number": "resumableChunkNumber", "chunk_size": "resumableChunkSize", "total_chunks": "resumableTotalChunks", "content": "file", "on_progress": "on_progress", "session": "session", "chunk_extra": "chunk_extra", "finish_extra": "finish_extra" }, "options": { "max_file_size": 209715200, "file_permissions": "0644", "file_types": [".csv", ".txt", ".pdf", ".img"] } } } }] }] }