File upload in AngularJS causes browser out of memory error

Posted on

Problem

I have an AngularJS code to perform upload of files. At first I thought it works OK but when I’ve tried to upload bigger files (e.g. 5 files, 10 MB each) then I saw that this code has very poor performance regarding memory handling.

For example: when I attach 5 files (10 MB each = 50 MB in total) then peak browser process memory demand reaches 2 GB!
I’ve tested it in Firefox, Chrome and IE. Additionally, in Chrome when browser process reaches 3 GB of memory committed then the page crashes.
In Firefox I can see an error in console when trying to upload more than 80 MB at once:
out of memory

I’ve tried to modify the parts where the actual request is made and removed data: JSON.stringify(formattedData) which was the first bottleneck. And it turned out that stringifying is redundant there.

But the second bottleneck still exists which is the function binArrayToJson which takes the ArrayBuffer and reads the data into the byte array.

var binArrayToJson = function (binArray) {
   var str = [binArray.length];
   for (var i = 0, binLength = binArray.length; i < binLength; i++) {
   str[i] = binArray[i];
   }
   return str;
}

var applyAttachments = function (data1) {
   angular.forEach(data1,
       function (value, key) {
           if (key === "Attachments") {
               for (var i = 0; i < value.length; i++) {
                   if (value[i].file) { //already saved item don't contain 'file' property
                       var reader = new FileReader();

                       reader.onloadend = (function (f) {
                           return function (result) {

                               var arrayBuffer = result.target.result;
                               value[f].fileContent = binArrayToJson(new Uint8Array(arrayBuffer));
                               value[f].isUploaded = true;
                           };
                       })(i);

                       reader.readAsArrayBuffer(value[i].file);
                   }
               }
           }
       });
};


var createMVCModel = function (output) {
   var defaultStringValue = "";
   return {
       id: output.id,
       Name: output.name || defaultStringValue,
       Status: output.status || "Draft",
       Date: output.date || null
       Attachments: output.attachments || []
   };
};

var saveModel = function (data, url) {
   var formattedData = createMVCModel(data);
   var deferred = $q.defer();

   applyAttachments(formattedData);

   var check = function () {
       if (allNewFilesUploaded(formattedData) === true) {
           $http({
                   url: url,
                   method: "POST",
                   data: formattedData,
                   headers: { 'Content-Type': undefined }
               })
               .then(function (result) {
                       deferred.resolve(result);
                   },
                   function (result) {
                       deferred.reject(result);
                   });
       } else {
           setTimeout(check, 1000);
       }

   }
   check();
   return deferred.promise;
};

I omitted the parts where the following requirements are being met:

  • checking number of files (limit is 10)
  • checking size of each file (limit is 10 MB each)
  • checking permitted file extensions
  • sending rest of the (text) data along with files to the server (ASP.NET MVC method)

Solution

Marshaling the file contents as a byte array is obviously problematic.

Browsers can easily upload files through HTML forms, with

<form method="POST" enctype="multipart/form-data">
  <input type="file">
</form>

Adapt the server to handle file uploads using that mechanism, then develop a replacement AJAX client.

Leave a Reply

Your email address will not be published. Required fields are marked *