How to Upload images without Base64 encoding

Events happening in the community are now at Drupal community events on www.drupal.org.
metow's picture

In my android application , I can upload files to my Drupal 6 server using services module file resource in which I first Base64 encode the file and send it via a StringEntity as follows:

HttpClient httpClient   =   new DefaultHttpClient(connectionParameters);
HttpContext localContext = new BasicHttpContext();

JSONObject json = new JSONObject();
json.put("file", Base64.encodeToString(photodata,Base64.DEFAULT));
json.put("uid", getUid());   
json.put("filename", file_name);

StringEntity se = new StringEntity(json.toString());
se.setContentType(new BasicHeader(HTTP.CONTENT_TYPE, "application/json")); 

HttpPost httpPost = new HttpPost(conProp.url);
httpPost.setEntity(se);
httpPost.setHeader("Cookie", cookie);           
try {                                     
   response = httpClient.execute(httpPost,localContext);                      
      } catch (Exception e) {
reasonphrase = e.getMessage();
}               

However, There are problems with this approach.
Base64 encoding is not the best way to do that in terms of bandwith.
Furthermore

  response = httpClient.execute(httpPost,localContext);

is a single line of code and sends the whole post data in one step, which is something I doubt in terms of healthy posting operations. I want to be able to send the data in buffers and show the progress of the upload to the user.

For this purpose I tried the following instead:


HttpURLConnection connection = null;
DataOutputStream outputStream = null;
DataInputStream inputStream = null;          

String pathToOurFile = file_path;
String urlServer = conProp.getUrl();
String lineEnd = "\r\n";
String twoHyphens = "--";
String boundary =  "AaB03x";

int bytesRead, bytesAvailable, bufferSize;
byte[] buffer;
int maxBufferSize = 1 * 1024 * 1024;

FileInputStream fileInputStream = new FileInputStream(new File(pathToOurFile) );

URL url = new URL(urlServer);
connection = (HttpURLConnection) url.openConnection();

// Allow Inputs & Outputs
connection.setDoInput(true);
connection.setDoOutput(true);
connection.setUseCaches(false);

// Enable POST method
connection.setRequestMethod("POST");
connection.setRequestProperty("Cookie",conProp.getCookie());
connection.setRequestProperty("Connection", "Keep-Alive");
connection.setRequestProperty("Content-Type", "multipart/form-data;boundary="+boundary);

outputStream = new DataOutputStream( connection.getOutputStream() );
String apos = "'";
outputStream.writeBytes(twoHyphens + boundary + lineEnd);
outputStream.writeBytes("Content-Disposition: form-data; name="+apos+"uid"+apos+lineEnd+lineEnd);

outputStream.writeBytes(uid+"");
outputStream.writeBytes(lineEnd);
           
outputStream.writeBytes(twoHyphens + boundary + lineEnd);
outputStream.writeBytes("Content-Disposition: form-data; name="+apos+"filename"+apos+lineEnd+lineEnd);
outputStream.writeBytes(filename);
outputStream.writeBytes(lineEnd);
        
outputStream.writeBytes(twoHyphens + boundary + lineEnd);
outputStream.writeBytes("Content-Disposition: file; name="+apos+"file"+apos+"; filename="+apos+filename+apos+lineEnd);
outputStream.writeBytes(lineEnd);

bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
buffer = new byte[bufferSize];

// Read file
bytesRead = fileInputStream.read(buffer, 0, bufferSize);

while (bytesRead > 0)
{
   outputStream.write(buffer, 0, bufferSize);
//outputStream.flush();
    bytesAvailable = fileInputStream.available();
  bufferSize = Math.min(bytesAvailable, maxBufferSize);
  bytesRead = fileInputStream.read(buffer, 0, bufferSize);
}
    
outputStream.writeBytes(lineEnd);
outputStream.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);

int status_code = connection.getResponseCode();
String conresponse  = connection.getResponseMessage();
         
          
fileInputStream.close();
outputStream.flush();
outputStream.close();

which seem to suit to my idea (splitting the upload into buffers and using multi-part form data). But what this code does when I debug is that it works pretty fast up to the line

int status_code = connection.getResponseCode();

waits 60 sec on that line and the get server timeout error. However the message includes 200 OK, but no file gets stored in the file table.

with that I guess I'm sending the following to the server:

Content-Type: multipart/form-data; boundary=AaB03x

   --AaB03x
   Content-Disposition: form-data; name="uid"

   3
   --AaB03x
   Content-Disposition: form-data; name="filename"

   MyFile.jpg
   --AaB03x
   Content-Disposition: file; name="file"; filename = "MyFile.jpg"

   {file byte data... }
   --AaB03x--

Something might also be wrong with the data I'm providing above. So as you guys know how services work, and me not knowing what the service functions demand (no documentation about that??) maybe someone can help get me get through

Comments

Take a look at the postFile

IncrediblyKenzi's picture

Take a look at the postFile method on the Dandy toolkit:

https://github.com/workhabitinc/dandy/blob/master/dandy-api/src/main/jav...

It simplifies the post to not require a JSON envelope in the upload, and hands the inputStream directly to the httpClient, thus conserving memory (which would otherwise have required 3+ times the file size in order to handle the post.

Hope this helps.

Also, take a look at

IncrediblyKenzi's picture

Also, take a look at http://stackoverflow.com/questions/254719/file-upload-with-java-with-pro... for how to wrap the Multipart request to count bytes being returned.

Services

Group organizers

Group categories

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds: