[DONE] HTTP put() not working as expected for binary file

[Kollaborateur]

Hi there,

trying to submit an image from Tape via http.put()
My code:

const image_field_url = linkedin_post_field_image_file_url=
const img_data = await http.get(image_field_url, { responseType: 'arraybuffer' })
const img_data_binary = await http.get(image_field_url, { responseEncoding: 'binary' })
const buf_a = Buffer.from(img_data.data)
const buf_b = Buffer.from(img_data_binary.data)

const call_url = 'https://webhook.site/daf73eda-e68c-4463-9acb-1cbf9e8194de'
const headers = {
    'content-type': 'image/png',
}
const repsonse_a = await http.put(call_url, {
    headers,
    data: buf_a
})
const repsonse_b = await http.put(call_url, {
    headers,
    data: buf_b
})
console.log('response_a', JSON.stringify(repsonse_a))
console.log('response_b', JSON.stringify(repsonse_b))

The result is always a broken image. I’ve tried several image/jpeg and image/png but got always the same result.

Results can be seen here: https://webhook.site/#!/view/daf73eda-e68c-4463-9acb-1cbf9e8194de/25e1d543-e60c-4af4-9541-4ec995889c73/1

This is how the file looks after it has been sent through http.put(…). The Buffer has been converted to a JSON text representation which means the file cannot be opened as an image anymore.

For context: For this particular API we must send the image data as raw binary inside the HTTP content body. This API does neither accept files as “Content-Type: multipart/form-data” nor as Base64-endoded data in a JSON field.

[Jason]

Hi @Kollaborateur, just in case this helps you (it only will if you can switch to ‘POST’ so may well not):

const fileLink = document_field_preview_file_url;
console.log(fileLink);
const filename = document_field_preview_filename;

if (!fileLink) {
    throw new Error ('File link not found!');    
}

// Download the file
const fileRes = await http.get(fileLink, {responseEncoding: 'binary'});
const fileContent: string = fileRes.data;

// const url = 'https://webhook.site/b1832408-fc03-4312-b7cb-df57c1f68f12';
const url = 'https://n8n.jmc.tools/webhook-test/4b9602f9-afaf-4847-82b0-cfc9f23ccc72';

// Compose form data to send
let formData = {
    'outputFormat': 'png',
    "fileInput": {
        buffer: Buffer.from(fileContent, 'binary'),
        filename,
        contentType: 'image/png'
    }
};

const headers = {
    'accept': '*/*',
}

// Send the request
const r = await http.postFormData(url, {
    headers,
    formData
});

console.info(JSON.stringify(r, null, 2));

However with the quick look I had I was also unable to send the file using normal http.post or http.put.

[Kollaborateur]

Thanks for confirming @Jason

@Jason

However with the quick look I had I was also unable to send the file using normal http.post or http.put.

Unfortunately I can’t switch to POST with this one.

@Leo @Tim could you please look into this.

[Kollaborateur]

Hi @Leo @Tim,

could you let me know a time frame when you’ll be able to look into this?

Jason and I both confirmed that it isn’t working properly, and it has unfortunately brought my project to a sudden stop. It’s a bit frustrating since there’s no workaround within Tape.

Thanks a lot!

[Leo]

Hi @Kollaborateur,

Thanks a lot for your report, and also many thanks to @Jason for double-checking and confirming the issue. We really appreciate your support here.

We’ll take a closer look at this with priority to better understand why the image cannot be sent. As soon as we have first findings, we’ll get back to you right away here.

Best regards,
Leo

[Tim]

Hi @Kollaborateur ,

thank you for sharing your issue.

In the past, users also encountered issues with external APIs that expect a binary file for processing, e.g. for a Autentique signature API :

[✅ Solution] Autentique API FormData requests

@gracosilva thanks for your patience - after we reproduced your use case inside our lab using an Autentique account, we figured it is hardly possible with the regular http.post method. In order to allow using the form data spec inside of workflow automations, we added a new method postFormData to the Workflow execution context HTTP client that handles the implementation details. With that, it is now easily possible to create the PDF documents from Tape. We added a simple guide that concludes yo…

The only solution at the end of the day was providing a dedicated postFormData function in the http client library, which was used in this showcase:

Integrate Tape with digital signature provider Autentique

Due to various community requests from Brazil , we created this guide that shows how to create a signature document inside of Autentique using their API and Tape workflow automations. We are going to use a PDF that is stored inside Tape - it could also be generated beforehand inside of Tape using the PDF action. This enables countless use cases, e.g. creating contract documents based on your data in Tape and having them signed by your clients via Autentique …

That corresponds with the solution proposed by @Jason (thank you!) - but if you cannot switch to POST does not provide a solution to your issue.

What we can offer: Add a matching PUT version for that function, that should be able to send the binary file directly to your external API’s PUT endpoint using the same form data wrapper. Do you think that could be an option? As it worked for POST in the past, we would expect it to work for your case as well.

Looking forward to your feedback
Tim

[uwekamper]

Hi Tim,

(TLDR: We found a work-around for the API that we need, but the root cause is still there.)

I am working with @Kollaborateur on this. The postFormData is sending data as “Content-Type: multipart/form-data” and not as binary.

The documentation of this particular API says to do the following:

• Create an an upload request (POST request to the initializeImageUpload endpoint), you will get an image ID and upload URL.
• Then do a PUT request to the upload URL with the binary image data as the content body of the HTTP request (NOT multipart/form-data).
• Do a GET request for the image ID and get the status. If the status is “AVAILABLE”, then everything went well.

Our work-around: We did a lot of testing back and forth with the API in question. We now have a solution that works, but not as documented. The API in question seems to both accept PUT and POST requests and if the POST request contains a multipart/form-data with a file field called “file” - only then it will accept also POST requests.

So we have a solution to our problem now!

The addition of putFormData is a good idea and you should make a feature request out of that.

But it would not have helped in this case. With the current state of the tape “http” interface it is impossible to send a binary HTTP payload:

http.post(url, {"data": imageFileBuffer, "headers": {"Content-Type: image/png"});

does not do an upload of the binary image data. Instead the “data” option is converted to JSON.

The following will convert the binaryImage data into “multipart/form-data” (used by most APIs) - but this is also not binary encoding:

const formData = {
  "file": {
    "buffer": binaryImageBuffer,
    "filename": "image1.png",
    "contentType": 'image/png'
  }
}
http.post(url, {"formData": formData);

As far as I can see there is currently no way to send actual binary as the content body of a HTTP request from Tape automations to APIs.

Hope that explains it a little bit more.