Layer7 API Management

Our clients needs to pass images with request to the back-end REST web services through the gateway. What is the best way to implement this?. Can the image be sent as an attachment to the request or can it be sent in the request body?. Is there any limitation for the data size in gateway to process such requests? What will be the best method to go about?

You can pass multipart mime messages through the Gateway. All of the parts are available to the Policy if needed, although usually, we may only work on the first one which would typically be the JSON payload being posted.

I just ran a test where I used curl to post a multipart/mixed request with a JSON payload as the first part and an 8MB mp4 movie as the second. The policy at the first Gateway has nothing more than a routing assertion that forwards it to a second Gateway. This simulates the passthrough. The service at the second Gateway parses the mime contents and returns a simple report. It also runs a JSON path on the request message, with the result added to the report.

Here is the policy on the first Gateway:

Here is the corresponding policy on the second Gateway:

And finally, the curl call in all its glory:

jay@harkness ~ $ curl -H "Content-Type: multipart/mixed" -F "metadata=@simple.json; type=application/json" -F "content=@Downloads/VID_20160829_211934.mp4; type=video/mp4" -X POST http://ssg91.l7tech.com:8080/mimeTest/proxy -i
HTTP/1.1 100 Continue

HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: text/plain;charset=UTF-8
Content-Length: 170
Date: Wed, 07 Sep 2016 23:12:38 GMT

OK from ssg83.l7tech.com

JSON Path result=SGML

multipart/mixed; boundary=----------------------------155dc43456f0 - 8673445
application/json - 583
video/mp4 - 8672506jay@harkness ~ $

As you can see it executed the JSON path correctly and accepted the entire video that was attached. I ran the same test with a 500MB video file without issue. I'm not sure what the actual limit is for mime. Note that the io.xmlPartMaxBytes cluster property limits the size of an XML document, but not any mime attachments.

You certainly could put it in the body, but that might cause a huge performance hit if the message requires parsing. Assuming the image itself is never actually touched by the Gateway, why bother parsing it?

So I think the best approach, in this case, is to use Multipart MIME to build out the request to POST rather than put it in the body.

I hope that helps.

Hi Jay/karthik,  

                 I have explored on webservice with attachments.The limits are set in port properties. Also there are some Cluster wide properties that can also apply a cap on request size.

Also you can find more details in the below posts -

FTP of large files 

What is the maximum variable size? 

Thanks,
Abhishek

To which port and cluster properties are you referring? It is important to differentiate the payload from an attachment in this discussion. As far as I am aware the only limit for attachments that are not parsed is the disk space available to /opt/SecureSpan/Gateway/node/default/var/attachments (or whatever is set by com.l7tech.server.attachmentDirectory ), since attachments above a certain size (as set by attachment.diskThrehold) are spooled there. I am told there are other settings that may affect handling attachments, but they are not very well documented.

Cheers!

Jay

Hi Jay,

             Override maximum message size in  Manage Listen Ports. This one does not allow large request (having/not having attachments).  

Thanks,

Abhishek

That setting only applies to the primary part of the request. I.e. if it is not multipart mime, then the whole request. If it is multipart mime then it only applies to the first part. When using attachments this means that the second part, which in most cases would be the actual attachment, is not affected by this setting. You can verify this behaviour yourself using the test jig I described above (with a 9.1 Gateway). My first part (simple.json) is 583 bytes, and the second part (the mp4 file) is 100+MB. Setting that value to 700 fails - note that also includes the HTTP headers, etc. When I set it to 7000, it passes through, meaning that the setting does not appear to be affected by the attachment, which is clearly larger than 7000 bytes. There needs to be a very clear distinction in your mind when configuring the Gateway that attachments are NOT treated the same as the message itself. I'll chat with the doc people about making this clearer in the docs.

Cheers!

Jay