Known issues in Cloud Connectors

Learn about the known issues in Cloud Connectors, the impact or changes to the functionality, and the workaround.

CDPD-76378 Uploading files to S3 takes longer than expected

Uploading files to S3 on long haul networks is delayed if the file is smaller than 64MB. As the 100-continue header is not received immediately, the files are uploaded only after several retries. The delay in uploading is caused by the fs.s3a.connection.expect.continue S3A configuration parameter. The fs.s3a.connection.expect.continue S3A configuration property controls whether or not a PUT request to the S3 store sets the "Expect: 100-continue" header, and awaits a 100 CONTINUE response before uploading any data. This allows throttling and other transient problems to be reported by the S3 store before there is an attempt by the client to upload any data.

By default, the value of the configuration property is set to true, which means that the header is set, and a 100 CONTINUE response is required.

If there is a significant delay in uploading files or if the write time of jobs increases, you can set the fs.s3a.connection.expect.continue configuration property for your service to false.