0
Not a bug

Uploading issue. Temp file size configuration.

Dmitry 11 months ago • updated by Vlad R 2 weeks ago 9 1 duplicate

I found an issue on using fast Internet connection. Uploading works quite slow and upload speed indication is quite inaccurate.


The Internet connection speed is 100Mbps. And it gets stuck on a few seconds every 20Mb. I found that 20Mb is the size of temp file that is creating in php temp folder. It means that the stuck happens when the file is moving from the temp folder to the final destination.


I didn't set limits that can influence such behavior (actual limits much higher everywhere). So it means that this setting is somewhere in FileRun.


I would like to edit it and increase this temp chunk file size. Because a moving file action takes the same time as uploading 20Mb for a such internet connection speed.


Upload speed indicator shows the speed excluding the time for a temp file moving. The actual uploading speed is 2x slower.


PS: when I use a local 1Gbits connection this stuck takes much less time and doesn't affect so much the uploading speed. I think a ping latency somehow affects this behavior.

Answer

Answer
Not a bug

The upload chunk size is automatically set by FileRun based on your PHP configuration. To increase this size, and have less chunking done, increase the values of "upload_max_filesize" and "post_max_size" in your PHP configuration (php.ini).


As observed, FileRun will limit the chunk size by default to a maximum of 20MB. If you wish to increase this, you will need to set $config['app']['upload']['chunk_size'] to a bytes value inside "customizables/config.php" (see http://docs.filerun.com/advanced_configuration).


Do note that some HTTP servers also employ various limitations, so if you go too high with the two values and uploads start failing, the solution is to decrease the values or remove the limitations in the HTTP or proxy server.


P.S. Note that a very large chunk size would also affect FileRun's ability to resume interrupted transfers, which might be needed for users that do not have a fast network connection.

Duplicates 1

Answer
Not a bug

The upload chunk size is automatically set by FileRun based on your PHP configuration. To increase this size, and have less chunking done, increase the values of "upload_max_filesize" and "post_max_size" in your PHP configuration (php.ini).


As observed, FileRun will limit the chunk size by default to a maximum of 20MB. If you wish to increase this, you will need to set $config['app']['upload']['chunk_size'] to a bytes value inside "customizables/config.php" (see http://docs.filerun.com/advanced_configuration).


Do note that some HTTP servers also employ various limitations, so if you go too high with the two values and uploads start failing, the solution is to decrease the values or remove the limitations in the HTTP or proxy server.


P.S. Note that a very large chunk size would also affect FileRun's ability to resume interrupted transfers, which might be needed for users that do not have a fast network connection.

"I didn't set limits that can influence such behavior (actual limits much higher everywhere). So it means that this setting is somewhere in FileRun."


- I wouldn't create a new topic if it was because of "upload_max_filesize" and "post_max_size" settings. I tried different settings from 8M to 500M. And behavior is the same when it is set to 20M+! If it is set less then 20M, so it gets stuck earlier. My IIS web-server doesn't have upload file size limits anywhere as well. I've checked it.


So I want a solution that can resolve this quite critical issue.


I think it would be good to create 2 temp files for each uploading file. So when it is moving to the final destination it is still uploading the next chunk.


PS: You said nothing about uploading speed indication issue that I've mention before.

PS2: I posted as Anonymous by mistake, please delete it.

The answer has been updated. (Additional clarification is enough for receiving more information and down-voting the post marked as answer is not necessary.)


You said nothing about uploading speed indication issue that I've mention before.

Accounting for the time your server spends on processing each request, for calculating the upload speed, is not only a difficult task but is also of little importance. If you need accurate transfer speed measurements, perhaps you should consider different upload methods. In the FileRun context, that particular information is by no means considered critical as it is supposed to be only a rough estimate.

Thank you for additional information. I had checked /advanced_configuration before but didn't found this parameter there. So it would be good to add it.

For my case (100Mbps & 20MB chunk size) the upload speed difference is 2 times, I think it is not accurate at all =)


PS: I dont know why, but each time I am logging in here my name is flushed to a default one.


Dmitry did you ever managed to resolve this issue ? i am facing the same problem.

Under review

Any customizations in your FileRun installation?

The 20MB mark means PHP is configured with either "upload_max_filesize" or "post_max_size" set to 20M, thus forcing FileRun to upload larger files in smaller pieces. The long break means that for some reason, the web server takes a long time to process the request (perhaps there is something causing PHP to takes a long time to move the 20M chunk from the temporary upload location).

Uploads through the user interface have no limitations. Uploads through web links are hard set to 20MB chunks. Most probably that is what the OP was referring to, but failed to mention.