0
Not a bug

Large file uploaded = wrong size in FileRun, OK file size on file system

Guillaume B 2 years ago updated 2 years ago 4

I uploaded a 9.2GB file on my FileRun install; it worked as expected, up until the uploaded finished.

FileRun is reporting the file as 1.1GB; no idea where this value is from, since when I look on the filesystem, I can see the .upload file increase in size during the upload, up to 9.2GB, and the final file is indeed 9.2GB.

Trying to watch the .mp4 file in question from my FileRun install, it stops at around 10%, and downloading it from FileRun results in a 1.1GB file. But the file is OK on the filesystem; I even did a checksum, and it's the same for the local and uploaded file.

Here's a screen capture of the file being uploaded: https://photos.app.goo.gl/SjTSFeC1r7BL5aWN9

I uploaded other large files with similar results: a 7.2GB file appears as 3.2GB in FileRun; another 6.1GB file appears as 2.1GB in FileRun, 7.8GB => 3.7GB, etc.

OS: Raspbian (Debian 9 Stretch on Raspberry Pi 3 armv7l)

FileRun version : 2018.11.11

PHP version: 7.0.27

I just noticed that the file size FileRun reports is always exactly equal to filesize modulo 4GB.

The 1.1GB that FileRun reports for this file is actually 1,198,614,939 bytes. And the original file is 9,788,549,531. And 9,788,549,531 minus 8GB (2*4*1024*1024*1024) = 1,198,614,939

So this looks like an overflow issue with PHP on the Raspberry Pi..?

Any known fix?

Looks like this is indeed (at least part of) the problem. Someone even created a composer package to solve this issue: https://github.com/jkuchar/BigFileTools/blob/master/src/Driver/NativeSeekDriver.php

If the curl extension is available, looks like that is the fastest way to get the real file size of a file above 2GB on 32-bit systems, with a function like this:

function filesize2($path) {
if (function_exists("curl_init")) {
$ch = curl_init("file://" . $path);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
$data = curl_exec($ch);
curl_close($ch);
if ($data !== false && preg_match('/Content-Length: (\d+)/', $data, $matches)) {
return (float) $matches[1];
}
}
return filesize($path);
}

Maybe you could allow users (admins) to add something in their customizables/config.php, to work around the filesize() issue on their system, if they need to..?


Thanks.

Not a bug

If you wish to deal with files larger than 4GB, you would need to use a 64 bit system. There is absolutely no other solution for this. Reading the file size, even if there might be workarounds for it, is not going to solve all the problems you will be hitting due to the 32 bit limitation.

Afaik, there are 64 bit OSes for the Raspberry Pi 3.

Thanks.

Might want to indicate 64-bit OS is required, somewhere in https://filerun.com/requirements