Thanks a lot for working on this.

What about a fixed chunk size? TransferJob seems to recommend 1MB at most:

/**
 * Request for data.
 * Please note, that you shouldn't put too large chunks
 * of data in it as this requires copies within the frame
 * work, so you should rather split the data you want
 * to pass here in reasonable chunks (about 1MB maximum)
 *
 * @param job the job that emitted this signal
 * @param data buffer to fill with data to send to the
 * slave. An empty buffer indicates end of data. (EOD)
 */
void dataReq(KIO::Job *job, QByteArray &data);

We definitely need to improve the documentation of this stuff. Adding @dfaure for more insights

The bigger, the better (=faster).
The 0xFFFFFF limit in KIO itself is due to the protocol only having 24 bits for size AFAIK - KLocalSocket can handle more than that at once without copying.
I'm not sure where TransferJob is used actually. It worked fine with more than 14MiB as well.

The only reason to use a smaller chunk size is to work better over TCP, but who uses KIO over TCP nowadays?

Yes to chunking.

Yes to changing the documentation, the 1MB must be very old in there, if 14MB works then let's say so.

As to the implementation, would it be possible to use QByteArray::fromRawData to avoid copying? After all it's all "synchronous" here, i.e. the lifetime of the chunk will always be less than the lifetime of the overall data.

How is this looking now that D15426 has gone in?

! In D15448#330352, @ngraham wrote:
How is this looking now that D15426 has gone in?

Those diffs are fully independant - this fixes the bug that larger file transfers (over 16MiB) simply failed at the end while D15426 improves performance of all transfers using the KIO AccessManager massively by removing the 100% CPU bottleneck.