Subject: Re: 'megatools-1.11.0-git-20210505-win64' not working for larger uploads
From: Ondřej Jirman
Date: Mon, 18 Oct 2021 11:56:08 +0200
Hi,

On Thu, Oct 07, 2021 at 04:46:52AM +0000, YruNlsr6371bDg wrote:
> Hello,
> 
> 
> 
> We tried the following using 'megatools-1.11.0-git-20210505-win64' (but we
> presume the same error message will apply to all recent megatools versions
> across all platforms).
> 
> We run 'megaput' with the following command-line arguments to increase
> verbosity/debugging:
> 
> megatools.exe put --ignore-config-file --no-ask-password
> --debug=http,api,fs,cache,tman --proxy=socks5h://127.0.0.1:9050
> --username=************************ --password=***************************
> '************************************************************************************************************************************'
> 
> 
> 
> After many months of successfully using megatools to access/upload to our Mega
> account, since approximately 5 days ago, we are now not able to complete upload
> larger files to our Mega account using megatools due to end-of-upload error:
> "error:1421C0F8:SSL routines:set_client_ciphersuite:unknown cipher returned"
> 
> (We also tried deleting the cache-files/temporary-directory, but that did not
> fix the problem either.)
> 
> This issue might be due to a change in the way Mega servers handle the upload of
> large files - and thus, the HTTP post fails due to the the initial connection
> being too old ("Too old connection (***** seconds), disconnect it"):
> 
> If so, the solution might be to keep the original connection alive - even if it
> is a long upload, so that reconnection is not required - this is probably what
> results in the failed upload, or at least provide a command-line option to keep
> keep the original connection alive such as "--keep-alive" similar to the OpenSSH
> "TCPKeepAlive" option.
> 
> We also looked at the source code from
> 'https://megatools.megous.com/builds/experimental/megatools-1.11.0-git-20210505.tar.gz'
> to try to debug this problem, but this seems to be a curl library error caused
> by disconnecting/reconnecting the "too old" original conneciton - which the Mega
> servers might not be allowing any more.
> 
> 
> The final error message received after reaching 100.0% is uploaded is posted as
> follows (with any private information redacted).
> 
> In all cases, even when we repeatedly try to upload large files, the upload
> correctly progresses from 0.00% to 100.00% - but when reaching 100.00%, it
> returns the following text within quotes:

Mega is likely just not caring for connection to API server. HTTP is stateless
protocol. It's weird that connection fails, but give that this is over tor,
it may not be mega.nz failing but something in between.

Maybe re-trying the API calls in case of this kind of failure may help.

I'd suggest you try to modify the code to add a bunch of retries to the final
API call that finalizes the upload.

kind regards,
	o.

> "
> .
> .
> .
> *********************************************************************************************************************************:
> 100.00% - done *.* GiB (avg. ***.* KiB/s)
> 1 left intact
> * Found bundle for host **********.userstorage.mega.co.nz: 0x***********
> [serially]
> * Can not multiplex, even if we wanted to!
> * Re-using existing connection! (#0) with proxy 127.0.0.1
> * Connected to 127.0.0.1 (127.0.0.1) port 9050 (#0)
> > POST
> /ul/**************************************************************************************/**********?c=****************
> HTTP/1.1
> Host: **********.userstorage.mega.co.nz
> Accept: */*
> Accept-Encoding: deflate, gzip
> User-Agent: Megatools (1.11.0)
> Content-Type: application/octet-stream
> Content-Length: *******
> 
> * We are completely uploaded and fine
> * Mark bundle as not supporting multiuse
> < HTTP/1.1 200 OK
> < Content-Type: text/plain
> < Content-Length: *
> < Access-Control-Allow-Origin: *
> < Access-Control-Allow-Headers: MEGA-Chrome-Antileak
> < Access-Control-Max-Age: 86400
> < Cache-Control: no-store
> < 
> * Connection #0 to host 127.0.0.1 left intact
> * Found bundle for host **********.userstorage.mega.co.nz: 0x***********
> [serially]
> * Can not multiplex, even if we wanted to!
> * Re-using existing connection! (#0) with proxy 127.0.0.1
> * Connected to 127.0.0.1 (127.0.0.1) port 9050 (#0)
> > POST
> /ul/**************************************************************************************/**********?c=****************
> HTTP/1.1
> Host: **********.userstorage.mega.co.nz
> Accept: */*
> Accept-Encoding: deflate, gzip
> User-Agent: Megatools (1.11.0)
> Content-Type: application/octet-stream
> Content-Length: ******
> 
> * We are completely uploaded and fine
> * Mark bundle as not supporting multiuse
> < HTTP/1.1 200 OK
> < Content-Type: text/plain
> < Content-Length: *
> < Access-Control-Allow-Origin: *
> < Access-Control-Allow-Headers: MEGA-Chrome-Antileak
> < Access-Control-Max-Age: 86400
> < Cache-Control: no-store
> < 
> * Connection #0 to host 127.0.0.1 left intact
> * Mark bundle as not supporting multiuse
> < HTTP/1.1 200 OK
> < Content-Type: text/plain
> < Content-Length: *
> < Access-Control-Allow-Origin: *
> < Access-Control-Allow-Headers: MEGA-Chrome-Antileak
> < Access-Control-Max-Age: 86400
> < Cache-Control: no-store
> < 
> * Connection #0 to host 127.0.0.1 left intact
> * Mark bundle as not supporting multiuse
> < HTTP/1.1 200 OK
> < Content-Type: text/plain
> < Content-Length: *
> < Access-Control-Allow-Origin: *
> < Access-Control-Allow-Headers: MEGA-Chrome-Antileak
> < Access-Control-Max-Age: 86400
> < Cache-Control: no-store
> < 
> * Connection #0 to host 127.0.0.1 left intact
> * Mark bundle as not supporting multiuse
> < HTTP/1.1 200 OK
> < Content-Type: text/plain
> < Content-Length: *
> < Access-Control-Allow-Origin: *
> < Access-Control-Allow-Headers: MEGA-Chrome-Antileak
> < Access-Control-Max-Age: 86400
> < Cache-Control: no-store
> < 
> * Connection #0 to host 127.0.0.1 left intact
> * Mark bundle as not supporting multiuse
> < HTTP/1.1 200 OK
> < Content-Length: **
> < X-Content-Length: **********
> < Content-Type: application/octet-stream
> < Access-Control-Allow-Origin: *
> < Access-Control-Allow-Headers: MEGA-Chrome-Antileak
> < Access-Control-Max-Age: 86400
> < Cache-Control: no-store
> < 
> * Connection #0 to host 127.0.0.1 left intact
> -> [{
> "a": "p",
> "t": "********",
> "n": [{
> "h": "************************************************",
> "t": 0,
> "k": "*******************************************",
> "a":
> "************************************************************************************************************************************************************************************************",
> "fa": null
> }]
> }]
> * Too old connection (***** seconds), disconnect it
> * Connection 0 seems to be dead!
> * Closing connection 0
> * Trying 127.0.0.1:9050...
> * SOCKS5 connect to g.api.mega.co.nz:443 (remotely resolved)
> * SOCKS5 request granted.
> * Connected to 127.0.0.1 (127.0.0.1) port 9050 (#1)
> * ALPN, offering h2
> * ALPN, offering http/1.1
> * SSL re-using session ID
> * error:1421C0F8:SSL routines:set_client_ciphersuite:unknown cipher returned
> * Closing connection 1
> ERROR: Upload failed for
> '************************************************************************************************************************************':
> API call 'p' failed: HTTP POST failed: CURL error: SSL connect error
> SAVE CACHE: {
> "version": 4,
> "last_refresh": **********,
> .
> .
> .
> "