WGet / Aria2c

Is it me or these tools have more and more difficulties to negotiate SSL connections.
I have 2 software that started to have problem this week. An SSL/TLS handshake failure occurs on download attempts.

Code: [Select]

Aria2c error #1: The message received was unexpected or badly formatted.

or

Code: [Select]

Aria2c error #22: The response status is not successful.

Is there a way to correct this. At the moment I tried Aria2c version 1.35 and the results are the same.
I tried with WGet and got... on both downloads. Tried a couple things with Aria's options... but, no go.

Code: [Select]

Wget: Unable to negotiate SSL connection.

I confirm both where working last weed and if I go to download them with a browser both works.
This is one download that gives the error.

Code: [Select]

https://download.nirsoft.net/nirsoft_package_enc_1.23.10.zip

Any ideas what going on ?

using just the download link provided - it results in an error with IE

Error 403: Missing HTTP referer in the HTTP request.

If accessing the download via the Hosts webpage - then download fine

It is also a password protected compressed package

The launcher is x86 - but the Applications are divided by arch..
So process would be best > if %app% selected then copyfile(s) & create shortcut

Although it would take time to create 232 App options w/shortcuts...
So would be a matter of only finding what would be useful in WinPE...

Also think you going to find a lot of system file dependencies'

Thanks James,

Again you got me on the right track in a minute...I didn't thought to simply paste the link in another browser.

Is it possible to add options to the download command ? It works when adding "--referer=*"

Code: [Select]

aria2c --referer=*  https://download.nirsoft.net/nirsoft_package_enc_1.23.10.zip

This fixes one of the download problem I'll try to gather more info on the other one.

Yes it is possible via Macro - But we must consider the impact it may have with all other downloads

Or you could run this one off type download manually from within the XPE App
ShellExecute,Hide,%GTools%\aria2c.exe,"your parameters"

Hi Malok,

Is it possible to add options to the download command ? It works when adding "--referer=*"

Code: [Select]

aria2c --referer=*  https://download.nirsoft.net/nirsoft_package_enc_1.23.10.zip

This fixes one of the download problem I'll try to gather more info on the other one.

Thanks Malok,

I updated Macro Library, following your feedback, now you can use Call,Download with your link 

Thanks Malok,

I updated Macro Library, following your feedback, now you can use Call,Download with your link 

You added --Referer=* to macro ? Or We can pass options to Aria2c trough Download command now ? 

The second one giving trouble is Wireshark.

Code: [Select]

2020-01-17 16:28:52.320421 [NOTICE] [Context.cc:311] Downloading 1 item(s)
2020-01-17 16:28:52.445221 [ERROR] [AbstractCommand.cc:351] CUID#7 - Download aborted. URI=https://1.na.dl.wireshark.org/win64/Wireshark-win64-3.2.1.exe
Exception: [AbstractCommand.cc:351] errorCode=1 URI=https://1.na.dl.wireshark.org/win64/Wireshark-win64-3.2.1.exe
  -> [SocketCore.cc:1021] errorCode=1 SSL/TLS handshake failure: Error: The message received was unexpected or badly formatted.
(80090326)
2020-01-17 16:28:52.445221 [NOTICE] [RequestGroupMan.cc:427] Download GID#e6e570c5256b5173 not complete: C:/Win10XPE/ProgCache/Wireshark/Wireshark-win64-3.2.1.exe

Malok, I downloaded via codebox five times and it error once for me

Code: [Select]

2020-01-17 17:05:49.400203 [NOTICE] [Context.cc:311] Downloading 1 item(s)
2020-01-17 17:05:49.759587 [NOTICE] [SingleFileAllocationIterator.cc:77] Allocating disk space. Use --file-allocation=none to disable it. See --file-allocation option in man page for more details.
2020-01-17 17:05:49.978335 [ERROR] [AbstractCommand.cc:351] CUID#9 - Download aborted. URI=https://1.na.dl.wireshark.org/win64/Wireshark-win64-3.2.1.exe
Exception: [AbstractCommand.cc:351] errorCode=1 URI=https://1.na.dl.wireshark.org/win64/Wireshark-win64-3.2.1.exe
  -> [SocketCore.cc:1015] errorCode=1 SSL/TLS handshake failure: Error: The buffers supplied to a function was too small.
(80090321)
2020-01-17 17:05:54.697191 [NOTICE] [RequestGroup.cc:1216] Download complete: H:/Win10XPE/ProgCache/WireShark/Wireshark-win64-3.2.1.exe
2020-01-17 17:05:54.931504 [NOTICE] [Context.cc:311] Downloading 1 item(s)
2020-01-17 17:05:55.228420 [NOTICE] [SingleFileAllocationIterator.cc:77] Allocating disk space. Use --file-allocation=none to disable it. See --file-allocation option in man page for more details.
2020-01-17 17:05:59.884741 [NOTICE] [RequestGroup.cc:1216] Download complete: H:/Win10XPE/ProgCache/WireShark/Wireshark-win32-3.2.1.exe
//
2020-01-17 17:08:04.817365 [NOTICE] [Context.cc:311] Downloading 1 item(s)
2020-01-17 17:08:05.161140 [NOTICE] [SingleFileAllocationIterator.cc:77] Allocating disk space. Use --file-allocation=none to disable it. See --file-allocation option in man page for more details.
2020-01-17 17:08:10.255058 [NOTICE] [RequestGroup.cc:1216] Download complete: H:/Win10XPE/ProgCache/WireShark/Wireshark-win64-3.2.1.exe
2020-01-17 17:08:10.520659 [NOTICE] [Context.cc:311] Downloading 1 item(s)
2020-01-17 17:08:10.833186 [NOTICE] [SingleFileAllocationIterator.cc:77] Allocating disk space. Use --file-allocation=none to disable it. See --file-allocation option in man page for more details.
2020-01-17 17:08:15.833237 [NOTICE] [RequestGroup.cc:1216] Download complete: H:/Win10XPE/ProgCache/WireShark/Wireshark-win32-3.2.1.exe
//
2020-01-17 17:09:28.186251 [NOTICE] [Context.cc:311] Downloading 1 item(s)
2020-01-17 17:09:28.514373 [NOTICE] [SingleFileAllocationIterator.cc:77] Allocating disk space. Use --file-allocation=none to disable it. See --file-allocation option in man page for more details.
2020-01-17 17:09:33.748804 [NOTICE] [RequestGroup.cc:1216] Download complete: H:/Win10XPE/ProgCache/WireShark/Wireshark-win64-3.2.1.exe
2020-01-17 17:09:33.998793 [NOTICE] [Context.cc:311] Downloading 1 item(s)
2020-01-17 17:09:34.295734 [NOTICE] [SingleFileAllocationIterator.cc:77] Allocating disk space. Use --file-allocation=none to disable it. See --file-allocation option in man page for more details.
2020-01-17 17:09:38.967619 [NOTICE] [RequestGroup.cc:1216] Download complete: H:/Win10XPE/ProgCache/WireShark/Wireshark-win32-3.2.1.exe

Thanks Malok,

I updated Macro Library, following your feedback, now you can use Call,Download with your link 

You added --Referer=* to macro ? Or We can pass options to Aria2c trough Download command now ? 

The second one giving trouble is Wireshark.

All automatic and works fine now. 

Call,Download,%ProjectTemp%\1\,https://download.nirsoft.net/nirsoft_package_enc_1.23.10.zip
Call,Download,%ProjectTemp%\1\,https://1.na.dl.wireshark.org/win64/Wireshark-win64-3.2.1.exe

Simple 

@James,

For me both at home and work it fails every time since last week.

@Lancelot,

mmmmmmaybe.... What did you modify ??? Can this be included easily in other projects ?

 

@Lancelot,

mmmmmmaybe.... What did you modify ???

I add your line + I guess 3 more lines to check related things.... No secret, you can easily follow log. 

Test and see how it goes ?

Can this be included easily in other projects ?

*
Gena, Win7PESE, Win8PESE, Win8.1SE, Win10PESE already updated 
Reminding Call,Download have size check support since syntax created
http://theoven.org/index.php?topic=45.0
eg.
Call,Download,%ProjectTemp%\1\,https://download.nirsoft.net/nirsoft_package_enc_1.23.10.zip,1048576
--> if download not 1 MB download failure

such size check is good protection for special download cases like yours. 

*
LiveSystemPro (4 Projects) do not use wget or aria, Kare embed all functions inside LiveSystemPro.exe so you should test and ask on its section http://theoven.org/index.php?board=23.0
*
WimBuilder also do not use wget or aria, last time I remember slore was looking for a way to use internal windows functions without 3rd party
eg. see bitsadmin https://stackoverflow.com/questions/4619088/windows-batch-file-file-download-from-a-url/30093723
Still If you create a "WimBuilder patch" with such link better ask slore for support http://theoven.org/index.php?topic=2390
*
Win10XPE I guess you should easily use aria2 inside tools folder following design principle with name "simple", simply use aria from tools, well only guess.


**
Better to mention, I use Plugins with LiveSystemPro and WimBuilder so It is to me available there too. 


**
Btw, see lines at Win10PESE\WriteMedia\"USB Creation Tool" Plugin which is also compatible to XPEPlugin 

Code: [Select]

[Process]
...
IniRead,%ProjectDir%\script.project,Main,Type,%t%
If,ExistSection,%ScriptFile%,Launch_%t%,Run,%ScriptFile%,Launch_%t%

This will get you use Call,Download on Plugin sections 

Thanks for clarifying this for me.

I think I will Use the shellExecute method because I want to leave My XPE the most compatible with upgrades... To much hacking always breaks projects. 

To much hacking always breaks projects. 

That is the reason Macro Library created    to avoid such cases.

You get simpler Plugin writing, Macro Library gets more complex and bigger, same like PowerShell (v6 = 140MB , v7 = 225 MB) or WimBuilder Macro,
Simpler Macro gets more complex Plugin writing ...

And anyone including Post Gamers can find a clever stupid reasons to complain about the complexity of the size of Macro and try to affect the public view, sometimes they succeed.

One way to suffer such Post Gamers is embed everything inside .exe , .dll (eg. Kare's LiveSystemPro.exe), This at least mostly protects from Post Gamers to take codes and put own name on them.
Still some other gamers copy design 


Shortly, It is the same old story and old Post Gamers since I start to use PC.

Like you, I always follow the design of the development to suffer from breaking.  and use my own solutions to avoid breaks. 
The previous post are things I know about projects related to this topic subject which I hope help you.