Win10PESE cannot build in no internet/connect to internet with proxy environment

I download WIN10PESE 300MB+ archive file, and copy it to USB, bring it to another PC.
When I try to build WIN10PESE, I failed, because the GWT.exe will download some files(dism, oscdimg, ...), but cannot finish.

as the 300MB+ WIN10PESE archive file already have wimlib-image.exe and some other tools, and the OS have dism command,
it is not nessesary, and packaged the lastest version in the archive file is not bad.

at least somebody already download such big size file, it should lead me to build one.

make a proxy settings and download files, or include the nessesary files in archive file.
detect if network is OK, download the most befitting tools, otherwise use the included tools.


======================================================================================
ANOTHER problem after: I copy Tools\x86,x64 files, to build, failed in copy the catroot\{F750E6C3-38EE-11D1-85E5-00C04FC295EE}, I can see InstallWimSrc mounted,
I don't know why halt here.
======================================================================================
     Run - Processing section: [AddCatalog] From file: [%BaseDir%\Projects\Win10PESE\Build\1-files.Script]
                   [Info] [// Add all catalog for drivers installation, WOWSupport... or standard Catalog .... ex: History143=JFX -> TCP/IP Registry Compatibility http://TheOven.org/index.php?topic=2335]
                   [Ignore] IF - [true] is not equal to: [false] evaluated string: [If,%AllCat_CheckBox%,Equal,False,Echo,"AllCat_CheckBox=False (Advanced Users only).",Warn]
                         [Success] IF - [true] is equal to: [true] evaluated string: [If,%AllCat_CheckBox%,Equal,True,Begin]
                         [Info] Add all (Install wim) catalogue...
                               [Success] IF - [2] is equal to: [2] evaluated string: [If,%api_type%,Equal,2,Begin]
                              [Failed] DirCopy - Failed to copy directory [%BaseDir%\Mount\Win10PESE\Source\InstallWimSrc\Windows\System32\catroot\{F750E6C3-38EE-11D1-85E5-00C04FC295EE}] to: [%BaseDir%\Target\Win10PESE\Windows\System32\catroot]: Unknown error
                              [Warning] Halt

Hi slore,

for proxy, I have some ideas to be tested by you.

I guess this one will be quick and easy (but not perfect..... I will ask some questions on later posts.... ):
extract following
https://www.dropbox.com/s/keuhcbgj863gtxn/Proxy%20Settings%20-%20WinHTTP_201812060134.7z?dl=0
to
H:\For.Windows\Win10PE_SE_2018-11-12\Projects\Win10PESE\Utils\

on builder screen at Utils You will notice new plugin
Proxy Settings - WinHTTP

Test1:
First test with "Test Proxy"
I guess "Test Proxy" button will fail on your side ??
( I do not know how to test, help me to improve better)

Test2:
Click "Import Proxy Settings From Internet Explorer"
If "Test Proxy" button failure, I hope now using "Test Proxy" success ??


ps:
You will notice information and other buttons on plugin interface
There are links and info about Netsh WinHttp available inside plugin.

**
If using netsh winhttp (dirty fix) is working on your side,
Test3:
Utils\GetWaik Tools
-> Launch
---> Select Waik Tools for Win7 (for test)
-----> Download
does GetWaik Tools downloading now ?

Summary:
Test1 Test2 Test3 ?

more ideas later.....



****
Extract your zip file to a shorter path
H:\For.Windows\Win10PE_SE_2018-11-12
-->
H:\Oven

Maybe your log error is only about having very long (36) base folder path

Hi, Lancelot

Extract your zip file to a shorter path
Maybe your log error is only about having very long (36) base folder path

H:\For.Windows\Win10PE_SE_2018-11-12
-->
H:\Win10PE

finish the building...(but too slow...)

Code: [Select]

    Scripts

 Config Source Version[30] ( )  0.788 seconds
 Main Configuration Version[67] ( )  9.274 seconds
 Images Configuration Version[66] ( )  5.748 seconds
 Retrieve Tools Version[32] ( )  86.400 seconds
 0 - PreConfig Version[73] ( )  187.598 seconds
 Chief PluginCache Version[74] ( )  18.082 seconds
 1 - Copy Files Version[151] ( )  201.756 seconds
 2 - Shell & Config Version[100] ( )  239.364 seconds
 6 - Common Files Version[9] ( )  4.319 seconds
 9 - Autorun Version[9] ( )  0.180 seconds
 CdDrive - X: - Y: Version[26] ( )  0.643 seconds
 Retrieve More Localized Info Version[25] ( )  2.032 seconds
 Themes Version[39] ( )  14.324 seconds
 Project Medicine Version[30] ( )  2.921 seconds
 0 - Shell Loader Version[31] ( )  2.199 seconds
 1 - Explorer Shell Version[60] ( )  61.474 seconds
 Basic PluginCache Final Version[16] ( )  92.133 seconds
 CMD Adds Version[25] ( )  20.458 seconds
 HwPnP Version[36] ( )  1.877 seconds
 MMC Version[45] ( )  8.544 seconds
 Ms Paint Version[23] ( )  3.154 seconds
 Notepad Version[49] ( )  3.352 seconds
 PENetwork Version[55] ( )  3.474 seconds
 Components PluginCache Final Version[10] ( )  103.557 seconds
 Optimizations Version[24] ( )  6.572 seconds
 1 - Pins Version[16] ( )  3.414 seconds
 StartMenu Local Apps Version[17] ( )  0.172 seconds
 PostConfig Version[65] ( )  234.902 seconds
 Create ISO Version[83] ( )  22.601 seconds

 29 scripts processed.
 Project time: 22 minutes, 21 seconds and 688 milliseconds

about the proxy test, I cannot download from dropbox ...

if it is not big, upload as attachment, I will try it next week.

finish the building...(but too slow...)

2 reasons:

1) Waiting Bob.Omb update .net plugin. 
2) you mount iso and build
on Main Plugin
a) Windows10 files in a Folder (Recommended)

you use b) and this cause slow build.

when you follow a) path and copy your source to a folder, source will be extracted once
 (I had noticed WimBuilderUI already support extracted folders. ) you will build faster on 2nd and later builds.

Hi slore,

There must be something very wrong !
See Reply 7 and 8 here:
http://theoven.org/index.php?topic=1930.msg29185#msg29185
Bob.Omb:
1st Build: 13 minutes 2 seconds (Extraction of WIM for future builds)
2nd Build: 2 minutes 17 seconds

***

about the proxy test, I cannot download from dropbox ...

I will work on proxy today and provide a better proxy test today or tomorrow.
(again limited free time goes away from testing WimBuilderUI  )

I wonder why you can not download from dropbox ?
reason proxy ? -> If yes why ?
and if dropbox prevented by a proxy, is it same with googledrive etc. ?

Hi slore,

+
I guess download buttons on plugins do not work behind proxy too ?

eg.
Win10PESE\Build\Macro Library -> green small down arrow button with tip "DOWNLOAD NEW Plugin"
(see warnings on log if failure....)

>a) Windows10 files in a Folder (Recommended)

I know if I use extracted install folder will be faster.

It is good for some body use little versions Windows media to create PE.

I have to test (zh-CN, en-US) * (x86, x64) * (17134.*, 17763.*, 18*), extract them kill my disk and time for:
large GB size; small file fragments; delete,/reexctract;
think about the WinSXS in install.wim, harklink to same file, but exctract in N GB, small files...
delete/remove them also take looong time(waste time and hard disk life).

map my part of 16GB memory to CacheDisk, put wimbuilder on it is good for me.
Actrually I consider make a cache about extracted needs files, but in this way make PE in faster 40 seconds already,
not major improvement to take time on it now.
(people can wait 13 / 2 minutes for building, and wimbuilderUI is also support extracted folder.)

I will work on proxy today and provide a better proxy test today or tomorrow.
(again limited free time goes away from testing WimBuilderUI  )

Take your time, WimBuilderUI's update is not major things, no big different from the tested version.
More improve will arrive (end of Dec):
* detect wim index info(description/version/arch/lang) and show in start page, build page
* _ISO_ bootable check, show warning ,and if non-bootable template exists, will recreate again if Windows Media mounted drive select
* VMWare Player test
* Advance Test page

also need time to create/update New Patch:
* StartIsBack settings with direct preview UI for what you select.(flat menu on WIN10XPE, I don't what is mean before I create a test PE to see it)
* PECMD loader(simple one is done, but the UI settings is missing)
* PENetwork(simple one is done, but the UI settings is missing)

I wonder why you can not download from dropbox ?
reason proxy ? -> If yes why ?
and if dropbox prevented by a proxy, is it same with googledrive etc. ?

There is a "Greate" Firewall in China 

I guess download buttons on plugins do not work behind proxy too ?

I will test it on Monday and feedback.

I have to test (zh-CN, en-US) * (x86, x64) * (17134.*, 17763.*, 18*), extract them kill my disk and time for:
large GB size; small file fragments; delete,/reexctract;
think about the WinSXS in install.wim, harklink to same file, but exctract in N GB, small files...
delete/remove them also take looong time(waste time and hard disk life).

btw,
"Extracted WimFolders" use pointer... to gain space (thanks to JFX ChrisR whoever added that)
so
2,73 GB extracted occupy 1.30 GB here (Win10_1809_English_x64.iso)
12,7 GB extracted occupy 5,94 GB ( Win10_SingleLang_EnglishInternational_x64.iso)
Numbers from file properties of folder (install_wim)

(I guess numbers were ~300MB in the past ;) .... )

Well I can only have time to test with Win10_1809_English_x64
(single mui, fastest build  )
But for detailed tests I also do similar things like you .....

***
When Bob.Omb update .Net plugin, It will be faster. 

There is a "Greate" Firewall in China 

I guess this means all dropbox, googledrive are blocked
(I could not find "list of blocked" by "Greate" Firewall in China)

ps: our "Greate" firewall limits wikipedia, I use wikizero links for friends. 

*
Opera Browser is using vpn out of the box which works fine here..... ?

It is only very quick to put file and share with dropbox and googledrive...

*
Tests will be ready till Monday. (at most tomorrow morning) 

There is a "Greate" Firewall in China 

It's the great wall of China for a free web

I can access https://mega.nz in 2016.
but can't now.
I set the IP to drivers/etc/hosts, now I can acess it without "SMART" DNS.

you can put on it when you done.(I don't test if I can download file, just for a try)

Hi slore,

I hope you can download this ?

http://lancelot.cwcodes.net/pub/1/20181209_1305_slore_Projects.7z

Extract to your base folder (H:\Win10PE)

*
1)
First Delete some folders from your projectemp folder
H:\Win10PE\Temp\Win10PESE\ML_Call_Download\
H:\Win10PE\Temp\Win10PESE\WGet_M\
2)
open builder
you will notice:
Win10PESE\Utils\Proxy Settings
Add your Proxy server (with username and password if exists) to "Proxy:" box.
Be sure "Enable Proxy" checkbox enabled
Click "Save Proxy Settings (Apply)" Button
3)
Test:
Utils\"Mount" plugin
There is green down arrow on plugin to download updated plugin from server.
Click this button
later click "Log" button at top to see log --> If there is no error you can download from server behind proxy now.


I hope all go well and you can download.

***
If this works,
all download buttons on plugins will work
eg.
Apps\Network\Mozilla Firefox ESR (P)
Download button will work following Language and updated ESR file on mozilla server.

After success on this, there will be another proxy plugin to download with GetWaikTools......

Hi, Lancelot

I hope you can download this ?
http://lancelot.cwcodes.net/pub/1/20181209_1305_slore_Projects.7z

Yes, I can download from here.(only 3.7MB attachment also be OK.)

I'm no in proxy environment now, so I test with fake proxy(default settings):

Utils\"Mount" plugin update failed with fake proxy(expected)
Utils\"Mount" plugin update successfully without proxy settings(expected)

Apps\Network\Mozilla Firefox ESR (P) download failed with/without proxy(unexpected)
also it has own proxy settings, use system proxy or no proxy fail to down.(unexpected)

Apps\Network\Flash Add download  failed with fake proxy(expected)
Apps\Network\Flash Add download successfully without proxy settings(expected)

I will test on real proxy environment 15 hour later.

Hi slore,

Apps\Network\Mozilla Firefox ESR (P) download failed with/without proxy(unexpected)
also it has own proxy settings, use system proxy or no proxy fail to down.(unexpected)

Apps\Network\Mozilla Firefox ESR (P)
--> "Download" button on plugin (at right of text "Provide File x64" , right middle of plugin)
 here (without proxy) downloads Firefox Setup 60.3.0esr.exe from mozilla server.

you can copy weblink used from log and test with other things (webbrowser etc.) too...


*
I just update Utils\"Proxy Settings" plugin (v3), use small green download button on top-right of "Proxy Settings" plugin
(or use weblinks available inside plugin Web1=.. Web2=... both same file.)

To make settings simple for new users,
I add 'Import Proxy Settings From Internet Explorer' button and add the following to "Blue Information" button

If you are using proxy with IE do following:
1) Click 'Import Proxy Settings From Internet Explorer'
2) Enable checkbox with name 'Enable Proxy'
3) click 'Save Proxy Settings (Apply)' Button

***
I wish my best luck on real proxy environment.
I never use proxy to connect to internet
(well only to connect university library database in very past ...)
I create plugins following information on web without any real test.
(tested a lot of things, but no real tests)
Rest depends on your feedback. 

Hi slore,

Is there any progress using proxy with "Greate" Firewall in China  ?

Hi, Lancelot

Sorry for no reply.
The road near my work place caused a fire occurred, the network electric cable is damaged.
I can't test it for somedays.

I see you modify the wget script? It should be OK, because I have 2 batch command:

wget.cmd
wget.exe %*

wwwget.cmd
set http_proxy=http://xx.xx.x.x:8080
wget.exe %*

When I download something from LAN server, I type wget http://internal_file_server/xxx/xxx.
I need something from the Internet, I type wwwget http://xxxxx/xx.zip

They works fine for years.

Apps\Network\Mozilla Firefox ESR (P) download failed with/without proxy(unexpected)

This is my mistake. I forgot to upate the plugins.
the origin "green download button" is update the plugin, and pointer to http://win7pese.codes....../Firefox_File.script,
and this is not exist on server.
After updated, I found out the "download ESR file on mozilla server." button, I can download the Firefox Setup 60.4.0esr.exe from the server, it isnot block by the GFW.

When the network is back, I will test it in real proxy environment.


well, I see it use aria2c.exe first.

Hi Slore,

Thanks for the update.

+
It is good to hear you figure out things going on with Mozilla Firefox ESR (P)
 

+
yes ChrisR like aria2 for some reasons with wget failure etc.
see posts here:
http://TheOven.org/index.php?topic=2488.msg27890#msg27890

ChrisR informed to change default downloader from wget to aria2,
I decide to make adjustment:
First aria2
Fallback wget

***
Following info about your usage,
 I feel proxy will work out of box,
  and It will be easier for you to update plugins with download button behind proxy. (without download all zip file)
----> This way It will be easier for you to test other proxy or no proxy related things when you are behind firewall.

Still 3 (maybe more) other proxy tests waiting......

  and It will be easier for you to update plugins with download button behind proxy. (without download all zip file)
----> This way It will be easier for you to test other proxy or no proxy related things when you are behind firewall.

well, I like ALL in ONE zip achive first(major release version), update function for small update(minor version).

because I can carry the  ZIP file to some environment EVEN no internet, and I can use tool(P2P) to download ZIP achive, it will be faster.
(I updated PESE plugins with 1 hour. download the ZIP toke 20 minutes.)

Providing different options fits all cases. 
eg.
If you had network plugin error, I would ask you to download new version with dl button which would take some seconds...
Or If there is a mistake on a plugin, I would fix and upload, you will quickly download and test (hopefully behind proxy without attachment or dropbox etc. and without goto folder to extract etc....)

Updating should be less time.... I will ask questions about this later....

For now dl buttons work behind proxy
+ will get you quickly get an updated plugin to test when required.
+ will get you to download updated (or language special) applications like FireFox ESR

move to http://theoven.org/index.php?topic=2669

move to http://theoven.org/index.php?topic=2669

move to http://theoven.org/index.php?topic=2669