Win10PESE cannot build in no internet/connect to internet with proxy environment

 

just a tip:
extract latest zip you have to your
E:\PESE\Workbench\Common\Download_Win10PESE\

this way blexactserver will only download files after zip.... faster...

eg.
AgentRansack File should not be downloaded with latest zip you have...
regular user do not have such download since 2017.08.24

ps:
I guess you extract zip and try this option,
 but this way it fails (download all from server instead of update) since on your first build files copied to \Workbench\Common\Download_Win10PESE\

with other words:
\Workbench\Common\Download_Win10PESE\
contains exact files with server and updated securely with blexact ....

for now, I copy my Projects manualy -> to the E:\PESE\Workbench\Common\Download_Win10PESE\.
It skipped the same ones.(I think it check with the MD5)

but I start from WIN10PESE\BuilderSE.exe, so it should be able to detect MY current Projects\*.scripts,
so Only update what is new, or mine is old one without COPYING.

something bad in script:
1. I do some option changed, or check/uncheck the plugin, it will write to script.
so SAME version script will be different, and download again.

2. Embbed Files in script. the script is updated, but the Embbed Files actually is same one,
but the MD5 will different and download again.
the biggest part of plugins is the Embbed things, if there is a way to avoid download them is better.

===============================================================
blexactserver seems only start/stop for downloading, I can't skip some of the plugins.
like the old firefox_file is enough for me now, I want to skip this one to next, but there is no option.

better to have update list show what needs update, and the size before updating.

Because the Embbed Files, I can't update with cache or P2P. so it is slow for me, and I can't choose another
download tools for updating. in my case <20kb/s. token 2 hours but unfinish...
if I can download Firefox from homepage, it could be very faster.

I would like small scripts update with this plugin, but a bigger size needs download,
on my side, ZIP will be faster, and simple.



===================================================
origin reply about SwRepo(simple apps updating idea for project) is achived in PDF.

Forgot to feedback the proxy pulgin.

proxy ping test is not exact. because the GFW, I can't ping or access google.com.
even some website I can access very fast, but the ping got timeout.

wget the http://wwww.xxx.com for the HTML content is better than ping.
and about the test URL or download file url, if there is a inputbox for changing them will be better.
I can change it to the suitable web site url in my envrionment.

Hi slore,

You miss to respond important question, does blexactserver works behind proxy ?

See you on next post...

It skipped the same ones.(I think it check with the MD5)

Yes MD5

but I start from WIN10PESE\BuilderSE.exe, so it should be able to detect MY current Projects\*.scripts,
so Only update what is new, or mine is old one without COPYING.

manual download ('Start Download') only downloads to Workbench\Common\Download_Win10PESE\
regular user use default interface (Basic Menu) and use "Update (Download + Copy)" button.

something bad in script:
1. I do some option changed, or check/uncheck the plugin, it will write to script.
so SAME version script will be different, and download again.

"Update (Download + Copy)" button restores plugin settings. There is "Preserve Plugin Settings" checkbox enabled.

2. Embbed Files in script. the script is updated, but the Embbed Files actually is same one,
but the MD5 will different and download again.
the biggest part of plugins is the Embbed things, if there is a way to avoid download them is better.

We already avoid by using container plugins like "AgentShack File" for plugins bigger than 10 MB

===============================================================
blexactserver seems only start/stop for downloading, I can't skip some of the plugins.
like the old firefox_file is enough for me now, I want to skip this one to next, but there is no option.

It is intentional, blexactserver idea is to get exact copy of server.

better to have update list show what needs update, and the size before updating.

good idea, but requires a separate app nobody spend time,
 the apps I knew 1) do not save an exact copy to a separate folder which cause requirement redownload same file again when file changes.
 (they do not save a copy to Workbench\Common\Download_Win10PESE\)
                         2) trouble do download all from server, they skip file with failure and cause failure during build.

Because the Embbed Files, I can't update with cache or P2P. so it is slow for me, and I can't choose another
download tools for updating. in my case <20kb/s. token 2 hours but unfinish...
if I can download Firefox from homepage, it could be very faster.

You can download Firefox from homepage.
But I can not understand trouble or difference to download from current server and firefox server.

I would like small scripts update with this plugin, but a bigger size needs download,
on my side, ZIP will be faster, and simple.

But such plugins fails to work out of box, requires download at first build.
 There are plugins for such cases where they are unselected by default.
  eg. Apps\HD Tasks\Imaging\HDClone   --> requires download at first build + unselected by default.
  It would good for WimBuilderUI to keep small size, unselected default.


got tired, coffee time... I will continue on next post. 

============================================================
for the appliaction updating,
I'm working on a SwRepo(software repoistory) project to maintenance the APPs for PE or others.
It will download apps from homepage, then fallback github, or http://win10se.cwcodes.net/projectindex.php somewhere defined in info.js.
if software need be updated, need not update patch main.bat or plugin script, it will auto update.

I do not clearly understand this, what I understand:
> download apps from homepage
some apps links changes, so requires first plugin updated.
some apps links do not change (FireFox - TeamViewer) on such plugins we already provide an option to update from apps homepage.
ps:
FireFox, TeamViewer, HDClone, ...plugins also have FileContainer plugins optional
 1) to get them work out of box
     ----> good for new user and good for first build of project.
 2) To provide a working version copy of app with plugin
     ----> sometimes app update changes requirement or other things that makes plugin fails to add to project. life goes....
     (eg. Chrome plugin on server have provide file option but fails with latest, when used with File container it works with old version plugin initially developed)
 ps) Sometimes app already very static, no version change...

FireFox plugin on SE case is a special case, to include with the project and enabled default, we have to add filecontainer.
ps:
Just an idea to share:
CD-RW on LiveSystemPro use QTWeb as default browser to keep size small which maybe a good idea for WimbuilderUI. 
I mean: WimbuilderUI -> QTWeb default enabled, FireFox optional not enabled default..

script always be:

Code: [Select]

call fetch PENetwork /ver:latest /arch:x64 /lang:%LANG%

.....

4) Keep in mind, Sometimes latest version is not good. A recent example BootICE.
BootICE latest have bug with command line, which is not good for other plugins that use BootICE command....
and development of BootICE seems ended after that point.
In past BootICE plugin was updated, when reported failure with command we reverse the update, but sadly BootICE never get a new update....

Holger updates PENetwork very frequently, still in the past he can not find time to update latest, so we revert back to previous version...

I wait some time after PENetwork updated, follow feedbacks and If everything fine I update plugin. 
PENetwork is a critical update that effects all user experience with network realated things.

*
5) Generally 1 Main Plugin + 1 FileContainer preferred, If size too big 1 Main Plugin + 2 FileContainer (x86 + x64) to keep things simple.
On some special cases more FileContainers can be used to help end user update small.

6) people mostly prefer all in one if app file is small. (<10MB) eg. BootICE

ps:
10 MB value is theoretical, I had to put a limit number in very past and I felt 10 MB is a good limit after some tests.

and SwRepo's PENetwork\info.is have the URL to =>
https://www.penetworkmanager.de/scripts/PENetwork_x64.7z
(optional MD5 check)

better not have optional MD5 check, you will need to update plugin for changed MD5.

I use a different method for MD5 check:
to demonstrate:

Code: [Select]

[Process]
If,ExistFile,%api%,AddVariables,%api%,ApiVar,GLOBAL
//--
Call,DownloadX,"%BaseDir%\Projects\Yomi\AppY\Network\Remote Connect\TeamViewer.Script"

http://theoven.org/index.php?topic=2672.0

Download button checks existing TeamViewerPortable.zip md5 and download from server.
If md5 changes, give Message info about new version. If not give Message info not changed.
no info inside plugin about md5.  no need to update plugin with updated md5 of latest.

***
I believe above mostly summarize why we use FileContainers or embed small apps (<10MB) to plugins with combination of Download buttons on plugins which effects update from server ideas....
~10 years experience came to this point to far. (+ some more ideas that requires time but not used for now)
I hope you benefit from them to design things to better on your development. 

.........

Forgot to feedback the proxy pulgin.

proxy ping test is not exact. because the GFW, I can't ping or access google.com.
even some website I can access very fast, but the ping got timeout.

"Test Internet - Ping" button goal is to test "WinHTTP"
Let me know something other than google without wget and aria2 if possible.
 (wget and aria2 is not good to test WinHTTP)

wget the http://wwww.xxx.com for the HTML content is better than ping.
and about the test URL or download file url, if there is a inputbox for changing them will be better.
I can change it to the suitable web site url in my envrionment.

"Test Internet - Download"
button goal is to test wget and aria2 using our server but do not test WinHTTP.
Bonus:
It make us sure end user can get updated plugins from server with plugin download button.

does "Test Internet - Download" works ok ?

==>
That is the reason of 2 buttons.

*
With other words:
I can add
"Test Internet - Custom" with wget as you suggest,
But it will not test WinHTTP and we loose Bonus.
to keep Bonus If "Test Internet - Download" button do not work,
 I can add change "Test Internet - Download" button download a .html file if that suits better on your side ?
  Or we need another way.....



Edit:
and reminding:

You miss to respond important question, does blexactserver works behind proxy ?

"Test Internet - Ping" button goal is to test "WinHTTP"

I though it is just another way to test the proxy without downloading file.
well, as you stated wget or aria2 can't test it, it is necessary.

Let me know something other than google without wget and aria2 if possible.
for me, www.baidu.com is good to test. if you mean other tools to test WinHTTP? sorry, I don't know.

does "Test Internet - Download" works ok ?
==>
That is the reason of 2 buttons.

OK. I missunderstand about the "ping test".

I can add change "Test Internet - Download" button download a .html file if that suits better on your side ?
  Or we need another way.....

no need for that. as the pulgins is on the http://win10se.cwcodes.net, should be a file on it.
(but for proxy plugin, the internet access test, a custom url input textbox will be good)

You miss to respond important question, does blexactserver works behind proxy ?

I can't test it yestoday. now I tested, it works without proxy settings plugin.

PS:
When I use everything to confirm the download file, I found blexactserver downloads file in IE.Content Temporary Internet Files first,
so I think it is using system Inet interface to download files, and auto using system proxy settings.

Hi slore,

Let me know something other than google without wget and aria2 if possible.
for me, www.baidu.com is good to test. if you mean other tools to test WinHTTP? sorry, I don't know.

Thanks for feedback.
added Ping www.baidu.com -n 1 -w 1000
->
download Macro Library v46.r30 on server.

See if "Test Internet - Ping" button now works better ?

ps: we can add more ping websites to test 

(but for proxy plugin, the internet access test, a custom url input textbox will be good)

Do we still need such option with ping www.baidu.com
If yes provide me a wget line for such test so I can work on a solution on this.....

PS:
When I use everything to confirm the download file, I found blexactserver downloads file in IE.Content Temporary Internet Files first,
so I think it is using system Inet interface to download files, and auto using system proxy settings.

Thanks for confirming blexactserver using system (IE) proxy settings. 

ps: For now we do not have a method to test "system Inet interface" 

You miss to respond important question, does blexactserver works behind proxy ?

I can't test it yestoday. now I tested, it works without proxy settings plugin.

Good news. Proxy works fine with blexactserver --> we can jump over GFW with all project tools that downloads from internet. 
 
 

(moved)

Do you know proxy cache server?

Yes I understand idea behind.

blexactserver should be using cache server. 

>There is most people download firefox from the Homepage, but rare for the embbed script of the firefox from PESE.
Yes but I explained previously,
 most people like to build a project out of box, instead of waiting first download ff etc. during a build.
  internet connection is not good for all, better a project work as offline as possible.....

with other words: If I put ff to project with default false, no need filecontainer (like HD Clone)
---> I explained all in detail with same examples on previous posts....

*

We already avoid by using container plugins like "AgentShack File" for plugins bigger than 10 MB

separate the script and binary file is good. but why not keep what they are than repacking them in embbed script.
easy for updating, and directly run or be replaced by end user.
(folder or exe -> 7z or zip to get small size)

"AgentShack File" already have 7z to and get small size. 

Your Question is elsewhere, you are asking for setup.exe or .zip distribution (bigger than 7z) on other plugins.
Reason behind is "Passing Time" and "Gain Time"
It is not possible to catch app changes if we repack to 7z,
 when you use original setup.exe or .zip file by the app author, after some months (Passing Time) when time comes to update,
  It is very easy and fast to compare changes following author distribution. (Gain Time)
When one use 7z inside a plugin
 You have to first figure out what plugin author changed from original app author distribution with custom 7z distribution with a previous version
------> frequently it is not possible to obtain old version original distribution to compare easily.
------> maybe plugin author removed files or add custom configuration files or ......... all in one 7z
-------------> lost big time when updating plugin...
with other words:
Packing all to 7z looks better at first (smaller & faster) but cause valuable time lost later.....
Bonus:
+ you have a copy of ready setup.exe which you can install to hostos
+ end user sure files are not bad (since from original)
+ app authors feel better with their untouched distribution (they write to webpages and license... unmodified copy...)
and bad side:
- distribution is bigger
- spend little more time during build (some rare plugins have internal cache if time is not little ;) )

Advantages are more than Disadvantages since time is important.

++++
when we can not use setup.exe or original.zip, (generally no way to extract from .exe)
 sometimes adding history info to recreate 7z helps.

eg.
See ChrisR notes:
Call,DownloadX,"D:\B ase\Projects\Yomi\Final_Y\NVDA_File.Script
[HistoryCreate7z]

on other cases, I simply pack original installation folder and do configurations via plugin....

*
It is good you have github advantages 
Comparing, Personally I can use any ftp or similar server independent from github.  but requires a lot work.....
I support your github ideas. 

*
About md5 optional:
on any plugin - edit - options - "Calculate MD5"
something nobody use 

like packing all plugins to 7z and check md5 , some ideas look good at first but does not live since not much useful.


****
Anyway, sometimes "live and learn" required 

I have to go out now.... See You....

(moved)

》 after download SwRepo, people can directly use fetch.cmd all to download apps for later building.
Partially good idea for small project.
Remind me old days I make a quick download of all and re pack distro to be sure things work as designed with downloaded files to avoid updates of apps download cause unwanted result.
( previously explained with chrome example and reason of file container plugins)


*

That is why this topic be posted.
missing only 4MB tools what is need GWT.exe to get them. now the proxy is fixed,  but the project cannot work as offline as possible.
even core building without APPs.

Default project work fine and offline on hostosx64 after 4 Mb download ?
What do fo you mean ?


》It is hard to explain what in my mind 
I understand.
I did very same for big innounp distro
Eg.
Plugin first build download or extract  Setup.exe
--》
Setup.7z
Created.
2nd build check Setup.exe exist, if yes md5, if yes use Setup.7z
3rd build
New setup.exe, md5 mismatch create new setup.7z
====》
Since setup.exe not frequently updated, plugin works very fast.
( I call this : Plugin Cache)
+++
Setup.7z contains same with Setup.exe extraction
Additional Setup_Plugin_x64.7z and Setup_Plugin_x86.7z to make plugin fastest possible.
( Faster Plugin Cache)
==========》
Your idea is doing same on server side to download smaller + will be faster plugin.
I did on local folder only to get plugin process faster.

(
It is also hard to explain what in my mind , I hope you understand
)

Disadvantage on my case is it makes plugin very complex and a bit ugly and hard to explain regular plugin creators...
There is not total 10 plugin that need such thing on my case ...
It maybe more useful for your server ideas.

*
I wait Galapo back from Summer Time fun.

+
It would be better share these ideas on WimBuilderUI
http://theoven.org/index.php?topic=2390.msg30797;topicseen#msg30797

Or another topic  ( Slore Builder ideas )

That will gather things better for you ideas.

Hi slore,

updated Utils\Proxy Settings plugin (v8)

so only using "Import Proxy Settings From Internet Explorer" button will get proxy with download buttons operational.
ps:
If user do not have IE proxy setting but use "Import Proxy Settings From Internet Explorer" button , no proxy setting added, download buttons continue operational.

+ other useful small things...

Macro Library v46.r31 on server.
->
If you already use Utils\Update XXX (Exact and Secure)
(or extract same copy of zip file as described earlier)
+
Utils\Proxy Settings plugin (v8)
If you use "DisableDownload" button


===>
all download buttons on plugins will only revert back to original plugin. 

(Good for Offline development....)

**
"Back to Default plugin" button was an idea for a long while,
But I had decided not to update all plugins for that
 and 2 or more years ago I felt it maybe not good to provide a "Back to Default plugin" button.
  Today too many plugins to update for that....


***
In reality, to me builder should provide Upload Plugin, Download plugin, Compare Plugin (=DUC Buttons), Back to Default Plugin buttons... but because of closed source winbuilder we can not add such thing.

Since I already update many plugins with DUC buttons,
 What I did was using Download plugin button, as Back to Default Plugin button with using latest proxy improvements + Call,DownloadX improvements with a fake proxy setting.

***
Additional: DUC + Back to Default plugin buttons are also something to keep in mind for your future developments.

Macro Library v46.r32 on server.
only codes improved with things about my previous reply.

Macro Library v46.r33 on server.
Compare button on plugins now have ability to work offline the same way described at Reply55

PreRequest: you should once use Compare button online with any plugin to get default compare app. (current WinMerge)

Default project work fine and offline on hostosx64 after 4 Mb download ?
What do fo you mean ?

Package   Date-MD5   Package Description   Download count
Win10PESE (321.27 MB)   2019-01-05
http://win10se.cwcodes.net/Compressed/download.php?loc=Win10C&httploc=win10se.cwcodes.net/Compressed/&file=Win10PE_SE_2019-01-05.zip

after download the zip, I copy the ZIP to another PC, which one can't connect to the Internet.
and it can't build PE, because it was blocking on downloading dism, oscdimg, imagex.

so packaged PESE zip achive can't use in offline. that was this topic:
Win10PESE cannot build in no internet/connect to internet with proxy environment

the proxy environment is done after your proxy settings plugin.

Macro Library v46.r33 on server.

I will teset the new plugin next week.

so packaged PESE zip achive can't use in offline. that was this topic:
Win10PESE cannot build in no internet/connect to internet with proxy environment

About:
no internet
A)
->
+ I already accidentally made DUC Buttons work with no internet. 
+- For now Win10PESE project requires minimum 4 MB ish files that needs to be provided to no internet pc.
Since you like manual methods: on an internet available pc:
1) extract Win10PESE zip to a folder (eg. D:\Oven)
2) extract Win10PESE zip to other folder ( D:\Oven\Workbench\Common\Download_Win10PESE\ )
(Memo: I tried to provide a cmd for these 2 steps in past, but requires better instruction.)
3) Open builder
4) Use "Utils\Update XXX (Exact and Secure) to update latest from server
5) make your build. --> this will download 4mb ish + files and make you sure build works fine.
6a) If your offline pc is x86 :
6a1) Apps\Network\Mozilla Firefox ESR (P) -> use "Download Container x86" button (right top) and/or use Download 2 files button.
6a2) Virtual Test\qEmu X Emulation -> use "Download Container x86" button (right top)
6b) Also if you plan to use:
6b1) Apps\HD Tasks\Imaging\HD Clone ->  use "Download Container" button (right top under yellow color Compare button ("Download and Compare with an Editor Button)" And/Or use "Download" button.
6b2) Use compare button on any plugin. (eg. Apps\HD Tasks\Imaging\HD Clone ->   right top - yellow color "Compare" button
*****
---- ALL 6* steps are optional If you do not plan to use such plugins or buttons on offline pc no need them.----
*****
7) If you had use Proxy settings on online pc and you do not want to share proxy info with offline pc user etc.:
Utils\Proxy Settings ->
+ click small "d" button at top right + "Save Proxy Settings (Apply) button"
+ click "open dir" button  (at left of small "d" button) ->Delete ALL Files.
(Later "Utils\Proxy Settings" will have more simplified UI, It is still under construction  )

8)
Pack
D:\Oven\BuilderSE.exe
D:\Oven\BuilderSE.ini
D:\Oven\Projects
D:\Oven\Workbench\Common
to a new file with current date.
(eg. Win10PE_SE_2019-01-12_HomeMade.7z)
ps: I Highly Recommend 7z to gain maximum space with double files..... (zip is quite bad with double files)

9)
Extract Win10PE_SE_2019-01-12_HomeMade.7z to offline pc (eg. D:\Oven)
Open Builder
Use Utils\Proxy Settings -> "Disable Download" to get DUC buttons operational on offline pc.
Further make your build (select source + click big blue play)

+
Additional Step6)
You can also provide Win10 cd with your packed 7z having a pre setting at "Config Source" plugin.
This will make your Win10PE_SE_2019-01-12_HomeMade.7z size very big (+Win10CD) but will get it work out of box without providing sourcecd.
eg. Simple instruction to new user: Only extract , open builder and click BigBluePlay

If you need such thing, let me know, there are things to do carefully for such thing.

*
DUC Buttons = Download and Compare buttons on plugins.

*****
There is also none manual, automatic way, but requires an update for latest proxy changes (Step 7)
Probably tomorrow or next week.
Since there is already a manual way and you seem to like manual ways more,
I believe It will be better to simplify Utils\Proxy Settings UI first. 

ps:
Writing manual way is a bit long, but in fact it is simple for frequent users, what we do simply:
 "RePack Project with Downloaded files without personal files and adjustments"


B)
Project very close to evolve to get zip out of box working to build project without internet.
This will limit some existing and new features of the project Bob.Omb working on.
With Other words:
The Project zip will work out of box, create a bootable Win10PESE on offline pc, but for further features, files will need to be provided from an internet access pc.






Edit:
Various fixes and additions.