WimBuilder2 - New Builder with Re:WIN10XPE project(~20H2 Supported)

*
I have both iphone and android
(well old iphone and android enough to test mtp)
I will try to test if possible with virtual (virtualbox or vmware ?)

@slore
since you use VirtualBox, is it ok to test mtp with VirtualBox ?

*

200 lines...I though WinBuilder's plugin script will be more than main.bat,
like 3~4 times. never though it be 10 times.

Bob.Omb hasn't start to use new syntax Call,RegCopyKey yet (2018.10.06) , it will be shorter in future.
Still easier to follow codes compared to cmd 

@slore
since you use VirtualBox, is it ok to test mtp with VirtualBox ?

YES, the screenshot is captured in VitrualBox.
(connect to HOST first, select transfer pictures or files(default is "charge") on android phone, then change USB device -> VirtualBox VM)

Hi slore,

In addition to your reply 41 (+44 +45) http://theoven.org/index.php?topic=2660.msg30784#msg30784

downloading files from app server to a separate folder other than project folder will make life easier for you when packing project.
(First natural idea that comes to mind is downloading or providing files with plugin subfolder, like BartPE plugins, looks simpler but complex when packing and other things...)
eg.
Prz42 plugin - http://theoven.org/index.php?topic=2693

Code: [Select]

[Process]
If,ExistFile,%api%,AddVariables,%api%,ApiVar,GLOBAL
//--
Call,DownloadX,"%BaseDir%\Projects\Yomi\AppY\HD Tasks\Imaging\Acronis_Universal_Restore.Script"

http://theoven.org/index.php?topic=2672.0

plugin have Download button that downloads to
..\Workbench\Common\apps\Acronis UniversalRestore_Files_WinPE\

as a result no trouble packing projects which are at different folders.
..\Projects\Gena
..\Projects\Win7PESE
..\Projects\Win8.1SE
..\Projects\Win8PESE
..\Projects\Win10PESE

.....
There are some more advantages other than packing project (+updating project easier with packed file etc. etc.) .... , in the end It is better. 

2019-01-24 updated:
 wimbuilder2-master.zip (3148.93 kB - downloaded 121 times.)

* [NEW]identify minimum SwitchToAdmin patch(and no depandecy on DWM, WinXShell can be used as shell)[use pecmdAdmin.ini, so need PECMD patch at Reply #51]
* identify minimum(files) DWM patch(enable DWM without windows explorer patch)
* add Korean translation(from nonameman)[WimBuilder2 and all  Patches' settings]
* update TextReplace macro
* other minor detail fixes

PUT WinXShell on
WimBuilder2\Projects\WIN10XPE\01-Components\00-Shell\WinXShell\X_x64\Program Files\WinXShell
WimBuilder2\Projects\WIN10XPE\01-Components\00-Shell\WinXShell\X_x86\Program Files\WinXShell

will be applied, but some improves I want to add, like a UI_AccountSwitch, update with feedbacks,
and Open-Shell(Classic-Shell) patch.



I hope I can make some time to do this.

about the 3rd useful APPs for PE, like
PECMD, PENetwork, 7-ZIP, StartIsBack, Open-Shell, ...

I make a Software Repo to manager them. When you want to update them, here is a way to do that without UPDATEing plugin script files,
just fetch and rebuild, it will be done 

The project calls swRepo now. and use fetch.cmd <name> <version> <arch> to download it, and return the file name,
so we can add the files into wim file.



also I use WimBuilderUI's framework make a fetchUI.
it will show the status of software, and offer  [delete] and [update] button.



Someday, it will be merge into WimBuilderUI as a plugin(add-on)

7za, aria2(x64/x86) make it little big.
 swRepo-master.zip (5477.17 kB - downloaded 111 times.)

run fetch.cmd without parameter first, will extract 7za, aria2 command,
then you can use fetch.cmd <name> <version> <arch>, or fetchUI to try it.

PS:
If someone know other download command, that support https and smaller, please tell me.

see PENetwork\info.js, 7-ZIP\info.js, you will know it is very simple.

Code: [Select]

function fetch_init() {
  if (arch == 'x64') add_var('_arch', '_x64')
  if (arch == '' && ver != 'latest') {
      error('Error:missing arch option')
      quit(1)
  }
}

info = {
  "desc":{
     "author":"Holger",
     "home_page":"https://www.penetworkmanager.de/"
  },
  "versions":[
    "latest",              // is offering latest version download URL from home page
    "v0.59.B10"
  ],
  'arch': ['x64', 'x86'],
  "fetch":{
    "latest":{
      "archive":"PENetwork%_arch%.7z",
      "from":"https://www.penetworkmanager.de/scripts/PENetwork%_arch%.7z",
      "fallback":"PENetwork/PENetwork%_arch%.7z"
    },
    "%version%":{
      "archive":"%version%/PENetwork_%arch%.7z",
      "from":"https://www.penetworkmanager.de/scripts/PENetwork_%arch%_%version%.7z",
      "fallback":"PENetwork/PENetwork_%arch%_%version%.7z",
      "verify":"MD5:xxxxxxxxxxxxxxxxxxxxxxxxx"
    }
  }
}

Code: [Select]

function fetch_init() {
  if (arch == 'x64') add_var('_arch', '-x64')
}

info = {
  "desc":{
     "author":"Igor Pavlov",
     "home_page":"https://www.7-zip.org/"
  },
  "versions":[
    "1806",   // first one be the 'latest'
  ],
  'arch': ['x64', 'x86'],
  "fetch":{
    "%version%":{
      "archive":"7z%version%%_arch%.exe",
      "from":"https://www.7-zip.org/a/7z%version%%_arch%.exe",
      "#fallback":"7-Zip/7z%version%%_arch%.exe"
    }
  }
}

  also you can point to the http://win10se.cwcodes.net/Projects/Win10PESE, and WBzip extract them, to get the customized files.

http://win10se.cwcodes.net/Projects/Win10PESE/Shell/StartIsBackPlusPlus.Script
http://win10se.cwcodes.net/Projects/Win10PESE/Apps/Network/Mozilla_Firefox_ESR_x64_File.Script

2019-1-30 updated:
 wimbuilder2-master.zip (4668.5 kB - downloaded 90 times.)

* 19H1 Supported
* Background Picture option for SwitchToAdmin
* added mspaint and Windows Photo Viewer
* add _CustomFiles_ folder for wallpaper.jpg, screen.jpg，PECMD.ini, PENetwork.ini, etc.
(for now only wallpaper.jpg, screen.jpg, ini file will be supported later)







19H1 is not functional for all features with current additions as RS5 used, test result:
 [OK]Windows explorer
 [OK]MMC
 [OK]DWM
 [OK]Search
 [OK]WOW64
 [OK]Remote Desktop(Client)
 [NG]Cred Console Dialog for accessing share folder list(\\192.168.56.1)([OK]net use command)
 [OK]Cred Console Dialog for accessing share folder(\\192.168.56.1\C$)
 [OK]IME indicator(added required file in 2019-02-05)
 [NG]IME
 [NG]SwitchToAdmin
 [unstable=>OK]IE(added required file in 2019-02-05)
 [OK]MTP Support(bug fixed in 2019-02-05)
 [OK]Windows Photo Viewer

PS:
Franch translation will be added.

Thanks for nonameman about 19H1 support, and Korean translation.
I'm glad to know, someone can use this framework to add CORE feature without document.

(Thanks to the simple batch script syntax)

Thanks for nonameman about 19H1 support, and Korean translation.
I'm glad to know, someone can use this framework to add CORE feature without document.

(Thanks to the simple batch script syntax)

The Real reason is, good users always feedback nicely, whatever framework behind,
 even without any document but only following codes if opensource since BartPE.

In time You will face others who only come around to only waste time in forum world... 

Anyway, Go WimBuilderUI go 

2019-02-05 updated:
PECMD, StartIsBack included
 wimbuilder2-master.7z (4732.95 kB - downloaded 156 times.)

• [NEW]19H1 perfection(IE, IME indicator, MTP support)
• [NEW]add HyperV.cmd test command(powershell script from noel)
• [NEW]add French translation(from noel)
• add ring0 kernel driver mtpHelper.sys for MTP_Support(yamingw[setwow64.exe's author])
• fix issue that the SwitchToAdmin can't work on notebook(yamingw)
• add generic drvinst.exe patcher(from JFX)

19H1 is not functional for all features now, test result:
 [OK]Windows explorer
 [OK]MMC
 [OK]DWM
 [OK]Search
 [OK]WOW64
 [OK]Remote Desktop(Client)
 [NG]Cred Console Dialog for accessing share folder list(\\192.168.56.1)([OK]net use command)
 [OK]Cred Console Dialog for accessing share folder(\\192.168.56.1\C$)
 [OK]IME indicator(added required file in 2019-02-05)
 [NG]IME
 [NG]SwitchToAdmin
 [unstable=>OK]IE(added required file in 2019-02-05)
 [OK]MTP Support(bug fixed in 2019-02-05)
 [OK]Windows Photo Viewer

Make attachment size(x86+x64) < 6MB by 7zip, so remove the x86 attachment.

Hi slore,

*

Make attachment size(x86+x64) < 6MB by 7zip, so remove the x86 attachment.

7z compress better with double files.

I use this feature with "Pack Projects" plugin to pack untouched copy of projects to get "Update XXX (Exact and Secure)" plugin "Restore" button (+ Restore Full checkbox) work out of box.
---> It simply help to do same with re-extracting project to an empty folder. 
------> (Like many do, but I was bored open new folder, extracting again and again ... and put a button for that goal  )

Just an idea for your WimBuilderUI, to take another advantage of distributing with .7z

Tip:
Not tested but an idea:
 you can also wim compress instead of .7z and filename "wimbuilder2-master.wim.7z" to give end-user impression end user can extract package with 7zip.
(use wimpack to high compress not 7z)
Maybe get your package smaller......

*
and special thanks to binmay.exe again, I use it a lot during the creation of nvidia plugin...

*
I am trying to catch MTP improvements, following noelBlanc, your (slore) and Bob.Omb posts.
- the only one thing I notice: Virtualization must have usb2 or driver integration fails (fails with usb1.1)
 I am trying to get note3 mtp work in virtual.....
  more later, still reading...

*
a little cosmetics:
Configures\"System" -> Administrator Options -> Background

on my side I renamed it separately to "WallPaper Login"
same Login Wallpaper also used by "Slide to Shutdown"

up to you, only cosmetics.

*
memo:
Configures\"System" ComputerName -> limitations: 15 chars (sandy) + special chars (ChrisR)

Not tested one by one or very deep, but so far:
limitations I guess only for
ComputerName\ComputerName
ComputerName\ActiveComputerName
other values seem to be fine without limitation.

ps: I also remove space from ComputerName and ActiveComputerName to keep on the safe side, not sure if required...

*
Maybe an "Edit" button will be nice to edit patch files main.bat with notepad ?
(Maybe not, maybe a hidden option or whatever, only sharing ideas)

*
"Open Patch Folder" looks like a good idea 
eg. Left tree Patches --- Configures --- System : A button or whatever to open \00-Configures\System\ folder.

***
Too many to do things around to finish and very low free time,
 I am quite slow to feedback or respond, More later 

Hi, Lancelot

Thanks for the feedback and suggestion.

THE UI settings part just be enough for some settings and patches selection,
I'm working on the CORE part, I will catch the UI part improvement later.

1. **19H1, 20H1 Support**
1.1 fix SwitchToAdmin issue
1.2 fix IME issue
2. improve AddFiles macro to auto add MUI files, make *.bat be more simple.
3. Merge SoftwareRepo to WimBuilderUI
4. Convert new core features from WIN10XPE and WIN10PESE

I also have things want to do on the UI part, like:
1. selection check(dependency), option check.
2. custom preset(config.js, full.js, net.js, custom.js ...)
   default option settings, and default patches selection suite.

...

2. improve AddFiles macro to auto add MUI files, make *.bat be more simple.

yes, I thought about that too....
But for today I decided to keep simple, and decide not to touch on that area.
This way It is easier and simpler to follow wimlib-imagex and 7z using filelist ...

Still only sharing ideas :
1)
Since Win10PESE already use line by line copy files (like cmd copy command) at core builds (1 - Copy Files)
using JFX commands eg. InstallCopySys32 and InstallCopySys32Mui etc.
Same commands, instead of copy, create a file list to be used by wimlib-imagex
This idea already implemented to relevant plugins, but not public for now.
(and slow because of wb bad design, no speed gain, same idea would be much faster with cmd)

I believe It will be bad for WimBuilderUI.

2)
2nd idea is using file list to check all mui files without using \??-??\
I guess that is what you mean.
eg.
[Require_FileList]
\Windows\System32\wuaueng.dll
\Windows\System32\asferror.dll
\Windows\System32\??-??\asferror.dll.mui
\Windows\System32\devenum.dll
\Windows\System32\??-??\devenum.dll.mui
--->
[Require_FileList]
\Windows\System32\wuaueng.dll
\Windows\System32\asferror.dll
\Windows\System32\devenum.dll

A)
I guess you will extract file list to a file (or already create file that way)
 and will automatically add \??-??\
eg.
\Windows\System32\wuaueng.dll
\Windows\System32\asferror.dll
\Windows\System32\devenum.dll
--->
\Windows\System32\wuaueng.dll
\Windows\System32\asferror.dll
\Windows\System32\devenum.dll
\Windows\System32\??-??\asferror.dll.mui
\Windows\System32\??-??\devenum.dll
\Windows\System32\??-??\wuaueng.dll

and use extract ....

B)
using dist mui languages instead of \??-??\  (method 1 above naturally use that)

since you already have Patches-Slim-Remove: other MUI files, no need to get things complicated with B)


******
Overall, I only wrote things I had thought before in this area,
As I wrote at the top, for today I decided to keep simple and decide not to touch on that area.

*******
Well See you on next post, I tend to split posts when a post is too long.... 

1. selection check(dependency), option check.

1.
Advice:
Keep dependency simple
 We have quite good experience with Gena,
  which has various dependencies because of the very high number of combinational kind of builds and requirements.

Well since VistaPE we do not have such troubles ....  everybody happy with a packed .wim file, and things can be done after packing....
 I do not think WimBuilderUI need such thing, Still One day, If you decide to improve dependency from simple to complex, let me know, I will share Gena experience.

+
Keeping dependency simple:
I simply write info to UI of a plugin.
giving example from your MTP Support patch converted to Bob.Omb MTP Support plugin
I recently wrote bold:
Requires:
\Drivers\'DrvInstPatch' Plugin
to UI of MTP Support plugin
(keeping simpler, I did not write only required at x64 - advanced users can read from history notes, %99 others do not...)
Similarly some Apps plugins have bold UI with
Require .Net ....
Require VC ....
etc.

I also use Description for requirements of windows versions and architecture
eg.
\Tweaks\Themes\"Theme Ades Theme for Windows 10 by unisira" plugin
Description=(Win10 - v1511 10.0.10586 - v1607 10.0.14393 - v1709 10.0.16299 and later (Tested ok with v1809 10.0.17763))

I put lots of Themes to server recently, that are compatible to different Win10 builds and os since Win7 ...

another example with a simple app, Q-Dir Nenad
Description=(x86/x64 7.4.5.0)

just an interesting example:
\Components\WallPaper Login plugin
If someone use this plugin without following bold info (or without enabling plugins on info)
It is useless to use such plugin, well up to end user .

Summary:
To keep dependency simple, I use notes on Description and Plugin UI.

With passing years, I can say It works fine with the public. 
 No complaint or misunderstanding yet.
   And easier for us, no need deal with complex dependency coding...

2. custom preset(config.js, full.js, net.js, custom.js ...)
   default option settings, and default patches selection suite.
...

As I wrote before,
 you can use the advantage of distribution with 7z (or .wim) + you have your own builder ui power.

giving a simple example:
going back to default settings of WallPaper_Login plugin is very simple, just:
copy
..\Workbench\Common\Download_Win10PESE\Projects\Win10PESE\Components\WallPaper_Login
to
..\Projects\Win10PESE\Components\WallPaper_Login

As you know from Proxy plugin development, I add this feature to download button ....
But since We do not have distribution advantage like yours, It is not out of box working.
(
-->
I mean for today we do not provide a 2nd copy of project at ..\Workbench\Common\Download.... on distribution, not good for .zip ...)
End user should once update from server using Utils\Update ....
)

Shortly: By adding a Description area to your patch :
1) You can use the same Description area to give some requirement info (windows versions and architecture)
2) You can add a button with "go back to default" at the right of your Description area. (use the advantage of having builder ui design of your own)

eg. with your IE patch, such "revert to default option" button should do:

rd ..\Projects\WIN10XPE\02-Apps\Browsers\IE
dircopy ..\Workbench\Common\Download_WIN10XPE\Projects\WIN10XPE\02-Apps\Browsers\IE to ..\Projects\WIN10XPE\02-Apps\Browsers\

 

To me, from lots of experience, It is simpler, safer and easier (+ easier to maintain) than adding default.js custom.js  etc...

Well only sharing my ideas, rest is on your hands.

Time to go....

Hi, Lancelot
 
split posts

2)
2nd idea is using file list to check all mui files without using \??-??\
I guess that is what you mean.

Yes. I mean this.

WimBuilderUI already done like next:

call AddFiles %0 :end_files
goto :end_files
@Windows\System32\
+mui
actxprxy.dll,atlthunk.dll,control.exe
+mui(en_US,%DIST_LANG%)
comctl32.dll

:end_files

will automatically add \??-??\ , and add specify language mui file
=>

Windows\System32\actxprxy.dll
Windows\System32\??\actxprxy.dll.mui
Windows\System32\atlthunk.dll
Windows\System32\??\atlthunk.dll.mui
Windows\System32\control.dll
Windows\System32\??\control.dll.mui
Windows\System32\comctl32.dll
Windows\System32\en_US\comctl32.dll.mui
Windows\System32\zh_CN\comctl32.dll

since you already have Patches-Slim-Remove: other MUI files, no need to get things complicated with B)

A),B) already be done ...

BUT I want to be more simple because:
1. from 19H1, Microsoft add new "mun" file in "\Windows\SystemResources\".

To support is, we should add the mun files in each patch/plugin.
I don't know how you will do with this.
for WimBuilderUI, I  don't want to pick up the files, and change the patches' main.bat for this meaningless update.
so I already deal with this in macro AddFiles for support 19H1, 20H1.
a). wimlib-extract file list of "\Windows\SystemResources\".
b). check the add file name whether it in the file list.

so, it is easy to be get the "\Windows\System32\%DIST_LANG%" file list, and deal with mui files as same as mun files.

2. I convert core feature from WIN10XPE. ChrisR wrote mui files clearly.
I'm lazy to add them one by one, so I add +mui on the top, then
wimlib-extract shows warnings, actually no problem for PE building.

Code: [Select]

[WARNING] No matches for path pattern "\Windows\System32\??-??\actxprxy.dll.mui"
[WARNING] No matches for path pattern "\Windows\System32\??-??\atlthunk.dll.mui"
[WARNING] No matches for path pattern "\Windows\System32\??-??\control.exe.mui"
[WARNING] No matches for path pattern "\Windows\System32\??-??\comsvcs.dll.mui"
[WARNING] No matches for path pattern "\Windows\System32\??-??\CoreMessaging.dll.mui"
[WARNING] No matches for path pattern "\Windows\System32\??-??\CoreUIComponents.dll.mui"
[WARNING] No matches for path pattern "\Windows\System32\??-??\cscapi.dll.mui"
[WARNING] No matches for path pattern "\Windows\System32\??-??\IconCodecService.dll.mui"
[WARNING] No matches for path pattern "\Windows\System32\??-??\mfperfhelper.dll.mui"
[WARNING] No matches for path pattern "\Windows\System32\??-??\MrmCoreR.dll.mui"
[WARNING] No matches for path pattern "\Windows\System32\??-??\mscories.dll.mui"

also when people got wrong, and commit the last_build.log, this is usless infomation for checking the real problem.
if this is done, I can remove lines of \Windows\System32\??\xxx.mui, make converting work be fast, easy, and less file list in main.bat, and reduce 40% file list.
with ChrisR's file list, I need +mui, -mui, ..., switch them now...

I only want to deal with file in \Windows\System32\, and no wildcard in file name, 90% core features's required mui files is in \Windows\System32\.
for \Windows\System32\winusb*.dll, still needs +mui, for other folders, still need +mui.


*********************************************
Even there are InstallCopySys32 and InstallCopySys32Mui, or AddFiles xxx, AddFiles MUI ... commands, macros,
but why should we need check if the file has or hasn't mui file to use different command(s) or macro(s), as it could be implemented with less codes.
*********************************************

wim, 7z compress

I thought that pack x86/x64 application in wim file, the same files will be just one,
and keep x86/x64 folders be separated, like StartIsBack, PENetwork, WinXShell.

and write Z2X macros, for extract StartIsBack.wim by DIST_ARCH =>
index X_x64,X_x86  => X:\
index X_PFx64,X_PFx86 => X:\Program Files\
index X_SYSx86,X_SYSx64 => X:\Windows\System32\

and leave custom ini, reg file in patch folder.

for WimbuilderUI, I didn't consider it.
both x86 or x64 dist file, just with different tools in bin, others is same.
both them can create x86/x64 PE.

I separate it, just because the forum attachment restriction.
after merge SoftwareRepo, maybe only the github master donwload url is fine.

I don't want to upload exe,dll files as source code to github repo.

If you use Git, just git pull ... will fetch the changed files only, no need extract files to new folder.

WimBuilderUI is smaller than 6MB, when it grows up, I will consider this more.

and special thanks to binmay.exe again, I use it a lot during the creation of nvidia plugin...

I mailed to Sloaring, but the address is out. I see the history log, the 2011 version just fixed the bug I found in v0.3,
and add some new options. the BASE feature is fine, if there are bugs, I will keep fixing it.

- the only one thing I notice: Virtualization must have usb2 or driver integration fails (fails with usb1.1)
 I am trying to get note3 mtp work in virtual.....

I don't much device to research so far...

I am trying to catch MTP improvements, following noelBlanc, your (slore) and Bob.Omb posts.

services.exe needs init something, but in WinPE, it failed.
the services.exe is hard to modify, so I wrote a dll for changing the MiniNT key, and now yamingw has offered mtp_helper.sys driver,
do this(cheat services.exe) with Ring0 kernel driver.