PEBakery Test Build

Hi ied206,

about weblabel :

Giving more width to WebLabel_Plugin can address this issue.
It is because DirectWrite, the key API for super-fast text rendering, tends to render texts with a little bit wider width.

I can not increase wider since wb cheat on this case 

Long story to short:

Giving example:
Q-Dir:
WebLabel_Plugin="Plugin Page",1,10,338,4,55,18,http://TheOven.org/index.php?board=9

Ignore width value (55) on PEBakery
just start with 338 and draw Plugin Page
let PEBakery autodecide width
-->
This will be full compatible to all wb versions since wb cheat same way on weblabel values.

Hi ied206,

When I tested, those two plugin's interface changed well with System,RefreshInterface.
Does this bug still appears in lastest version?

yes,
here are 2 videos:
https://www.dropbox.com/s/xq955ozwcxk1gm7/Video_2017-11-16_001728.7z?dl=0
https://www.dropbox.com/s/wofsnaufvc8bskb/Video_2017-11-16_002131.7z?dl=0

ps:
notice buttons do not work fast, it waits a lot time before starting processing.

*
I will continue further tests and respond other things mentioned when Set - Nil implemented.

I hope Set - Nil is the last critical thing to test things easier. 

We will see, slowly...
Good night.

Cosmetics:

we always instruct users to click "Big Blue Play" button

It will nice to have a "Big Blue Play" button instead of "Big Dark Build" button so we do not need to update tutorials. 

Good Night.....

TestBuild 20171117

EDIT
TestBuild taken down because of potential license issue.
Please use beta 2 or later, sorry for inconvenience.

Changelog
- Support Set,(...),NIL
- Experimental HTML log format support

The build date was deliberately set to 1117, to flush existing plugincache.

1.
To support System,SaveLog command, HTML log format support was added.
Samples :  FirstLog_20171116.7z (534.27 kB - downloaded 84 times.)

System,SaveLog command was extended to give freedom of choice:
- System,SaveLog,%Dest% -> HTML export
- System,SaveLog,%Dest%,HTML -> HTML export
- System,SaveLog,%Dest%,TEXT -> TXT export

You can manually export system log / build log from LogWindow, too.
More work is on the way.

2.
I investigated Lancelot's video.
However, those operation are done very fast in my system.
I am testing PEBakery on SSD, so I suspect File IO may be the unsuspected bottleneck.
This is my video and logs:  Troubleshooting.7z (774.75 kB - downloaded 91 times.)

Lancelot, what is your testing environment? Are you using HDD, or SSD?

We need to figure out which command hangs PEBakery.
After running buttons, can you go to LogWindow -> Build Log -> Log (Detail) and investigate timestam?

Ex)


EDIT
Another bottleneck can be CPU power...

Hi ied260,

Asus laptop hd ( hdd 5200 rpm I guess) ,
 i7 cpu
Win10x64 host (1706 I guess...)

I will give more info and more test
 (project at ramdrive , cpu check ...) when I get back home.

*
It maybe good idea to see ms ( miliseconds ) between each command on PEBakery log window to figure out such things...

Hi ied260,

My Laptop:

Asus N56VZ S4402H Notebook
https://www.asus.com/Laptops/N56VZ/HelpDesk_Download/

CPU:
Intel Core i7-3630QM
# of Cores 4
Max Turbo Frequency 3.40 GHz
Processor Base Frequency 2.40 GHz
Cache 6 MB SmartCache
https://ark.intel.com/Search?q=Intel%20Core%20i7-3630QM

GPU:
Intel(R) HD Graphics 4000  (2112 MB)
NVIDIA GeForce GT 650M  (4 GB)

Disk:
HGST HTS541010A9E680  (1 TB, 5400 RPM, SATA-III)

Memory:
Total: 16 GB
+ 6 GB used for RamDrive
+ 5.7 GB always free

HostOS:
Microsoft Windows 10 Home Single Language - x64
Version 10.0.15063

+ No AntiVirus (Defender Disabled)

Hi ied206,

small bug,

enable .link file do not work when clicked "Build"

ex: halt plugin on one of previous post

ps: I will continue posting when things found...

Hi ied206,

when I put project (plugins) to ramdrive,
Interface change works in both cases. 

But still, I can reproduce interface change bug on normal disk.

movies:
after pressing the button (or after pressing "Build") ,

PEBakers waits a lot of time before starting.

See Movie:
https://www.dropbox.com/s/h6ebzbgrtau8br2/Video_2017-11-17_011116.7z?dl=0

I move mouse over circle ..... when clicked first time to only show PEBakery waits a lot time before starting....

ps: I used only Utils\Downloads plugin on movie since it should be very very fast to change interface of Utils\Downloads plugin when pressing "a" button.

Hi ied206,

CopyOrExpand - Command [CopyOrExpand] is deprecated

We need CopyOrExpand command for Gena.....

In fact it is a simple command,
ex1:
%99 this used
CopyOrExpand,%Source_Win%\WINLOGON.EXE,%Target_Sys%
or:
%0.8 this used
CopyOrExpand,%Source_Win%\WINLOGON.EXE,%Target_Sys%\WINLOGON.EXE

ex2:
very rarely (%0.2)
Explorer Plugin.
[Copy-XpRes]
If,ExistFile,%Source_Win%\xpsp2res.dl?,CopyOrExpand,%Source_Win%\xpsp2res.dll,%Target_Sys%
Else,If,ExistFile,%Source_Win%\sprt%LocalizeLID%.dl?,CopyOrExpand,%Source_Win%\sprt%LocalizeLID%.dll,%Target_Sys%\xpsp2res.dll



ex1 explanation:

copy %Source_Win%\WINLOGON.EXE if there is such file
if there is no such file
expand %Source_Win%\WINLOGON.EX_ to %Target_Sys%

If there is no WINLOGON.EXE OR WINLOGON.EX_ --> Error

CopyOrExpand -> Expand command uses same internals with windows expand.exe (extract cab compressed files)
https://technet.microsoft.com/en-us/library/cc722332(v=ws.10).aspx


*
During a cheated Gena test other things I notice:

*
Warning            StrFormat - Command [StrFormatType,ShortPath] is deprecated

ShortPath is rare command required to extract some .msi files on some plugins....

I feel it is easier to implement to PEBakery than we creating a replacement Macro ?


*
Set also used (rarely) to add Macro
Set - Invalid variable name [MenuAdd_GRUB]

ex macro: Add_Shortcut,
ex:
Set,MenuAdd_GRUB,"Exec,%ScriptFile%,Add-grub4dos-entry",PERMANENT

logic is same with variables. 

Add Macro to [Variables] or do with a command

ex:
Build\1 - Copy Files
[Variables]
BootCopyDirMui=Run,%ScriptFile%,PBootCopy_DirMui
or
[Process]
Set,BootCopyDirMui,"Run,%ScriptFile%,PBootCopy_DirMui"

should both work.


*
Error              Exec - The given key was not present in the dictionary. (If,ExistFile,%PrepPEGenScript%,Exec,%PrepPEGenScript%,Process

If Exec, not implemented to PEBakery, here is the explanation:

Different than Run, command:
1)
Exec, first reads [Interface] and [Variables] of target plugin and Runs...
2)
After Exec, all variables during process of other plugin deleted.
ex: You can not get variable of other plugin.

With other words:
Exec, do exactly same thing builder do during build of each plugin, only section selected.
(and keep in mind, both builder and Exec, reads [Main] Interface= value to decide reading [Interface] section)

ps:
We rarely use Exec since I discovered Exec, is very slower than Run, ... Still sometimes we must use it.


*
Halt,
command do not end plugin, it wait to the end of plugin.
Halt should stop build without processing next line.



***
Download latest Gena Appetizer
http://gena.cwcodes.net/Compressed/index.php

Delete:
\Projects\Gena\Finals\0 RegFactory\RegFactory.script file
or
\Projects\Gena\Finals\0 RegFactory\ folder
(To avoid PEBakery reported bug -> Level=0 plugin on build ! )

see how things go....
(you will get mouclass.sys not found since CopyOrExpand not working now)





*****
Gena Basic (Appetizer) is very small and build is very fast compared to any SE.
I always use Gena during some development tests with my available low free time.
It will be much easier and faster to Debug PEBakery with Gena......

For now, waiting for next version....

See You

Wow - so much progress.

 

Thanks guys for all the hard work and persistence with this!

Regards,
Galapo.

Yes amazing progress.

Note on Tweaks / Wallpaper on Win10PE SE X64 1709 source if I click the Browse button to the right of the text box under Use Custom Wallpaper app crashes to the desktop.

Build time is 17 minutes versus 14 minutes with BuilderSE.

Also I think the Command Progress progress bar could be removed. It flashes so much it almost makes you feel sick.

Here is the crash to desktop call stack:

System.ArgumentException occurred
  HResult=0x80070057
  Message=The path is not of a legal form.
  Source=<Cannot evaluate the exception source>
  StackTrace:
   at System.IO.Path.NormalizePath(String path, Boolean fullCheck, Int32 maxPathLength, Boolean expandShortPaths) in f:\dd\ndp\clr\src\BCL\system\io\path.cs:line 432
   at System.IO.Path.InternalGetDirectoryName(String path) in f:\dd\ndp\clr\src\BCL\system\io\path.cs:line 238
   at PEBakery.WPF.UIRenderer.<>c__DisplayClass20_0.<RenderFileBox>b__1(Object sender, RoutedEventArgs e)
   at System.Windows.EventRoute.InvokeHandlersImpl(Object source, RoutedEventArgs args, Boolean reRaised)
   at System.Windows.UIElement.RaiseEventImpl(DependencyObject sender, RoutedEventArgs args)
   at System.Windows.Controls.Primitives.ButtonBase.OnClick()
   at System.Windows.Controls.Button.OnClick()
   at System.Windows.Controls.Primitives.ButtonBase.OnMouseLeftButtonUp(MouseButtonEventArgs e)
   at System.Windows.RoutedEventArgs.InvokeHandler(Delegate handler, Object target)
   at System.Windows.RoutedEventHandlerInfo.InvokeHandler(Object target, RoutedEventArgs routedEventArgs)
   at System.Windows.EventRoute.InvokeHandlersImpl(Object source, RoutedEventArgs args, Boolean reRaised)
   at System.Windows.UIElement.ReRaiseEventAs(DependencyObject sender, RoutedEventArgs args, RoutedEvent newEvent)
   at System.Windows.UIElement.OnMouseUpThunk(Object sender, MouseButtonEventArgs e)
   at System.Windows.RoutedEventArgs.InvokeHandler(Delegate handler, Object target)
   at System.Windows.RoutedEventHandlerInfo.InvokeHandler(Object target, RoutedEventArgs routedEventArgs)
   at System.Windows.EventRoute.InvokeHandlersImpl(Object source, RoutedEventArgs args, Boolean reRaised)
   at System.Windows.UIElement.RaiseEventImpl(DependencyObject sender, RoutedEventArgs args)
   at System.Windows.UIElement.RaiseTrustedEvent(RoutedEventArgs args)
   at System.Windows.Input.InputManager.ProcessStagingArea()
   at System.Windows.Input.InputManager.ProcessInput(InputEventArgs input)
   at System.Windows.Input.InputProviderSite.ReportInput(InputReport inputReport)
   at System.Windows.Interop.HwndMouseInputProvider.ReportInput(IntPtr hwnd, InputMode mode, Int32 timestamp, RawMouseActions actions, Int32 x, Int32 y, Int32 wheel)
   at System.Windows.Interop.HwndMouseInputProvider.FilterMessage(IntPtr hwnd, WindowMessage msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
   at System.Windows.Interop.HwndSource.InputFilterMessage(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
   at MS.Win32.HwndWrapper.WndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
   at MS.Win32.HwndSubclass.DispatcherCallbackOperation(Object o)
   at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
   at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
   at System.Windows.Threading.Dispatcher.LegacyInvokeImpl(DispatcherPriority priority, TimeSpan timeout, Delegate method, Object args, Int32 numArgs)
   at MS.Win32.HwndSubclass.SubclassWndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam)
   at MS.Win32.UnsafeNativeMethods.DispatchMessage(MSG& msg)
   at System.Windows.Threading.Dispatcher.PushFrameImpl(DispatcherFrame frame)
   at System.Windows.Application.RunDispatcher(Object ignore)
   at System.Windows.Application.RunInternal(Window window)
   at PEBakery.App.Main()

Seems to be cause by the directory path being a blank string.

Also a cosmetic issue. I don't like the black folder icons in the script tree. The light brown folder icon in the original builder was much more pleasing to the eye.

You can find many nice icons here:

https://www.iconfinder.com/iconsets/musthave

when I put project (plugins) to ramdrive,
Interface change works in both cases. 

But still, I can reproduce interface change bug on normal disk.

Figured out why PEBakery worked too slow on HDD.
The SQLite 3 database used for logging was the bottleneck, since it caused file write too often.
However my system's SSD was too fast to find out this problem myself.
Delaying writing of log into database will fix this issue.

Also I think the Command Progress progress bar could be removed. It flashes so much it almost makes you feel sick.

The command progress bar was not working properly, so I was considering about removing it.
I will add on option to hide command progress bar.

Also a cosmetic issue. I don't like the black folder icons in the script tree. The light brown folder icon in the original builder was much more pleasing to the eye.

I am using black icons for simplicity of development, currently powered by MahApps.IconPacks and Google Material Icons.
(And in my personal opinion I prefer google material icons, it gives me UWP-like look)
I want to add an option to let user choose theme, but I am very clumsy at design and times are being invested on compatibility.
So please propose preferred icon set for buttons.

Hi ied206,

Meanwhile,
I just updated Macro Library (v43.r91) to avoid warning with DirDeleteE 

*

I am using black icons for simplicity of development

I agree. 

Let cosmetics be last, I am sure others (ChrisR, JFX, Galapo and public) can participate more about icons etc. when things work better and faster.
I have lots of ideas for many tests to be made....

Sadly this is the lowest free time I have through year...... Still I will try to catch as much as I can. 
Maybe Summer time coming to Galapo..

Build 20171118

EDIT
TestBuild taken down because of potential license issue.
Please use beta 2 or later, sorry for inconvenience.

This build focuses on optimization.

Changelog
- Delayed Logging
Too verbose logging slowed PEBakery's performance on HDD.
Build logs will be written to database at the point when a plugin's process is done.
It boosted lots of performance at the cost of losing on-the-fly logging.
For debugging, disable delayed logging on settings.

- Disable logging on Interface
If a section is run by button in interface, PEBakery generated verbose logs.
It slowed down performance in HDD, so logging on interface is disabled by default.
Notice logging on interface can be turned on again in settings.

- Command ProgressBar deleted
Command ProgressBar did not work well as I intended, and frequent UI updates takes up CPU and GPU.
For performance, command progressbar was deleted.

- AddVariables
Do not use cached section from memory when AddVariables is called.
Caching caused some variables to malfunction, notably %MuiLangAll%.
(Remove [MuiLang] section in script.project and build in last version of PEBakery reproduces this issue).
So in Build 20171118, AddVariables will always fetch values directly from file.

- Performance Boost
While WB082 took 14min to build Win10PESE x64, PEBakery 20171118 took only 7min on SSD.
On HDD, WB082 took 53min and PEBakery took 45min.


Caching Plugins to Memory
PEBakery preloades plugin files into memory at startup, solely for performance.
For test, I disabled memory plugin caching and it slowed build down to 24min.

It means if a plugin file is modified on-the-fly on build, PEBakery cannot recognize the change.
Is there any plugin uses modify-on-the-fly pattern in SE?
If not, PEBakery can use memory-cached plugins safety with very fast speed.

Current PEBakery's behavior:
- AddVariables : Always read from file
- Run/Exec/Loop : If the plugin is contained in Projects folder, the section will be fetched from memory. Elsewhere it will be read from file.
Ex) Run,%BaseDir%\Projects\Win10PESE\Build/Korean_IME.script,HelpTxt_TopicLink -> Fetched from memory
Ex) Run,%BaseDir%\Temp\Win10PESE\DirDeleteE_Folders.ini -> Read from file
- Ini/Text commands : Read/Write is done directly to/from file


Testing Gena
I tried to test Gena with PEBakery, but getting weird error.
When I try to build Gena, I always get Error 7 from saydin77 core in WB077.
This error pops up consistently on various Host OS, XP, 7, 10.
Is there any known bug related to CJK XP source?
(I used MSDN-provided Korean Windows XP source, Windows XP_x86_cd_x14-92431.iso.)

Hi ied206,

send me Windows XP_x86_cd_x14-92431.iso through pm

and send me log of failures so I can compare..

+
I guess CoprOrExpand added. ?

*

It means if a plugin file is modified on-the-fly on build, PEBakery cannot recognize the change.
Is there any plugin uses modify-on-the-fly pattern in SE?
If not, PEBakery can use memory-cached plugins safety with very fast speed.

There are some rare cases......


It is easy to update such plugins for PEBakery following your active development. 

Remember Reply 2:
http://theoven.org/index.php?topic=2271.msg24579#msg24579

We need a new PEBakery command
I feel
System,ReScan

ex:
System,ReScan,%ProjectDir%\Downloads
System,ReScan,%ProjectDir%\blabla.script

so when a plugin changes other plugin etc. , we can add line:

Code: [Select]

If,ExistVar,%EngineVersion%,System,ReScan,%ProjectDir%\blabla.script

or when we add a new plugin to project we can add same line
ex:
\Downloads\ComponentsY\PXE Support - Plugin Download
after Line 77 :

Code: [Select]

If,ExistVar,%EngineVersion%,System,ReScan,%BaseDir%\Projects\Yomi\ComponentsY\0_PXESupport.script
If,ExistVar,%EngineVersion%,System,ReScan,%ProjectDir%\ComponentsY\0_PXESupport.Link

or when we add a new set of plugins to a folder we can use:

Code: [Select]

System,ReScan,%ProjectDir%\Downloads

Definition:
System,ReScan --> add/remove/reload plugin(s) to project tree (+cache)
-->
So shortly, it is a command to add/remove/reload things to database (memory, sql etc.)

===> This way "memory plugin caching" always enabled with PEBakery 

--->
for all cases, it is faster than wb System,RESCANSCRIPTS which scans all project folder 

ps:
Such command is something we miss due to closed source ended development with wb 


*
I can not test for now, I have to go. Tonight or Tomorrrow....