Handling UTF-8 strings in plugins: workarounds, hacks?

The Classic WinBuilder is an aged programme by now, but still in use for current projects.
(I know, there are successors, but as I did not test them I cannot say anything about them.)
Even if the ML module has polished WB quite up, there are limitations plugin authors sometimes encounter.

For instance, more and more applications (like Mozilla Firefox) deal with utf-8 data.
As long as we're using external scripts doing stuff we're safe, but importing it in WB is difficult:
E.g. instead of reading in the string "Wikipédia (fr)" we'll get something like "Wikipédia (fr)"—let alone strings like "Википедия".

One workaround would be to largely avoid the detour via WB variables and to realise these information flows externally.
I'm looking into a case (i.e. unreleased plugin version) right now whether it'll be feasible...

How do you solve the problem?
Or will it vanish with a WB successor?

Hello
why do not you just take the LiveSystempro Builder?
He can handle all popular formats like Ansi, Unicode and Utf8.
For this he is actively developed further. The program has been developed for more than 10 years, and therefore no mayflies.
Many developers have come and gone in the last 14 years.

Kare
--

RegFileImport

why do not you just take the LiveSystempro Builder?
He can handle all popular formats like Ansi, Unicode and Utf8.

I'd rather check out a WB replacement which is syntax compliant, i.e. that offers the full command set including ML.
As far as I know, this isn't the case with the—syntactically simplified—Live System Pro Builder, so that's no realistic option.

RegFileImport—hm. That's somehow like the Inifile workaround.
Although I can imagine suitable scenarios, I am not fully convinced by this "hack"...

the Other builders (LSP, Pebakery, wimbuilder2) seem to support the most common encoding formats: unicode, utf-8, utf-8 with bom and Ansi 
On its side, WB does not support unicode, but seems to support UTF-8.

With the file encoded in Utf-8, TXTReplace or TXTAddLine seems to do the job 

Code: [Select]

TXTReplace,%Target_Prog%\%ProgramFolder%\distribution\policies.json,"#$qDisablePocket#$q: true#$c","#$qDisablePocket#$q: false#$c"
or TXTAddLine,%Target_Prog%\%ProgramFolder%\distribution\policies.json,xxxxxx,Append

In running Firefox Quantum and using policies.json encoded in UTF-8, "Wikipédia (fr)" is well recognized and properly removed from the list of search engines

I don't know what my tests were last night but they weren't good.
I edited my previous post.

the Other builders (LSP, Pebakery, wimbuilder2) seem to support the most common encoding formats: unicode, utf-8, utf-8 with bom and Ansi 

That seems like a good reason for switching to one of them... PEBakery perhaps?
Are there any difficulties concerning syntax compatibility (incl. external commands/ML)?

On its side, WB does not support unicode, but seems to support UTF-8.

With the file encoded in Utf-8, TXTReplace or TXTAddLine seems to do the job 

Code: [Select]

TXTReplace,%Target_Prog%\%ProgramFolder%\distribution\policies.json,"#$qDisablePocket#$q: true#$c","#$qDisablePocket#$q: false#$c"
or TXTAddLine,%Target_Prog%\%ProgramFolder%\distribution\policies.json,xxxxxx,Append

I recently added one TxtReplace line to handle the XPE audio topic, for the .cfg file (untested yet).

For files like policies.json I use the parser jq* (in a cmd session), so if I write the output of jq into a file, it will be utf-8 encoded.
The current problem I'm facing is getting a utf-8 encoded string into WB, IniRead etc. won't do. (Cannot tell another method.)
That's why I thought of circumventing the problem by entirely generating the intended parsing command externally...


* Parsing example:

Code: [Select]

<policies.json jq ".policies.DisablePocket=false" >policies.new && copy/y policies.new policies.json

I use latest develop branch version of pebakery for win10pese. It has continuous development on the develop branch. Master branch seems to be largely ignored by the developer.

Hi sandy

Master branch seems to be largely ignored by the developer.

odd, and incorrect thing to say

when ied206 has completed his development goals for the next release - beta6, that would be updated, as would be perfectly normal
and hopefully, after beta6 we will go to Release 1.0

I worked around the encoding problem by remodeling the code and made sure that the processing takes place independently from WinBuilder.
It's a major drawback of WinBuilder that it can't handle UTF-8 properly. I will upload the current plugin in the corresponding thread soon.

@sandy & APT:
Out of curiosity, did you use one of my recent plugins—Firefox Quantum ESR 68 (x64) or Notepad++ x64—with PEBakery yet?

Was not trying to be critical about pebakery. In fact I see no reason not to use it for the currently active projects.

Hi Mikka

Out of curiosity, did you use one of my recent plugins—Firefox Quantum ESR 68 (x64) or Notepad++ x64—with PEBakery yet?

Yes, but not extensively, only to check they both run up to working apps in a VM on the x64 build I tested on, with latest nightly PEB

both have minor syntax errors, but nothing to stop build

nothing to do with PEB, but is it ness for Firefox to download source file every run? - quite annoying when running test builds

anyway, thanks for the plugin updates 
APT

ps I think we are wandering a bit off topic here!