Recently I've completed a number of small dynamic website projects, and I want to discuss my view on the way to choose to implement these sites.

This is to be read as an opinionated essay. YMMV.

SPA, a Single Page Application

Here are a few sites I completed recently:

• Rorschach™ Inkblot Coding
• Tangram Viewer
• Randall Train Status 

They all have in common that they are “Single Page Application” websites. What days does that even mean? In short, that means it’s a single HTML web page that uses JavaScript to implement a “full” web site by simulating several pages.

Let’s take an example: we want to create a small e-commerce site. We'll structure our site as different pages, each rendered by their own HTML file:

• http://example.org/mysite/index.html    -- the main entry point
• http://example.org/mysite/about.html    -- a page “about this company”
• http://example.org/mysite/catalog.html  -- the catalog browser
• http://example.org/mysite/product.html          -- a page detailing a single product
• http://example.org/mysite/purchase.html -- the purchase form

Instead an SPA web site uses a single file:

• http://example.org/mysite/index.html    -- the sole page of the web site

However, using JavaScript, we’ll change the content of the page to simulate showing the “about” page vs the “catalog” page, etc. One way to know which one to render is to encode the desired “page” using the hash portion of the URL:

• http://example.org/mysite/index.html
• http://example.org/mysite/index.html#about
• http://example.org/mysite/index.html#catalog
• http://example.org/mysite/index.html#product/{id}
• http://example.org/mysite/index.html#purchase

From the web browser point of view, it's always the same page being rendered (index.html) yet JavaScript is used to read the hash name and decide how to fill the page -- when the page loads, we use JavaScript to literally inject the content we want in the web page at runtime. If you were to read the “source” of the main index.html, you’d find it basically empty (or more exactly having just a single <div> container).

The mechanism above using the hash is called a “hash router”. There are other ways to do it, that's just the one I prefer.

It’s all about the frameworks

In this kind of web app, we rarely write “vanilla Javascript” (e.g. “code every by hand”). It’s just too tedious. Instead we use a framework to help us write the web page. There are a ton of frameworks out there, but essentially there are 4 that I consider the most popular and important:

• React JS, with code written in JavaScript or TypeScript.
• Angular JS.
• Vue JS, with code written in JavaScript or TypeScript.
• Flutter, with code written in Dart.

I believe that also more or less represents their level of popularity. My opinion on them, in a one line summary each:

• Flutter is my favourite one, but… We’ll discuss the “but” later.
• React JS. I started that way, and honestly I hated it. Ironically it’s only after switching to Flutter that I finally properly understood React.
• Vue JS. I tried this one for the sake of trying it. It’s said to be more approachable when starting than React. I can see why. Still I’d prefer React for the long run.
• Angular is a mastodon. It’s made to build large sites that scale. It’s a very complex framework, and it seems like it would scale very well. But for a single little SPA, it’s totally overkill IMHO.

Languages: JavaScript, TypeScript, and Dart

I’ll put this upfront to avoid repeating it below.

React, Vue, and Angular can equally be coded using JavaScript ES6, or TypeScript. In my option, JavaScript is always the wrong choice, no matter what the question is. Period. It’s an absolutely terrible language. Yes we can hand-code it without a compiler, and yes it runs directly in any web browser. Well, mostly. But it’s kind of flying without a net. Working with JavaScript is trying to admit that “hope is a strategy”. It’s not. Kids, just learn to say no to JavaScript.

TypeScript is the “sanity” version of JavaScript. It forces us to use a compiler (since it compiles to JavaScript after all), and that’s a good thing. It forces us to understand the types we’re using -- as the name of the language implies, it’s JavaScript with actual types, not just “whatever I can randomly put in a variable and hope it’s the right thing”.

The only drawback of TypeScript is that it’s essentially “JavaScript with lipstick”. Nothing prevents us from doing all the crappy things that JS allows us to do. However, TypeScript comes with a linter tool which tries to catch us on exactly the common mistakes that everybody makes. Don’t ignore the warnings, they are here to help us.

Flutter is coded in Dart. That’s its strength. That’s also its main hindrance. We have to be willing and make the jump to learn one more language just to write that small SPA thingy. That actually has dragged me down, and I avoided Flutter for a long time just because of that.

But… once I jumped into it, I realized that the TL;DR for Dart is “Java meets TypeScript”. Seriously. And if, like me, you find the Java syntax to be actually quite agreable, then Dart almost becomes “TypeScript has it should have been from the beginning”.

Flutter

I'm not going to try to give you a coding example of a Flutter app here - there are many examples out there already. Instead I'll try to explain why I liked it.

One thing you need to know upfront is that Flutter does not pretend to be a web framework. It is first and foremost a cross platform toolkit designed to create apps for mobile and the web (and likely desktop too).

As such, it will look foreign and weird to an HTML designer. We're not coding an HTML app with div and span and CSS. Not at all. We're coding screens using views (named widgets) and layout managers. Exactly like we would do in Java AWT or the Android view system.

--]

---

I’ve had a couple people ask me about upgrading to Windows 11, and I have a process to do an in-place update -- meaning it preserves all files and applications -- so I decided to write this down to have a clear instruction guide.

What this is Not

TL;DR: This is no shortcut TL;DR here.
If you’re not willing to read it all, I suggest you don’t bother.
If you don’t understand what you’re doing, I suggest you don’t bother.

If you’re not familiar and comfortable with changing BIOS options or changing an SSD, I suggest you don’t bother either.

I’ve updated 5 machines using the steps listed below so I’m confident I know what I’m doing. But I’m not claiming it’s an easy nor obvious nor quick process at all. It’s neither of these things.

If you need a lazy one-click do-it-all-for-me thingy, this guide is not for you. Try something like Flyoobe at your own risk -- I haven’t tried it so I’m not going to claim anything for or against it, nor even link to it. Just google it.

Also just to state the obvious, if you don’t want to put in the effort, then you don’t have to. You can either just keep running Windows 10, or get a new machine with Windows 11. After all, that’s what MS wants you to do.

High Level Summary

To do a successful update to Windows 11 in-place, we need the following steps:

1. Examine the system: CPU, TPM, disk mode.
2. Absolutely do a backup clone of the main drive and verify it.
3. Enable TPM if needed.
4. Perform an MBR to GPT disk conversion if needed.
5. Switch the BIOS to UEFI if needed.
6. Finally install Windows 11 in “unsupported” mode.

Let’s detail these steps below one by one.

Click here to continue reading...

---

The hype: “Rust is safer than C++”

At that point in my early ESP-RS experimentation, I want to address an important point about stability.  And to clarify, by “ESP-RS”, I mean the “std” mode built around the esp-idf-sys/svc/hal crates.

One of the arguments for Rust over C++ is a stronger memory management with clear memory ownership. Allegedly that's supposed to translate into better stability, avoiding the typical C/C++ crashes when memory references are improperly used.

My experience with Tangram rgen supported that.

The experience with ESP-RS here however does not match expectations. One little mishap in an ESP-IDF call… sneeze and the entire framework crashes.

For example, trying to initialize the EspMQTT client before the EspWifi has connected didn't just fail -- it rebooted the entire ESP32.

The reality is that ESP-RS “std” with the esp-idf-sys/svc/hal crates is essentially a ton of unsafe wrappers directly around the C ESP-IDF stack. And the ESP-IDF is generally quite touchy. For all its promises of type and memory safety, ESP-RS doesn't save us from that.

It’s all ESP-IDF and FreeRTOS

OTOH, I find the ESP-IDF to be generally predictable. Once things work, they do without surprise. I don’t have a problem with APIs blowing up predictably during development and giving me a chance to notice newbie mistakes early and taking care of them upfront.

Since I discovered the ESP32 platform 7 years ago, I preferred it over the Arduino platform. I’ve experimented with the “raw” ESP-IDF toolchain yet I rarely use it directly -- instead it’s been a backbone for something else like the Arduino platform, or the MicroPython/CircuitPython platform, and in the current case for the ESP-RS platform. Still, I’m more familiar with the ESP-IDF modules and don’t mind the mix and match, especially knowing that I can just directly call into ESP-IDF or FreeRTOS functions at any time.

Case in point in this case, we have FreeRTOS tasks for std::threads, ESP Wifi, ESP MQTT, etc., and supporting the ESP32 Camera is done directly as an ESP-IDF component right there in the ESP-RS project. It’s all working quite well. Still, we need to remind ourselves that for all that hype about how Rust is so magically superior to C/C++ (hype that I do not really buy into, in case it’s not already obvious), ESP-RS is just a bunch of fancy and sometimes way overkill wrappers around a C API with multi-core multi-thread complexity to deal with.

A Comparison Matrix

Here’s my guide to select the proper toolchain for my own projects:

• If performance does not matter, MicroPython is the easiest way to go (or CircuitPython on any modern Adafruit board).
• If performance and space matters, C++20 with the ESP-IDF 5 toolchain is the proper way to go.
• If performance and ease of programming matters, Rust with ESP-RS in “std” mode with the esp-idf-sys/svc/hal crates is a trade off, at the expense of a larger memory footprint.

The build size for a basic “blinky” project using ESP-IDF-HAL is around 700 kB (dev mode) on an ESP32-CAM. Add ESP Wifi and ESP MQTT, and that project’s build goes up to a whopping 1500 kB. That’s half the 3 MB flash partition available on an ESP32-CAM.

---

Some notes on how to perform basic Tasks, Threads, Synchronization functions using Rust with ESP-RS. And to clarify, by “ESP-RS”, I mean the “std” mode built around the esp-idf-sys/svc/hal crates.

This isn’t meant to be used as a canonical guide. Most of these are just a quick reminder for myself when I need something so that I don’t need to re-read the docs every time. So basically that only covers stuff I know I would need. I’m not pretending to be exhaustive here. And since I’m a newbie in Rust, I’m not pretending to be right either. Do your own due diligence by reading the official docs.

Tasks & Threads

The canonical example is in this ThreadSpawnConfiguration example:

https://github.com/esp-rs/esp-idf-hal/issues/228#issuecomment-1492483018

 

I haven’t seen any more official examples of that stuff.

The implementation is here:

https://github.com/esp-rs/esp-idf-hal/blob/master/src/task.rs 

This feels like a wrapper around the ESP-pthread API:

https://docs.espressif.com/projects/esp-idf/en/v4.2.2/esp32/api-reference/system/esp_pthread.html 

esp_pthread_set_cfg sets a “global” config that indicates how subsequent pthread_create() calls behave. That’s what ThreadSpawnConfiguration does.

Usage example:

ThreadSpawnConfiguration {
   name: Some("thread_name\0".as_bytes()),
   stack_size: 4096,
   priority: 15,
   pin_to_core: Some(core::Core1),
   ..Default::default()
}
.set()
.unwrap();

let thread_handle = std::thread::Builder::new()
.stack_size(stack_size)
.spawn(move || {
   //do stuff
})
.unwrap();
thread_handle.join().unwrap();

Click here to continue reading...

---

Since I’ve installed Rust for my exploratory ESP32 project twice already on different machines, I decided to actually write down my install instructions. There are a couple tricks I need to remember.

These steps are for MSYS2 on Windows. I used similar steps on my other box where I use my old fashioned Cygwin. Part of these instructions also work with PowerShell if you hate yourself that much.

My default shell is the “purple” MSYS2 MSYS one (the other shells are MinGW UCRT/Clang x86/x64; I can never remember why I would care about the difference so don’t ask me).

I already have Rust installed on this machine. I likely used “Rustup” following the default Rust install instructions.

I always customize ~/.bash_aliases, where I already added a line that sets up Rust and reminds me it’s available:

if [[ -d "$USERPROFILE/.cargo/bin" ]]; then
   RUST_PATH=$(cygpath "$USERPROFILE/.cargo/bin")
   export PATH="$PATH:$RUST_PATH"
   echo "Rust Cargo: installed."
fi

So first let’s check we have Rust and which version:

$ rustup -V
rustup 1.28.2 (e4f3ad6f8 2025-04-28)
info: The currently active `rustc` version is `rustc 1.86.0 (05f9846f8 2025-03-31)`

$ rustup show
Default host: x86_64-pc-windows-msvc
rustup home:  C:\Users\$USER\.rustup

Of note: there is no “esp” toolchain installed (yet).

OK now we need to install a bunch of stuff:

Click here to continue reading...

---

I looked at using Rust a few years ago on ESP32, but quite frankly the platform did not seem mature enough. It was really bleeding edge. Like razor sharp. I can barely stand the “better-than-thou” hype around Rust, so that was too much. Then I checked again last year when I did the SDB project and it still wasn’t looking stable enough for my needs.

This time it looks like I could at least do what I want: use Rust with ESP-RS on the ESP32-CAM and actually capture image frames. There’s now some good ESP Camera component circulating around with some tangible support:

• https://github.com/espressif/esp32-camera: An Espressif ESP32 Camera module with an OV2640 driver like the ESP32-CAM does have.
• https://github.com/jlocash/esp-camera-rs: The “original” example of how to wrap the esp-camera component into a Rust embedded project.
• https://github.com/jlocash/esp-camera-rs: A fork of that previous project.
• https://github.com/MathiasPius/esp-camera-rs: Yet another fork.
• https://github.com/Kezii/esp32cam_rs: A real example using the esp32-camera IDF component above, with an “espcam.rs” wrapper around it, and a wifi query.

To use this in an ESP-RS project, we need 3 changes:

First, we need a git checkout of the https://github.com/espressif/esp32-camera project.
In a serious project, that would be a git submodule, but for a throw-away test that can be just good ol’ boring git clone:

$ mkdir components
$ pushd components
$ git clone https://github.com/espressif/esp32-camera.git

In the main Cargo.toml, we need the esp-idf-sys crate and we need to use (import?) that esp32-camera component:

$ cargo add esp-idf-sys
$ vim Cargo.toml
[[package.metadata.esp-idf-sys.extra_components]]
component_dirs = "components/esp32-camera"
bindings_header = "components/bindings.h"
bindings_module = "camera"

Now the first I compiled this, it failed at runtime in the esp_camera::init:

I (657) cam_hal: Allocating 384000 Byte frame buffer in PSRAM
E (657) cam_hal: cam_dma_config(509): frame buffer malloc failed

That sounds like “esp_psram” is not enabled on the project. That’s done in the sdkconfig.defaults file:

CONFIG_ESP32_SPIRAM_SUPPORT=y

My own “webcam sample” for the ESP32-CAM is located here:

https://github.com/ralfoide/arduino/tree/main/ESP32-CAM/Rust/esp-rs-std-webcam

---

At Randall, I’ll soon have the Distant Signal panel installed. This connects to the local wifi and gets the turnout state from the local MQTT broker. I want to track the “health” of that wifi connection, as that has been an issue in the past. The simplest way is to reuse my existing Google Analytics dashboards, thus I want the CircuitPython script on the ESP32 to send pings to GA4.

The goal is to measure the “uptime” of the display. The display is working when it is able to receive messages from the MQTT server. It can receive them when it can connect to the wifi. Thus we want to track the wifi state, or some kind of proxy for it.

Since this is CircuitPython, we have AdaFruit libraries to already deal with JSON and network.

So really “all” I need is something similar to the Analytics class in Conductor 2:

• Queue stats to be sent.
• Stats are sent as JSON payloads.
• POSTs can fail, in which case they need to be retried with a backoff delay.
• ⇒ Obviously a “wifi lost” event would have no wifi to send the stat.
• Do I need to put a real date/timestamp in the events?
• ⇒ These ESP32 don’t have a “date” clock, so it’s nice if I don’t have to.
• There’s an NTP library which I’ve used on the LitterTimer experiment but it makes the logic more complicated since we need to expect that the device may not have wifi.

So what do we need to do custom pings to Google Analytics 4?

• A “GA4_CLIENT_ID”, typically configured in settings.toml
• A “GA4_MEASUREMENT_ID”, also configured in settings.toml
• A “GA4_API_SECRET”, also configured in settings.toml
• The feature is disabled if either of these settings is missing.
• Placing them in setting.toml avoids leaking these in any GIT source, and it makes it easier to configure a new panel.
• See below on where to find these.
• A base URL:
• https://www.google-analytics.com/debug/mp/collect    -- debugging version
• https://www.google-analytics.com/mp/collect          -- real pings version
• The POST URL is:
• %(base)s?api_secret=%(GA4_API_SECRET)s&measurement_id=%(GA4_MEASUREMENT_ID)s
• POST mime type: text/plain
• Payload is JSON, and is used as such in Conductor:

{
        'client_id': GA4_CLIENT_ID,
        'events': [ {
                'name': event_action ,
                'params': {
                        'items': [ ] ,
                        'value' : value_int ,
                        'currency' : USD
                }
        } ]
}

• The AdaFruit “requests” library is the simplest way to send a POST request. It accepts either a JSON (using a Python dictionary) or a string payload.
• When POSTing to the debug URL, print the response.status_code and the response.content.decode() to get valuable information on why requests fail.
• When POSTing to the real ping version, the GA server replies with 204 (Success w/ No Content) and a complete lack of response body.

Click here to continue reading...

---

Well, that escalated quickly.

I’ve been battling with this in Distant Signal:

@@ MQTT: Failed with  pystack exhausted
Traceback (most recent call last):
 File "code.py", line 498, in <module>
 File "code.py", line 354, in _mqtt_loop
 File "adafruit_minimqtt/adafruit_minimqtt.py", line 956, in loop
 File "adafruit_minimqtt/adafruit_minimqtt.py", line 1027, in _wait_for_msg
 File "adafruit_minimqtt/adafruit_minimqtt.py", line 381, in _handle_on_message
 File "code.py", line 328, in _mqtt_on_message
 File "script_loader.py", line 27, in newScript
 File "script_parser.py", line 316, in parseJson
 File "script_parser.py", line 289, in _parseGroup
 File "script_parser.py", line 214, in _parseInstructions
 File "script_parser.py", line 278, in _parseInstructions
 File "adafruit_display_text/label.py", line 88, in __init__
 File "adafruit_display_text/__init__.py", line 273, in __init__
 File "adafruit_display_text/__init__.py", line 294, in _get_ascent_descent
RuntimeError: pystack exhausted

It turns out that CircuitPython is built with a max stack depth of 15-16 calls (*), at least for this version of the MatrixPortal S3.

It’s entirely possible to rebuild it with a different stack depth, if one is inclined to do so.

The stack size is controlled by CIRCUITPY_PYSTACK_SIZE which is defined globally here:
https://github.com/adafruit/circuitpython/blob/HEAD/py/circuitpy_mpconfig.h#L455
and is also architecture dependent -- e.g. the SAM arch overrides the default.

Documentation on CIRCUITPY_PYSTACK_SIZE is here:
https://github.com/adafruit/circuitpython/blob/HEAD/docs/environment.rst#circuitpy_pystack_size 

(*) As can be seen above, the “stack size” is not really a number of calls but really a size in bytes. Thus I expect calls with more arguments will exhaust the stack faster, or whatever else needs to be pushed on the stack for each call.

At runtime, the “ustack” module can give the current max and used size:
https://docs.circuitpython.org/en/latest/shared-bindings/ustack/index.html 

Well, not on the MatrixPortal S3 port at least.

So how does one solve that? Since I do not want to rebuild CircuitPython, it means I can just amend the code to reduce stack usage.

For example the main starts with this:

if __name__ == "__main__":
   setup()
   loop()

This conveniently mimics an Arduino sketch setup. But we don’t need that “loop()” function. Just place everything under the “if main” and save one call stack entry.

The MQTT error above happened when a message was received -- I’d parse the script right there in the MQTT handler. Instead, what I do is save the script in a temporary global variable, and process it from the main loop, after the MQTT handler has returned. That way we’re “saving” about 5 calls depth on the stack.

The parser is about 5 calls deep. I could probably save one call in there, if I really have to.

Another easy strategy is to create lambdas, and queue them for processing in the main loop.

That’s more or less what I do with the delayed processing of the script, except since there’s only one thing to do, I keep the string reference around rather than wrap the processing in a lambda. Delayed processing can quickly become hard to follow, it’s harder to understand and debug.

---

Over there in the trains/electronics blog, I have a little write up about the HUB75 Protocol for LED Matrix Displays, as used in the Distant Signal project. Go read it there.

---

I’m still in the process of updating RTAC to work properly on Android 13.

RTAC uses a number of APIs which have been deprecated since Android 10. Since it was running on 2017 hardware with Android 9, that wasn’t much of a problem. Now I want to move to newer hardware that runs Android 13 so I need to deal with it.

WifiLock:

• The WifiManager.WifiLock API is still present and usable.
• I’ve added a handler class for it in TCM.
• However it’s not absolutely clear what it really provides.

WifiManager#enableNetwork is reserved to “Device Owners” and system apps.

• It seems that ConnectivityManager#requestNetwork could be an adequate replacement?
• A first try of the ConnectivityManager#requestNetwork API wasn’t conclusive.
• The app suddenly displayed a model dialog “connecting to device” with a spinner, which has the undesirable side-effect of pausing the main TCM activity.
• Configuring the request for a known SSID did… nothing. It stayed on that spinner and did not switch to that wifi network.
• Configuring the request for a known SSID and its WPA password did something weird where the wifi list showed a second entry of the SSID “for this particular app”.
• So far that didn’t work as expected. My goal is to switch the wifi to an already known SSID that has already been configured in the Android wifi setting and already set up with password et al.

To be continued.

---