A data pipeline commonly has three phases: extract, transform, and load (ETL). The Unix philosophy of “do one thing well” can apply here by making each program handle only one ETL phase; for example, a program that only extracts data. This strategy works well whether orchestration is done with a shell script or with a Python-based de facto tool such as Apache Airflow. In contrast to mixing ETL within a single program, Airflow and shell scripts using GNU Parallel can handle retries, concurrency, logging, etc. Data pipeline engineering is not a rare case. Still, other kinds of applications, such as GUI, TUI, and web apps, are rarely built on shell scripts; however, shell scripts may work behind the scenes for many apps. Moreover, “do one thing well” still applies, but the unit may be a function, module or service instead.

wsl --install installs Ubuntu by default. If I had to use WSL, I’d try this first too.

The hard part for me was dynamic scope, which I found it was explained nicely in The Common Lisp Condition System: Beyond Exception Handling with Control Flow Mechanisms. Since then I have begun to feel that dynamic scope in Bash is a feature, not limitation.

Cachy, Endeavour, Garuda, etc should be fine.

Since you are a coder, I suppose you using CLI should be fine. However, there are many other alternatives, which is fast and even based on Arch Linux.

In the 90s, QBasic (IDE) and Edit didn’t work on Slackware. So, I tried Pico, Vim, and Emacs. Now, I still use Emacs.

With some certain distros, it is easy.

With CachyOS and Mint, it is very easy.

Remark: I disabled secure boot.