How to get code/library used for these? Publication references observable notebook, but it’s empty with 3 comments. Or is it a paywall?

I wonder if there is a way to somehow combine datasets to fill in the gaps.

It would’ve been zero fun and same amount of success. Basically, creating a new taxonomy database while a lot of them already exist. I didn’t expect there are so many taxonomy databases, almost all of them being backed by scientific organizations and being freely accessible and downloadable. Other areas (books, movies, history) are not even close to this diversity of data sources.

I didn’t find your code though, was wondering how you have written this

Apart from Gephi Commander (already on Github), which is used for generating PNG tiles when you already have x and y for every taxon, there is also a CLI tool to build Voronoi (assign x,y) and another CLI tool to split those points across zoom levels and PBF vector tiles. Neo4j as a database and Powershell to bring all of this to life.

Oof, I didn’t like this at all!

Not a fan either. There was another tool looking similar to Voronoi, made by a person working in scientific organization, but I can’t find it right now… There is a lot of interesting on this topic.

I will try to check if there are any discrepancies between visualization and ITIS db, and between ITIS db and other taxonomic sources.

Hymenoptera in ITIS has two direct children: Apocrita and Symphyta with 1904 and 39 genera each.

Hymenoptera in insecta.pro has three direct children (third is a dead end, will ignore): Apocrita and Symphyta with 6768 and 153 genera.

Apocrita is ~45 times larger than Symphyta in both databases, ITIS is representative in this case. In visualisation each clade gets as much space as it needs to fit all its leaf nodes (taxa without children). Apocrita probably got ~45 times more space than Symphyta, which is what I’d expect.

Also, I’ve tried ti find Symphyta in lifemap, but NCBI page (LifeMap is based on NCBI) for Hymenoptera has a comment about Symphyta being a paraphyletic group and therefore NCBI doesn’t have this suborder at all.

There are ~100 species in Microgaster clade and ~62 in Symphyta, not a big difference, they got comparable amount of space, I think it is also as expected.

in the ITIS one they are all at least displayed close together (although not within one rank). But in the GBIF visualization, e.g. Apidae and Halictidae are at totally different ends

As LLM would’ve said, “you’ve got to the heart of how Voronoi Treemaps work”. In GBIF they do not keep track of intermediate taxa at all, therefore Apidae and Halictidae in their system are equally related to Hymenoptera, there’s nothing else to group them together. While in ITIS, they do have a lot of taxa with intermediate rank, including Aculeata and Apoidea. These two additional links prevented spreading of Apidae and Halictidae to the opposite ends of Hymenoptera as it is in GBIF. I’ve decided to color points only by six main ranks, and I’ve made zoom to jump between these ranks, therefore intermediate polygons are somewhat obscure, but they already did their job, and you’ve noticed that, cool! When I will continue to work on these maps, probably I will not consider using GBIF as data source because of this exact detail you’ve mentioned - some branches can be placed further from each other than you expect.

I feel like there are some locations where a huge number of points link to a single origin

Yes, I also was looking at them, usually it’s artificial groups like “unclassified Lepidoptera” with a lot of taxa which doesn’t even have a name, they have a code instead, like “BOLD:ACO0165”. You can find such groups in GBIF as well, e.g. in Lepidoptera there is a huge ball in the center with a lot of unnamed taxa squeezed together. This is somewhat similar. I think next time I will nuke them because they are not interesting, take a lot of space and don’t add up to the structure and readability.

Also, you can checkout this foamtree demo which is also a treemap, but it displays polygons instead of points, and you have to move through all the intermediate taxa by double clicking to get anywhere. To the right you can switch to Metazoa. They don’t use space as efficiently, Korarchaeota has a single known specie but got a huge polygon anyway. I am not related to this foamtree, they’re trying to sell visualisation library and to showcase it they’ve made a demo with taxonomy tree.

You have some level of expertise and with this visualisation you notice how underlying database leans towards some filling strategies, you see how maintainers put more work into one direction than another. This is quite cool.

Thank you for the detailed analysis! Can you check out pngs in wikimedia commons category I’ve linked in the post? There are some for GBIF, ITIS and NCBI. Some of them are more readable than online tool.

For visualisation, I’ve used whole database each time, so what you see is determined by what is in database. If you can’t find some genera or species in this tool, probably they are not present in the db.  Usually these databases are backed up by some scientific organisations, which can be focused on specific areas, probably it can affect level of detail.  E.g. I know some services are specialised in marine species. Catalogue of life doesn’t have any non-avian dinosaurs, etc. for ITIS, I’ve noticed they have a very low amount of Fungi. I don’t know why. GBIF is trying to get data from all sources they can find, probably that’s the reason why size of branches is close to what you expect. Also, they are the biggest, but don’t keep track of intermediate ranks — subphylums, subkingdoms etc.

I wonder if NCBI is representative?

These two files are from Commons category:

GBIF Tree of Life - colored by class, force-directed.png

NCBI Tree of Life - Voronoi Treemap - 4 Order.png

Check out “ITIS Tree of Life” link in the post, largest group in Arthopoda is Insect

Poor Jimmy, why is he banned

This title and a photo of a smiling women are very informative, thank you.

For Bash - try blesh, it will enable some of common controls by default, and probably you will be able to manually enable other shortcuts. PSReadLine is calling xclip each time for copy/paste action for clipboard sync, probably it will be possible with Blesh too.

I’m fine, I got what I wanted. Apparently, it isn’t un-feature of Linux, it’s un-feature of ReadLine which is shipped with Bash.

I’ve got what I wanted, but I’m a powershell user. For Bash, blesh looks very promising - it’s functionally same component as PSReadLine which makes all this stuff possible in pwsh.

one of many un-features in Linux

What exactly? Shift-selection is already possible with Blesh. I think I’ve seen scripts for synchronizing buffer with clipboard. And everything else is a matter of redefining existing shortcuts.

I’ve heard about Linux being highly customizable and decentralized OS, and suddenly I can’t define my own shortcuts because there is a list of un-features?

I don’t care about Vi and Emacs, I already have my workflow and I’m trying to transfer it to Linux. When I will succeed, then (maybe) I will spend some time to explore other ways of interacting with terminal. Otherwise, it’s not freedom, it’s becoming a victim of OS.

I was able to test out what you’re looking for on macos and its default out of the box

This is so cool. Unfortunately, macos isn’t an option for me and never had been (for common reasons why people usually switch to linux).

I’ve got what I wanted with wezterm + powershell. I can edit my commands the same way I edit any text anywhere in the system, both in Windows and Linux, and I can copy-paste back and forth between terminal and any other app. This is awesome. This is freedom. This is UX done right.

I will paste below some observations I’ve made.

Possible solutions for Bash

• https://github.com/jirutka/zsh-shift-select - plugin for zsh as pointed out by @[email protected] and several other sources;
• https://github.com/akinomyoga/ble.sh - ReadLine replacement for Bash (most promising);
• https://github.com/magenta404/natural-selection - plugin for FishShell;
• https://www.reddit.com/r/pop_os/comments/p0jjno/how_to_use_shift_arrow_keys_to_select_text_in/ - most useful source on this topic.

Blesh

https://github.com/akinomyoga/ble.sh/wiki/Manual-§4-Editing

• Super simple installation.
• Home/End - Jump to start/end as expected. ✅
• Ctrl+Backspace removes left char instead of left word. ❌
• Ctrl+Delete removes next word as it should. ✅
• Shift+arrows - char-wise text selection ✅
• Shift+Ctrl+arrows - word-wise text selection ✅
• Shift+Home/End don’t do anything. ❌
• Backspace/Delete: When smth is selected they delete it. ✅
• Copy/Paste/Cut: ❌
  • It’s Alt+W/Ctrl+Y/Ctrl+W instead of Ctrl+C/Ctrl+V/Ctrl+X.
  • All work with selection as expected.
  • All work with internal buffer instead of system clipboard.
• System clipboard:
  • Can’t copy selection to clipboard, can’t paste clipboard into selection.
  • Ctrl+Shift+C/V work as they do in vanilla bash: copy what is selected with mouse to clipboard, paste from clipboard.
• Ctrl+C prints current command and starts new one like in vanilla bash.

zsh-shift-select

• Stated to have best compatibiliy with Alacritty.
  • Alacritty requires Cargo (440MB).
    • cannot install package alacritty 0.16.1, it requires rustc 1.85.0 or newer, while the currently active rustc version is 1.75.0 Fail. Will use Gnome Terminal instead.
• Needs zsh, super simple installation.
  • Zsh should be default shell, gnome-shell crashed with SIGSEGV.
  • Plugin itself has simple installation, just git clone .zsh file and source it in .zshrc
• Ctrl+arrows - prints CD instead of moving word-wise ❌
• Ctrl+Backspace, Ctrl+Delete - are not deleting left/right word ❌
• Home/End - Jump to start/end as expected. ✅
• Shift+Left/Right - char-wise text selection ✅
• Shift+Ctrl+arrows - word-wise text selection ✅
• Shift+Home/End don’t do anything. ❌
• Shift+Up/Down - Select one line up/down ✅
• Backspace/Delete - When smth is selected - delete it. ✅
• Copy/Paste/Cut: ❌
  • Documented as Alt+W/Ctrl+Y/Ctrl+W instead of Ctrl+C/Ctrl+V/Ctrl+X.
  • Alt+W/Ctrl+Y work as copy/paste.
  • Ctrl+W removes previous word instead of Cut selection.
  • Work with internal buffer instead of system clipboard.
• System clipboard:
  • Ctrl+Shift+C/V work as they do in blesh and vanilla bash.
  • Can’t copy selection to clipboard, can’t paste clipboard into selection.
• Ctrl+C prints current command and starts new one like in vanilla bash.

wezterm + Powershell

PSReadLine starts with EditMode = Emacs by default.

Set-PSReadLineOption -EditMode Windows Fixes Ctrl+arrows, Ctrl+backspace, Shift+Ctrl+arrows.

Set-PSReadLineKeyHandler -Chord Ctrl+Delete -Function KillWord - Fixes Ctrl+Delete.

Set-PSReadLineKeyHandler -Chord Ctrl+o -Function AddLine - allows Ctrl+o instead of Shift+Enter to create a new line without trying to execute. Shift+Enter is not possible in Linux.

Reassigning Shift+Home/End in Gnome Terminal from scrolling viewport to something else is a rabbit hole, so I switched to wezterm, which fixed Shift+Home/End, and apparently also fixed a bug of Shift+arrows printing D;D;D; instead of selecting. But broke Shift+Ctrl+arrows. But you can fix it back by disabling this assignment in lua config.

Ctrl+C/V/X work fine, but without system clipboard synchronization. To fix it, install xclip. If it makes terminal freeze on Ctrl+C/X, update PSReadLine module.

• Ctrl+arrows ✅
• Ctrl+Backspace, Ctrl+Delete ✅
• Home/End ✅
• Shift+Left/Right ✅
• Shift+Ctrl+arrows ✅
• Shift+Home/End ✅
• Shift+Up/Down ❌
• Shift+Enter - Ctrl+o instead ✅
• Ctrl+C,Ctrl+V,Ctrl+X - Flawless ✅

Windows + conhost + Powershell Core

PSReadLine starts with EditMode = Windows by default.

• Ctrl+arrows ✅
• Ctrl+Backspace, Ctrl+Delete ✅
• Home/End ✅
• Shift+Left/Right ✅
• Shift+Ctrl+arrows ✅
• Shift+Home/End ✅
• Shift+Up/Down ❌
• Shift+Enter ✅
• Esc - clear current command ✅
• Ctrl+C,Ctrl+V,Ctrl+X - Flawless, all with system clipboard. ✅

Macintosh has nothing to do with it. Maybe I want to grow selection to the left with E and to the right with R - they are not Macintosh keys, but still I will have a hard time trying to get what I want.

Do you mean this whole concept of growing selection from the same cursor you type with and performing operations on this selected area as a single entity is a Macintosh-way, originated in GUI? And its foreign to terminals and terminals developed a different way of editing text and you propose learning this native text-editing without using foreign techniques like cursor-based text selection… This makes sense. And sure it’s possible to be effective in terminal while using it traditional way. If this is what you mean, now I understand.

I made a glimpse on the world where this shift-selection doesn’t exist and got excited about this feature even more, and I even think it’s genius.

System-wide clipboard is probably not the “traditional way” either, and it doesn’t work quite well in Linux terminals too.

A couple of years ago I invested some time into Vim and it was a pleasant experience, but it was detached from all other experiences I had on my PC. Mentally switching between different text editing modes is disgusting, I hate it so much. I don’t want a new one. I’m fine with the one I have.

Linux and its terminals are meaningful only as long as they do what I want them to do. I don’t care if some of my activities are “not Linux-way”.

In Gnome Terminal: Ctrl+arrows work, Ctrl+delete work, home/end work, Ctrl+backspace — not, and there is no text selection, of course. So, something is working as expected without config at all, and for what isn’t working, config will not help.

I feel like you’re just defending the final state you observe. E.g. why do you hate shift+arrows for text selection but not arrows themselves for moving a cursor? And how selecting text with a mouse is better? I find it even funny to think one of these techniques is cursed and other are blessed. Never I could imagine selecting text with Shift can encounter such opposition.

It’s nice to see you think of it as of movement towards consistency. I also look at it this way.

But what is it about Ctrl? Text editing is historically the main task of computers, and Ctrl is the main “modifier” key. To me it seems fair it’s dedicated for some text editing shortcuts. Probably they are consistent since 1980’s.

What terminal emulator are you using where ctrl-c copies instead of sending SIGINT when text is selected?

This is what I experienced in conhost.exe (legacy windows console experience, predecessor of Windows Terminal) + Powershell. In windows terminal it works this way too. This is why I suspect it’s related not to terminal itself (conhost.exe/wt.exe/gnome terminal etc), and not to specific shell (bash/powershell), but to an extension for shell (ReadLine,PSReadLine).

As for various types of buffers and clipboards, I always felt like one system-wide clipboard with clipboard history is enough. When I cut something from terminal, quite often I paste it into another app, and not back to terminal.

Than you for the link, it looks like a very close match, even though I don’t quite understand how it is related to eMacs.

About what I’m trying to do being stupid — very interesting. What about learning vim only to notice you can’t use this skill in 70% text editing areas you interact with, and searching for firefox vi extension afterwards and trying to apply .vimconfig to it, is it stupid? Probably not. But trying to select, copy and paste text in a text-based program — that is stupid. I got it.