Inserting Python Command’s Output in VIM

1) r!

:r! python test.py

Let’s say test.py is a file with print(123) inside it.

This will run the existing file with Python. It will enter the output in Vim. Here is the breakdown:

  • : (command line)
  • r reads
  • ! executes linux command
  • python runs python with test.py file

2) !! (Bang Bang)

:.! python test.py

Alternatively, if you type !! in normal mode, this will type ! with a dot in command line like above. The same outcome will be achieved. Here is the breakdown.

  • . enter to current line
  • ! execute system command
  • python runs Python
  • % feeds current file to Python

3) Feed Python with a Quick Command

:.! python <<< "print(123)"

Previous outcome can be matched with Python command execution style mentioned here.

  • <<< feeds Python code from system’s temporary file system to Python Interpreter.
  • “” inside the quotes our content for Python command is formed
  • print(123) actual Python command itself
This is incredible as so many systems are used on the fly and in harmony. Vim, Linux OS and Python are utilized in two words and a few characters and the program’s result is written in the same file.

4) Feed Python with the Existing File

:.! python %

Finally, you can feed the existing Vim file to the Python directly and insert the output from Python on the cursor’s line. For this task, you need to feed % to Python representing the whole current file.

  • . enter to current line
  • ! execute system command
  • python runs Python
  • % feeds current file to Python
This is really next level scripting pipeline. I always appreciated Vim’s intuitive style for writing code and human language. When mix n matched with Python or Linux commands it gets even more impressive.

Fixing Pacman Errors When Updating Arch Linux After A While

There is a specific error you might get when updating/upgrading the Arch Linux OS after a long break (or sometimes even after a short break if you’re unlucky with the intervals) via sudo pacman -Syu. No worries though. Linux in general and Arch Linux specifically are incredibly robust and repairable systems so there are many different ways to address the issue. Table of Contents
  • Introduction and Pacman Errors
    1. Updating the Archlinux Keyring
    2. Updating the Pacman Mirrorlist
    3. Removing the Pacman Cache Files
    4. Internet restriction and VPN or Hotspot Solutions
    5. Compiling and Installing static-pacman
    6. Booting Via Live Arch Installation USB & Chroot
  • References

The errors thrown by the Arch Linux and its package manager Pacman look something like this:

error: openssl: signature from “Pierre Schmitz <pierre@archlinux.org>” is marginal trust
:: File /var/cache/pacman/pkg/openssl-3.0.7-4-x86_64.pkg.tar.zst is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n] Y
error: failed to commit transaction (invalid or corrupted package)
Errors occurred, no packages were upgraded.

Starting from the least complicated and the least invasive, here are some solutions to these pacman errors.

Disclaimer: This tutorial is created for journaling & educational purposes only. The author cannot be responsible for any damages due to the guidance provided in this post. Please proceed at your own risk and always keep backups of your files.

1. Update the Archlinux Keyring first

While partial updates aren’t recommended as they can mess up the dependencies, SSL / TSL encryption mechanisms and the sync between packages and the package manager, you can always get away with updating the Archlinux-keyring first. If you haven’t run any updates/upgrades in a while, the update operation might be interrupted due to out of date Archlinux keyring libraries.

You can simply do this by running this following command:

sudo pacman -S archlinux-keyring

After that the keyring will be upgraded all your problems regarding the package signatures and marginal trust might potentially go away. You can try to proceed with a full system upgrade with the following command:

sudo pacman -Syu

If everything goes well, issue is sorted out and you don’t have to follow the following suggestions.

Note about openssl: Openssl package is a dependency of the GNU Core Utilities package: coreutils. In this specific case due to out-of-date archlinux-keyring package, the required version of openssl package can’t be found / appears corrupted and as a result core database cannot be retrieved.

Another way to fix broken keyrings is to update the keyring with these commands:

pacman -S archlinux-keyring
pacman-key --init
pacman-key --populate archlinux
pacman-key --refresh.

If everything goes well, issue is sorted out and you don’t have to follow the following suggestions.

(According to Arch Wiki: this is something you want to do in the chroot of your installation (arch-chroot /mnt))

This section is inspired by Nan Xiao’s post about How to update Archlinux keyring in old systems.

2. Update the Pacman Mirrorlist

You can also try updating the pacman mirrorlist by simply adding anything in the official Arch mirrorlist in the link below to the file in your system in : /etc/pacman.d/mirrorlist.

Here is the mirrorlist link: Arch Pacman Mirrorlist.

After updating the Pacman mirrorlist, you may want to give it a try by attempting the update/upgrade command again.

If it doesn’t work you can try the next step and try this command again.

sudo pacman -Syu

Note about openssl: Openssl package is a dependency of the GNU Core Utilities package: coreutils. In this specific case due to out-of-date archlinux-keyring package, the required version of openssl package can’t be found / appears corrupted and as a result core database cannot be retrieved.

This post is inspired by Nan Xiao’s How to update Archlinux keyring.

3. Removing Pacman cache files

Every time you update your system pacman keeps cache files of the old packages in case there is a need to downgrade. These files take lots of space cumulatively and if they have the wrong version this might be confusing the pacman’s update mechanism as well. Cache files are kept under

/var/cache/pacman/pkg &

/var/lib/pacman.

 

Cache files can be removed by running the following command:

sudo pacman -Sc

You’ll be prompted twice to delete the cache files.

You can then try to update the system with pacman again by running sudo pacman -Syu. There is a way to completely remove ALL cache files including other than other packages as well.

sudo pacman -Scc

Then you will be prompted twice and one of the prompts will have no (N) as default choice so you might want to make sure to enter y (for yes).

Cache directory: /var/cache/pacman/pkg/
:: Do you want to remove ALL files from cache? [y/N]

Database directory: /var/lib/pacman/
:: Do you want to remove unused repositories? [Y/n]

sudo pacman -Syu

4. Internet Restriction & VPN usage

In some rare cases there can be access restriction if you are using restricted internet. Server will return a 403 access denied error. In those cases you know it’s something to do with the internet connection most likely. You will see errors similar to below:

error: failed retrieving file ‘core.db’ from archmirror.it : The requested URL returned error: 403
error: failed retrieving file ‘extra.db’ from archmirror.it : The requested URL returned error: 403
error: failed retrieving file ‘community.db’ from archmirror.it : The requested URL returned error: 403
warning: too many errors from archmirror.it, skipping for the remainder of this transaction

In such cases temporarily connecting to a VPN or a Hotspot and trying to run a system update can also be helpful.

5. Compiling a Static Pacman (pacman-static) from AUR

grep -R  --ignore-case "IMPORT" /

Here is the AUR repository for pacman-static where you can find the git clone URL of pacman-static. Then following this tutorial you can compile the pacman-static:

Once pacman-static is installed, you will be able to update the openssl package or other crucial system packages that need to be repaired. In many cases it’s the openssl package which can be updated / installed with the following command.

sudo pacman-static -S openssl

You can also see the version of your openssl with the following command.

pacman -Q openssl

In the case of partial updates, one important package gets upgraded while others don’t. Since those other system packages depend on the correct version of the upgraded package they will stop functioning properly and it will break the whole system.

Here is the Arch Wiki entry for using pacman-static.

5. Firing Up the Live USB (Arch ISO)

If all fails, it’s still fairly easy to boot with a live Arch Installation USB. It’s also a good idea to always have a Live USB ready for such circumstances. Then you can chroot and install necessary packages to your local installation to solve the more complex issues. It’s probably also a good idea to back up what’s needed at this step while you still can. Here is the command to chroot:

arch-chroot /mnt

Fixing Broken Pacman

If partial upgrades, broken libraries, missing SSL TSL encryption is causing bigger issues, you can follow this procedure:

  • Boot through Arch ISO USB
  • Connect to internet using iwctl
  • mount the broken system on /mnt
  • use pacman –sysroot /mnt … to reinstall/update the broken/outdated packages.
  • Install pacman-static
    • either via AUR as explained above
  • or if system files aren’t working to makepkg/install you can use  extracted binary compiled by official Arch sources.
  • use pacman-static instead of pacman to reinstall or update the broken or outdated packages.

Please note, you will need to mount the necessary system partitions before chrooting to the root of your local installation like above.

Official files and mirrors for Arch Linux can be found here.

References

[1] Arch Wiki: chroot

Launching desktop environments with startx

You can have total control over launching your operating system’s graphic server in Linux.

Most desktop environments use a graphic server called either X (or Xorg) or Wayland. X is approximately 40 years old and it is the legacy graphic server that serves its purpose really well. X still dominates the market.

Wayland on the other hand is a modern approach that’s been gaining traction and it also has lots of support and is being used by quite a few desktop environments in Linux such as KDE.

I find it very useful to be able to manually launch and quit desktop environments in Linux and shut down (or never launch) the graphic server when I want to.

This Linux tutorial  will explain how to take control over launching the Xorg graphic server in almost any Linux distro and the reasoning behind it.

Table of Contents

  • X Graphic Server Terminology
  • Launching Desktop Environments with startx
    1. Gnome .xinitrc
    2. XFCE .xinitrc
    3. Cinnamon .xinitrc
    4. KDE Plasma .xinitrc
    5. Qtile .xinitrc
  • Handling the Login Manager
  • Benefits of Manually Launching Graphic Environment in Linux
  • Troubleshooting startx & .xinitrc

X Graphic Server Terminology

Here is a breakdown of the most commonly used desktop environments and the codes you can use to start them with startx which will initiate X graphic server/client through Xinit (Server Launcher Program) and Xorg (Graphic Server).

Xorg is synonymous with X server or sometimes just X.

You can create .xinitrc under user’s home and add just a one line code such as:

Xinit : Program which launches Xorg graphic server.

Xorg : X Graphic Server

startx : Command used to launch graphic applications which defaults to .xinitrc file.

.xinitrc : File under the home directory that instructs startx command

startplasma-x11 start. : an example .xinitrc entry. See below for specific entries for each Desktop Environment or Window Manager.

Screenshot of Endeavour OS based on Arch Linux and KDE Plasma Desktop

1. Launching desktop environments with startx

Here is a breakdown of the most commonly used desktop environments and the codes you can use to start them with startx which will initiate X graphic server/client through Xinit (Server Launcher Program) and Xorg (Graphic Server).

Xorg is synonymous with X server or sometimes just X.

You can create .xinitrc under user’s home and add just a one line code such as:

startplasma-x11 start.

So the code goes inside .xinitrc, and the directory structure would be:

/home/userxyz/.xinitrc

Make sure .xinitrc file has a dot before it as shown. Xinit and Xorg directly looks for this file under home directory. (dot makes the file hidden which can be seen by ls -la command)

Here are the specific codes to include in the .xinitrc file for each desktop environment.

There are also a few points down below which might be helpful for troubleshooting or avoiding pitfalls if you are doing this kind of stuff for the first time. You can also read about the benefits of manually launching desktop environments or at least my reasoning for it. If you have comments feel free to drop it at the bottom.

Please review “Disabling the Login Manager” down below before starting to use the startx command.

Gnome .xinitrc

You can instruct startx either via exec in the beginning or start in the end.

gnome-session start

or

exec gnome-session

XFCE .xinitrc

startxfce4 start

or

exec startxfce4

Cinnamon .xinitrc

cinnamon-session start

or

exec cinnamon-session

KDE Plasma .xinitrx

startplasma-x11 start

or

exec startplasma-x11

Qtile .xinitrc

qtile start

or

exec qtile

Now, almost a year later, we have Google’s Imagen published which makes Dall-e 2 results look primitive. Imagen’s image outcomes are so impressive that, it looks beyond the artistic capabilities of human designers and illustrators. Of course art and design are subjective terms but every single Imagen image published is truly mind-blowingly accurate, well-designed and photorealistic.

Starting desktop environment with startx

2. Handling the Login Manager

If you are building a Linux system from the ground up then you might not have a Login Manager but if you are trying to modify an existing distro such as Endeavour OS, Mint OS, Manjaro OS, Fedora, Cent OS, Ubuntu, OpenSUSE etc. then you will want to disable to existing login manager.

Starting and Stopping the Login Manager

You can also immediately start and stop the SDDM Login Manager service using the commands below. (Stopping the login manager on the go might be tricky sometimes but disabling fixes that problem for the next boot.)

  • For starting SDDM:
    • systemctl start sddm
  • For stopping SDDM:
    • systemctl stop sddm

Enabling and Disabling the Login Manager

First you need to find out which Login Manager your system is using if there is one. Login Managers (or Display Managers or DMs) are graphical login screens that can be useful in some cases. I personally don’t find them to be very useful. They take resources and complicate things and running Desktop environments from tty is perfectly minimalistic and aesthetically pleasing anyway.

Here are some commands to list login managers, find them, enable/disable them and start/stop them…

You can list the services enabled at systemd level via systemctl by using either of the following commands.

First one is the native way of filtering enabled services with systemctl’s own flag while second method makes use of grep command for a similar output.

systemctl list-unit-files --state=enabled

systemctl list-unit-files | grep enabled

SDDM Login Manager

So disabling and enabling login manager at startup level is quite easy with systemd. Here are the example commands for SDDM login manager.

For disabling SDDM:

systemctl disable sddm

For enabling SDDM:

systemctl enable sddm

Make sure you are comfortable with working with the CLI (command line interface) before disabling the login manager first because you might not have the login manager which triggers the graphic interface of the desktop manager automatically behind the scenes.

Login Managers are quite persistent meaning even if you stop them, they might auto-launch and get in the way when you are trying to manually launch your graphic environment (through desktop environments, windows managers or sometimes specific individual apps.)

If you disable your login manager and reboot your Linux system, you will just get a TTY command line without graphics server activated. Normally you will get TTY1.

You can usually navigate between multiple virtual terminals using the Ctrl+Alt F2 (or F3 for TTY3, F1 for back to TTY1, F4 for TTY4 and so on.)

Light DM (LDM) Login Manager

You can enable / disable Light DM as below:

For disabling Light Display Manager:

systemctl disable lightdm

For enabling Light Display Manager:

systemctl enable lightdm

3. Exiting Xorg Graphic Server and Desktop Environments

After launching the desktop environment from the tty command line, you might want to go back to the tty environment and/or leave your computer on without the Xorg graphic server.

Termination of Xorg and desktop environments can be conveniently achieved when there is no Login Manager such as SDDM running. That’s because SDDM can be persistent and auto launch itself which can make it difficult to kill without disrupting the system.

To kill the desktop environment just execute the appropriate one for your setup from the following commands.

Commands to kill various Desktop Environments

So disabling and enabling login manager at startup level is quite easy with systemd. Here are the example commands for SDDM login manager.

For killing Gnome session:

killall gnome-session

For killing XFCE session:

killall startxfce4

For killing Cinnamon session:

killall cinnamon-session

For killing KDE Plasma session:

killall startplasma-x11

For killing Qtile session:

killall qtile

It’s also perfectly possible to map the commands above to a key combination to kill the desktop environment. One combination that works perfectly for me is Ctrl+Meta+Q. It’s usually a free combo and has the Q (quit) in it so it’s easy to remember. It’s also not easy to accidentally hit it.

Benefits of Manually Launching Graphic Environment in Linux

It’s starting to feel like we are there. AI’s real-world successes are being increasingly felt. Every new applied-AI milestone leaves your mouth open. It’s hard not to imagine the societal impacts. Millions of humans who derive lots of self-worth and satisfaction from their work suddenly becoming unemployed or even worse irrelevant.

  • Achieving focus: Being immersed in a graphic environment has its pros and cons. Having the graphic environment can be counterproductive when you need to focus on certain tasks. For some tasks that can be done in the command line, I find it helpful to not having a desktop environment launched at all. There is something special about not having the graphic server launched at all versus having the GUI programs available in the environment. That being said of course sometimes the graphic environment can be very useful and even inevitable. Some tasks I find myself performing through the CLI are:
    • Writing code
    • Writing scripts
    • Playing music (cmus)
    • Taking notes
    • Writing articles
    • Reading notes/books
  • Increased Ram & CPU resources: In most computers, the most resource hungry components are often the graphic environments. Graphic server communicates with each input and each output at pixel level which understandably creates a burden on RAM and CPU resources.
  • Increased battery life: Manually launching X graphic server when needed can optimize battery life enormously. I forgot my Dell XPS on for one week and the battery was still 90%+. Since there’s no graphic server launched the necessary processes to keep the OS running are at absolute minimal. X graphic server comes with many support libraries as well which can bloat the system.
  • Having more control: Being able to launch and terminate the graphic server and all the desktop environment and other related programs in a very practical way gives you more power over using your computer’s operating system and even its hardware. It’s also helpful to understand the inner workings of the operating system’s graphic components.

graphical.target vs multi-user.target

systemctl get-default

systemctl set-default graphical.target

systemctl set-default multi-user.target

Troubleshooting startx and .xinitrc

1- startx is not meant to be run as root

2- .xinitrc is not under home directory.

3- .xinitrc has incorrect entries.

start startplasma-x11 instead of startplasma-x11 start

entries after using exec

forgetting & character at the end of the line when there is multiple entries.

4- Having multiple .xinitrc in multiple locations which will confuse Xorg server or make it ignore intended .xinitrc.

5- User doesn’t have access to video group

6- Not having necessary Xorg packages installed.

7- Not having necessary graphic firmware installed.

References

[1] X Graphic Server Documentation: https://www.x.org/wiki/UserDocumentation/
[2] Arch Xorg Documentation: https://wiki.archlinux.org/title/xorg
[3] Debian’s Xinitrc Documentation: https://wiki.debian.org/Xinitrc

Installing AUR Packages with Pacman

We will compile and install software packages from source code.

Content Table:

  • AUR vs Arch Packages
  • Installing packages from source
    • Step 1: Cloning the source code
    • Step 2: Compiling the package
    • Step 3: Installing the package
  • Post Installation: Updating & removing external packages
  • Troubleshooting makepkg
Software packages can be comopiled from source code in Arch Linux.

AUR - Arch User Repository

AUR or Arch User Repository is a repo which holds PKGBUILDs or source code of various computer programs.

AUR can be used to find source code ready to be compile and make packages. You will realize while AUR lists these packages, source codes themselves are hosted on Github.

Let’s get to it.

AUR vs Arch Packages

87K packages can be found on AUR as of December 2022 while there are about 13K packages on Arch Packages.

Arch packages can be installed directly using the pacman package manager.

sudo pacman -S <packagename>

Installing packages through AUR requires a couple of extra steps.

In this mini-tutorial we will learn how to install packages directly from source code using 2 methods:

  • makepkg
  • pacman

1. Cloning the source code

git clone https://aur.archlinux.org/librewolf.git

Make sure git clone isn’t run as root, a common mistake…

git clone will create a folder with the necessary program files along with the PCKGBUILD (a script used to build packages). You can then navigate to this folder and continue with the instructions in the following steps from inside this folder. 
cd -newpackagefolder-

2. Compiling the package

We will use makepkg to compile source code. Makepkg is a convenient package build utility. You can read more about it here.
We can compile a package using makepkg -s.
-s flag: Stands for syncing and syncs the dependencies of the package being compiled so you don’t have to compile them one by one.
makepkg -s

You can also use the following command to continue with installation of the package built from source code.

makepkg -si

If you’ve used makepkg -si, you don’t need to follow the 3rd step to install the package using pacman.

3. Installing the package

I prefer this method because I am used to using pacman to install packages.

pacman -U <.zst file> 
pacman -U <.zst file>

Post Installation (Updating or Removing Packages)

You can still manage packages installed through compiling source code or other external sources to an extent.

You can’t update them automatically since they were compiled and they aren’t included in the pacman’s arch packages repo. But you can use pacman for related operations below:

  • remove manually installed packages
    • pacman -R <packagename>
  • get information about manually installed packages
    • pacman -Qi <packagename>

If you find out your package’s version is becoming outdated. You can update it using the same compiling and installation steps explained above and the package will be updated.

Troubleshooting makepkg

Here are a few of the most common makepkg errors and their solutions. It’s usually a simple fix to solve makepkg errors.

1- Pacman failed to install missing dependencies

When makepkg -s is used, dependencies will be installed given they exist in the pacman repository. Sometimes this is not the case and dependencies can’t be satisfied. In which case you’ll get an error similar to below:
==> ERROR: 'pacman' failed to install missing dependencies.
==> Missing dependencies: -> ocaml-ocamlsdl -> ocaml-graphics -> lablgtk2
==> ERROR: Could not resolve all dependencies.
In that case dependencies also need to be compiled manually. You can find them on AUR and repeat the steps above. Once you have the dependencies installed you can continue with makepkg -s to compile the main package you intended to install.

2- You do not have write permission for the directory $BUILDDIR

This is usually caused by wrong directory privileges and particularly package directory being owned by the root.

If you run sudo git clone you might get write error.

Solution is to just remove the directory and rerun git clone without sudo or change directory ownership to the user. Simple as that.

==> ERROR: You do not have write permission for the directory $BUILDDIR (/tmp/librewolf-bin).
Aborting...

If you run git clone as root you would be required to run makepkg as root as well. The problem is running makepkg as root is not allowed. You can see the specific explanation for that specific error in the following section.

3- Running makepkg as root is not allowed as it can cause permanent damage

Makepkg should never be run as root and the system won’t allow you to do so. If you attempt running makepkg as root you will get the following error.

==> ERROR: Running makepkg as root is not allowed as it can cause permanent, catastrophic damage to your system.

The rationale behind this restriction is that if you compile a source code acquired from external resources as root then you would completely open the computer and the linux os to intentional and unintentional risks as these packages are created  by users and added to the AUR (or another repo or Github) repo.

Resources

[1] Pacman Package Manager Documentation: https://wiki.archlinux.org/title/pacman

[2] AUR (Arch User Repository): https://aur.archlinux.org/packages

[3] Makepkg package build utility documentation: https://wiki.archlinux.org/title/makepkg