Home Page
Home
Search:

Welcome to the digital realm of Jeffrey Riaboy. Here, you’ll find a curated collection of my endeavors, musings, updates, and assorted reflections. As a self-taught programmer and avid computer enthusiast, I work in C++ as a favorite choice, though the realities of our digital world have led me to become proficient in dozens of other languages.

Programming has been a lifelong journey for me which I have been hacking on since before I can rememember. It is my passion, profession, first love, and my constant challenge, offering both fulfillment and frustration in equal measure.

This space is dedicated to sharing insights, innovations, and inspirations I’ve gathered along the way. My aim is for you to discover something here that sparks your interest or serves your needs, as that is the driving force behind my commitment to compile and share this content. Dive in and explore. Your presence is highly appreciated.

Dakusan~
The original (well... last) intro page to my website before this became the home. It is a flash portal to my personal sites of the past.
Intro
[1999-2001?] My ancient NES emulator made in Visual Basic (which was made to prove the power and flexibility [not speed] of the language).
HyNes
[2002] A chronicle of my experiences and tinkering from early ’02 to early ’04 on an addictive yet horribly crappy MMORPG. Site also has some nice “hacking”/reverse engineering tutorials.
Ragnarok Hacking
I’ve temporarily set this to link to the Projects section of this website until I’m ready to announce the new website this will link to.
Projects
Updates Archive
Updated:11/21/24

I added a live “total progress” progress bar to the MD5Sum List Script. It utilizes a temp fifo file and a backgrounded “pv”.

Updated:04/15/24
The nulltypes system has been overhauled so all null types are under a generic type named NullType in the top level package. For example, instead of using nulltypes.NullUint8 you would now use NullType[uint8]. This also really helped clean up the null types code. This is a version breaking change, hence the minor version number update.

Other minor changes:
  • Readme file and package information has been updated with the following changes:
    • The type support section has been redone for clarity
    • The structs in the code examples have had the members labeled to explain their used supported type
  • Marshled JSON strings are now properly json escaped
  • Added bypass for my RawBytes bug fix, now that it has been fixed in go v1.23
  • Removed test case that is no longer compatible with go 1.21+
Download Content
Section: Misc > Resume
Updated:03/23/24

I made a pretty massive overhaul of my resume. Both the PDF and the copy on this site’s resume section have been updated.

Download Content
Post Archive
RSS Feed
Sekiro letter opener sword

In honor of my recent Sekiro accomplishment, I bought myself a cute little Sekiro letter opener sword to go on my office side desk.


I honestly think I may consider Sekiro the best first player game ever made. It’s by far better than any of From Softwares’ other games (I am a big Dark Souls fan. Elden ring was boringly easy). I had been trying to do a perfect run for about a month and finally got it done.


In 10.5 hours I killed every enemy exactly once (no more, no less), 0 death screens, no cheese, no exiting battles, bought out all vendors, and collected all items. It was a lot of fun.


One of the reasons I love Sekiro so much is that it’s about the zen perfection of a specific skill set, patience, timing, and reaction speed (I am also a big fan of Super Hexagon - almost have a 4 minute run on it). There is no experience grinding to get past bosses, though certain skills acquired through exp are extremely helpful. This makes it harder than dark souls, and winning gives much more of an accomplishment feeling.


It goes far beyond that though. The level design is top notch and art assets are visually stunning. It’s an integrated cohesive world more so than any of the Dark Souls, and I’m a bit of a sucker for Japanese themes.


Combat is wonderful. Pretty much all the fights are fun and fair once you learn how to play, and you can even freaking pause the game! :-D You have to keep on your toes at all times cause ANYONE can kill you. The game both lets you sneak or just run in and murder everything if you’re good enough. The replayability is much higher than any of the other FromSoft souls like games too.


Btrfs RAID1 inside LUKS encrypted volumes

I recently moved to Linux and have all my hard drives Luks encrypted, including the primary. I decided to convert my ext4 partitions to Btrfs recently, which I’m totally loving. I also decided to grab another nvme drive and use it as a RAID1 (mirror) drive against my primary drive, using Btrfs’ RAID mechanics. Below are the instructions to accomplish this.

Do note that this is for a situation where you already have a BTRFS volume and want to add a device as RAID1. This assumes you already have your system booting to the LUKS encrypted drive with the root being btrfs. Many modern Linux OS installers can do this for you automatically. Parts of these instructions can still be used in other situations.


  • Hopefully you also have a swap partition under the same LVM as your LUKS root (the Linux Mint installer does this by default), as we’ll be using it. If not, you’ll need to modify the instructions. This script resizes the swap partition and adds an “extra” partition to hold your drive key. This is required because a drive key cannot be loaded off your btrfs volume as both drives need to be unlocked first.
  • This should be ran from another operating system. I would recommend using Universal USB Installer to do this. It allows you to put multiple OS live cds on a USB key, including optional persistence.
  • Run the following script as root (you can use sudo). Make sure to fill in the variables section first. Or even better, run the script 1 line at a time to make sure there are no problems.

#!/bin/bash
#-----------------------------------Variables----------------------------------
#Current root drive
CurPart="nvme0n1p3" #The current drive partition in /dev. This example uses nvme disk #0 partition #3
CurCryptVol="vgmint" #What you named your LVM under LUKS
CurCryptRoot="root" #What you named your root partition under the LVM
CurCryptRootSubVol="/" #The path of the subvolume that is used as the root partition. For example, I use “@”
CurCryptSwap="swap_1" #What you named your swap partition under the LVM
CurCryptExtra="extra" #What you WANT to name your extra partition under the LVM
CurCryptExtraSize="100M" #How big you want your extra partition that will hold your key file
CurKeyPath="" #The path to a key file that will unlock both drives. If left blank then one will be created

#New drive
NewDrive="nvme1n1" #The new drive in /dev. This example uses nvme disk #1
NewPart="nvme1n1p3" #The new partition in /dev. You will be creating this with the cfdisk. This example uses nvme disk#1 partition#3
NewCryptName="raid1_crypt" #What we’ll name the root LUKS partition (no LVM)

#Other variables you do not need to set
CurMount="/mnt/primary"
ExtraMountPath="$CurMount/mnt/extra"
BtrfsReleasePath="kdave/btrfs-progs"
BtrfsReleaseFile="btrfs.box.static"
DriveKeyName="drivekey"

echo "---------------------------------Update BTRFS---------------------------------"
echo "Make sure you are using the latest btrfs-progs"
cd "$(dirname "$(which btrfs)")"
LATEST_RELEASE=$(curl -s "https://api.github.com/repos/$BtrfsReleasePath/releases/latest" | grep tag_name | cut -d \" -f4)
wget "https://github.com/$BtrfsReleasePath/releases/download/$LATEST_RELEASE/$BtrfsReleaseFile"
chmod +x "$BtrfsReleaseFile"

echo "Link all btrfs programs to btrfs.box.static. Rename old files as .old.FILENAME"
if ! [ -L ./btrfs ]; then
  for v in $(\ls btrfs*); do
    if [ "$v" != "$BtrfsReleaseFile" ]; then
      mv "$v" ".old.$v"
      ln -s "$BtrfsReleaseFile" "$v"
    fi
  done
fi

echo "--------------------------Current drive and key setup-------------------------"
echo "Mount the current root partition"
cryptsetup luksOpen "/dev/$CurPart" "$CurCryptVol"
vgchange -ay "$CurCryptVol"
mkdir -p "$CurMount"
mount -o "subvol=$CurCryptRootSubVol" "/dev/$CurCryptVol/$CurCryptRoot" "$CurMount"

echo "If the extra volume has not been created, then resize the swap and create it"
if ! [ -e "/dev/$CurCryptVol/$CurCryptExtra" ]; then
  lvremove -y "/dev/$CurCryptVol/$CurCryptSwap"

  lvcreate -n "$CurCryptExtra" -L "$CurCryptExtraSize" "$CurCryptVol"
  mkfs.ext4 "/dev/$CurCryptVol/$CurCryptExtra"

  lvcreate -n "$CurCryptSwap" -l 100%FREE "$CurCryptVol"
  mkswap "/dev/$CurCryptVol/$CurCryptSwap"
fi

echo "Make sure the key file exists, if it does not, either copy it (if given in $CurKeyPath) or create it"
mkdir -p "$ExtraMountPath"
mount "/dev/$CurCryptVol/$CurCryptExtra" "$ExtraMountPath"
if ! [ -e "$ExtraMountPath/$DriveKeyName" ]; then
  if [ "$CurKeyPath" != "" ]; then
    if ! [ -e "$CurKeyPath" ]; then
      echo "Not found: $CurKeyPath"
      exit 1
    fi
    cp "$CurKeyPath" "$ExtraMountPath/$DriveKeyName"
  else
    openssl rand -out "$ExtraMountPath/$DriveKeyName" 512
  fi
  chmod 400 "$ExtraMountPath/$DriveKeyName"
  chown root:root "$ExtraMountPath/$DriveKeyName"
fi

echo "Make sure the key file works on the current drive"
if cryptsetup --test-passphrase luksOpen --key-file "$ExtraMountPath/$DriveKeyName" "/dev/$CurPart" test; then
  echo "Keyfile successfully opened the LUKS partition."
  #cryptsetup luksClose test #This doesn’t seem to be needed
else
  echo "Adding keyfile to the LUKS partition"
  cryptsetup luksAddKey "/dev/$CurPart" "$ExtraMountPath/$DriveKeyName"
fi

echo "--------------------------------New drive setup-------------------------------"
echo "Use cfdisk to set the new disk as GPT and add partitions."
echo "Make sure to mark the partition you want to use for the raid disk as type “Linux Filesystem”."
echo "Also make it the same size as /dev/$CurPart to avoid errors"
cfdisk "/dev/$NewDrive"

echo "Encrypt the new partition"
cryptsetup luksFormat "/dev/$NewPart"

echo "Open the encrypted partition"
cryptsetup luksOpen "/dev/$NewPart" "$NewCryptName"

echo "Add the key to the partition"
cryptsetup luksAddKey "/dev/$NewPart" "$ExtraMountPath/$DriveKeyName"

echo "Add the new partition to the root btrfs file system"
btrfs device add "/dev/mapper/$NewCryptName" "$CurMount"

echo "Convert to RAID1"
btrfs balance start -dconvert=raid1 -mconvert=raid1 "$CurMount"

echo "Confirm both disks are in use"
btrfs filesystem usage "$CurMount"

echo "--------------------Booting script to load encrypted drives-------------------"
echo "Get the UUID of the second btrfs volume"
Drive2_UUID=$(lsblk -o UUID -d "/dev/$NewPart" | tail -n1)

echo "Create a script to open your second luks volumes before mounting the partition"
echo "Note: In some scenarios this may need to go into “scripts/local-premount” instead of “scripts/local-bottom”"
cat <<EOF > "$CurMount/etc/initramfs-tools/scripts/local-bottom/unlock_drive2"
#!/bin/sh
PREREQ=""

prereqs()
{
    echo "\$PREREQ"
}

case "\$1" in
    prereqs)
        prereqs
        exit 0
        ;;
esac

. /scripts/functions
cryptroot-unlock
vgchange -ay "$CurCryptVol"
mkdir -p /mnt/keyfile
mount "/dev/$CurCryptVol/$CurCryptExtra" /mnt/keyfile
cryptsetup luksOpen /dev/disk/by-uuid/$Drive2_UUID "$NewCryptName" "--key-file=/mnt/keyfile/$DriveKeyName"
umount /mnt/keyfile
rmdir /mnt/keyfile

mount -t btrfs -o "subvol=$CurCryptRootSubVol" "/dev/$CurCryptVol/$CurCryptRoot" /root

#If you are weird like me and /usr is stored elsewhere, here is where you would need to mount it.
#It cannot be done through your fstab in this setup.
#mount --bind /root/sub/sys/usr /root/usr

mount --bind /dev /root/dev
mount --bind /proc /root/proc
mount --bind /sys /root/sys
EOF

chmod 755 "$CurMount/etc/initramfs-tools/scripts/local-bottom/unlock_drive2"

echo "--------------------Setup booting from the root file system-------------------"
echo "Prepare a chroot environment"
for i in dev dev/pts proc sys run tmp; do
  mount -o bind /$i "$CurMount/$i"
done

echo "Run commands in the chroot environment to update initramfs and grub"
chroot "$CurMount" <<EOF
echo "Mount the other partitions (specifically for “boot” and “boot/efi”)"
mount -a

echo "Update initramfs and grub"
update-initramfs -u -k all
update-grub
EOF

echo "-----------------------------------Finish up----------------------------------"
echo "Reboot and pray"
reboot

Checking permissions before updating Flatpaks
AI IS FUCKING CRAZY

I wanted to make a script to check if flatpak update app permissions have changed before updating them. I decided to use ChatGPTs newest model, 01-preview for this task and holy crap am I impressed. I had to give it 14 commands to get to the final product and it seems to work great. That number could have been reduced quite a bit had I done things a bit differently.

I still had to find the problems, look up reference docs to correct it, and even debug in an IDE a little. But just telling it where the problems were, it got there in the end, and its user interaction output is way better than what I was planning on doing.

Anywho, here are the commands I gave it, followed by the code it gave me in the end. I have not throughly checked and tested the code, but it seems to work from a few quick trials. For demonstration purposes, I have left all of my original typos uncorrected below.


Write me a program in php that checks if there are any updates for my installed flatpaks. and then for each flatpak check and see if the permissions have changed between my current version and the new version. report to me the changes. then ask me if i want to continue with the updates.
your code doesnt work.
first $command = 'flatpak list --app --updates --columns=application,app,branch,origin,installation'; should be flatpak remote-ls --updates --columns=application,app,branch,origin (installation is an invalid column)
and then flatpak info --show-permissions doesnt give the original permissions. you have to get the original non-user-modified permissions from /var/lib/flatpak
the line if (empty($output)) { is incorrect, it should be if (count($output)) { as $output is an array, not a string
$columns = preg_split('/\s{2,}/', trim($line)); is incorrect. its tab delimited, so it just needs to be explode("\t", trim($line));
you also have a problem with the $metadataPath. its not always in /var/lib/flatpak/app/ . It may be in /var/lib/flatpak/runtime/. You can confirm which directory it should be in with the options column having "runtime" as an option.
in the --columns for your first command, "application" should be "name"
actually, you cant get the options from flatpak remote-ls --updates . you have to get the options from flatpak list
flatpak remote-info does not have an option --show-permissions
in your flatpak remote-info command, there is no --branch option.
there is no --assumeno option
there is no --dry-run permission. perhaps use --no-deploy?
you are incorrect. flatpak remote-info does not have an option --show-permissions . we already went over that. perhaps flatpak remote-info --show-metadata to get the permissions
flatpak remote-info does not have --branch
when checking the metadata, do not compare the info under [Application]
also, move the "Get architecture" line above its for loop
you have a problem with exec($remoteInfoCommand, $remoteOutput); you need to always empty $remoteOutput before running the exec. Otherwise, it appends the data to what is already there.


<?php
/**
 * This script checks for Flatpak updates and reports permission changes.
 */

// Function to parse permissions from Flatpak metadata
function parsePermissions($content) {
    $permissions = [];
    $lines = is_array($content) ? $content : explode("\n", $content);
    $currentSection = '';
    $skipSections = ['Application']; // Sections to skip

    foreach ($lines as $line) {
        $line = trim($line);

        if (empty($line)) {
            continue;
        }

        // Check for section headers
        if (preg_match('/^\[(.*)\]$/', $line, $matches)) {
            $currentSection = $matches[1];

            // Skip the [Application] section
            if (in_array($currentSection, $skipSections)) {
                $currentSection = '';
                continue;
            }

            $permissions[$currentSection] = [];
        } elseif ($currentSection !== '') {
            // Only process lines within non-skipped sections
            $parts = explode('=', $line, 2);
            if (count($parts) == 2) {
                $key = $parts[0];
                $values = explode(';', trim($parts[1], ';'));
                $permissions[$currentSection][$key] = $values;
            } else {
                // Handle keys without '=' (e.g., single permissions)
                $permissions[$currentSection][$line] = [];
            }
        }
    }

    return $permissions;
}

// Function to compare permissions
function comparePermissions($current, $new) {
    $differences = [];

    // Get all sections
    $sections = array_unique(array_merge(array_keys($current), array_keys($new)));

    foreach ($sections as $section) {
        $currentSection = isset($current[$section]) ? $current[$section] : [];
        $newSection = isset($new[$section]) ? $new[$section] : [];

        // Get all keys in this section
        $keys = array_unique(array_merge(array_keys($currentSection), array_keys($newSection)));

        foreach ($keys as $key) {
            $currentValues = isset($currentSection[$key]) ? $currentSection[$key] : [];
            $newValues = isset($newSection[$key]) ? $newSection[$key] : [];

            // Compare values
            $added = array_diff($newValues, $currentValues);
            $removed = array_diff($currentValues, $newValues);

            if (!empty($added) || !empty($removed)) {
                $differences[$section][$key] = [
                    'added' => $added,
                    'removed' => $removed,
                ];
            }
        }
    }

    return $differences;
}

//Dakusan: Added this function and wrapped other echo statements into it after the fact
//Function to output a line with appId (if given), colored red (IsError=true) or green (IsError=false)
function outputColor($appId, $str, $isError=true)
{
    //Determine if coloring should be used
    static $hasColors=null;
    if(!isset($hasColors)) {
        $hasColors=stream_isatty(STDOUT) && (!trim(`command -v tput`) || intval(`tput colors`)>=16);
    }

    echo
        (!$hasColors ? '' : ($isError ? "\e[31m" : "\e[32m")).
        ($appId ? "$appId: " : '').
        $str.
        (!$hasColors ? '' : "\e[0m").
	"\n";
}

// Function to display permission differences
function displayDifferences($appId, $differences) {
    outputColor($appId, 'Permission changes:');
    foreach ($differences as $section => $keys) {
        outputColor(null, "  [$section]");
        foreach ($keys as $key => $changes) {
            if (!empty($changes['added'])) {
                outputColor(null, "    + $key = " . implode(';', $changes['added']));
            }
            if (!empty($changes['removed'])) {
                outputColor(null, "    - $key = " . implode(';', $changes['removed']));
            }
        }
    }
    echo "\n";
}

// Get architecture (moved above the loop)
$archCommand = 'flatpak --default-arch';
exec($archCommand, $archOutput);
$arch = trim(implode('', $archOutput));

// Step 1: Get the list of installed Flatpaks
$installedCommand = 'flatpak list -a --columns=application,branch,origin,options'; //Dakusan: Added -a after the fact
exec($installedCommand, $installedOutput);

// Remove header line if present
if (!empty($installedOutput) && strpos($installedOutput[0], 'Application ID') !== false) {
    array_shift($installedOutput);
}

// Build an associative array of installed applications
$installedApps = [];

foreach ($installedOutput as $line) {
    // The output is tab-delimited
    $columns = explode("\t", trim($line));
    if (count($columns) >= 4) {
        $appId = $columns[0];
        $branch = $columns[1];
        $origin = $columns[2];
        $options = $columns[3];

        $installedApps[$appId] = [
            'appId' => $appId,
            'branch' => $branch,
            'origin' => $origin,
            'options' => $options,
        ];
    }
}

// Get the list of available updates
$updatesCommand = 'flatpak remote-ls --updates --columns=application';
exec($updatesCommand, $updatesOutput);

// Remove header line if present
if (!empty($updatesOutput) && strpos($updatesOutput[0], 'Application ID') !== false) {
    array_shift($updatesOutput);
}

// Build a list of applications that have updates
$updatesAvailable = array_map('trim', $updatesOutput);

if (empty($updatesAvailable)) {
    echo "No updates available for installed Flatpaks.\n";
    exit(0);
}

$permissionChanges = [];

foreach ($updatesAvailable as $appId) {
    if (!isset($installedApps[$appId])) {
        outputColor($appId, 'Installed app not found. Skipping.'); //Dakusan: Added this line after the fact
        continue;
    }

    $app = $installedApps[$appId];
    $branch = $app['branch'];
    $origin = $app['origin'];
    $options = $app['options'];

    // Determine if it's an app or runtime
    $isRuntime = strpos($options, 'runtime') !== false;

    // Paths to the metadata files
    if ($isRuntime) {
        $metadataPath = "/var/lib/flatpak/runtime/$appId/$arch/$branch/active/metadata";
    } else {
        $metadataPath = "/var/lib/flatpak/app/$appId/$arch/$branch/active/metadata";
    }

    // Check if the metadata file exists
    if (!file_exists($metadataPath)) {
        outputColor($appId, 'Metadata file not found. Skipping.');
        continue;
    }

    // Read current permissions from the metadata file
    $metadataContent = file_get_contents($metadataPath);
    $currentPermissions = parsePermissions($metadataContent);

    // Get new metadata from remote
    $ref = $appId . '/' . $arch . '/' . $branch;
    $remoteInfoCommand = 'flatpak remote-info --show-metadata ' . escapeshellarg($origin) . ' ' . escapeshellarg($ref);

    // Clear $remoteOutput before exec()
    $remoteOutput = [];
    exec($remoteInfoCommand, $remoteOutput);

    if (empty($remoteOutput)) {
        outputColor($appId, 'Failed to retrieve remote metadata. Skipping.');
        continue;
    }

    // Parse new permissions from the remote metadata
    $newPermissions = parsePermissions($remoteOutput);

    // Compare permissions
    $differences = comparePermissions($currentPermissions, $newPermissions);

    if (!empty($differences)) {
        $permissionChanges[$appId] = $differences;
        displayDifferences($appId, $differences);
    } else { //Dakusan: Added this condition after the fact
        outputColor($appId, 'No permission changes found.', false);
    }
}

// If there are no permission changes, inform the user
if (empty($permissionChanges)) {
    echo "No permission changes detected in the available updates.\n";
}

// Ask user if they want to proceed
echo "Do you want to proceed with the updates? (y/N): ";
$handle = fopen("php://stdin", "r");
$line = fgets($handle);
$answer = trim(strtolower($line));

if ($answer == 'y' || $answer == 'yes') {
    // Proceed with updates
    echo "Updating Flatpaks...\n";
    passthru('flatpak update -y');
} else {
    echo "Updates canceled.\n";
}

fclose($handle);
?>