Zhanga Redux
The chronicles of the work and personal life of a boring software developer with an awesome dog.
Generating a decrypted PDF using QPDF
Saturday, March 14, 2020
If you have a password-protected PDF and want to generate a decrypted version, it’s very simple with QPDF:
$ qpdf --password=123 --decrypt input.pdf output.pdf
To avoid entering the password on the command line (perhaps you don’t want it saved in your shell history):
$ qpdf @- --decrypt input.pdf output.pdf
Then enter --password=123
and hit enter and ^D
.
Disabling kdump to reclaim missing RAM on CentOS 8
Saturday, January 25, 2020
After setting up a Droplet on DigitalOcean (a VPS) using their CentOS 8 image, I found that various sources (like top
, free
, and even /proc/meminfo
) were reporting only 821 MB of total RAM, even though the instance should have had 1 GB. Where did the missing ~200 MB go?
It turns out that kdump is enabled by default. In short, it uses a second kernel to capture dumps in case the running kernel crashes. I don't need this since I'm not going to do anything useful with those dumps anyway, so here's how to disable it and get the memory back.
First, check to see if it's enabled by looking for a nonzero value in /sys/kernel/kexec_crash_size
:
$ cat /sys/kernel/kexec_crash_size
You'll also see a line like this in dmesg
:
kernel: Reserving 160MB of memory at 672MB for crashkernel (System RAM: 1023MB)
To disable it, edit /etc/default/grub
and change crashkernel=auto
to crashkernel=no
, then:
# grub2-mkconfig -o /boot/grub2/grub.cfg # systemctl disable kdump # reboot
Now you should have your memory back!
Identifying file associated with a bad sector on ext2/ext3/ext4
Monday, April 16, 2018
I got some SMART warnings about a bad sector on my hard drive, and I wanted to know which specific file had the bad sector.
First, I looked at the SMART logs to see where the problem was:
# smartctl -x /dev/sdd ... After command completion occurred, registers were: ER -- ST COUNT LBA_48 LH LM LL DV DC -- -- -- == -- == == == -- -- -- -- -- 40 -- 51 00 08 00 00 04 4b 5b c0 40 00 Error: UNC at LBA = 0x044b5bc0 = 72047552 ...
fdisk -l
is useful for looking at the partition info and sector size:
# fdisk -l /dev/sdd Disk /dev/sdd: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0x54afc7e9 Device Boot Start End Sectors Size Id Type /dev/sdd1 2048 3907029167 3907027120 1.8T 83 Linux
Then I used badblocks
to look around that physical sector for more bad sectors. My sector size is 512 bytes, shown above; also, badblocks
takes the end sector number first, followed by the start sector:
# badblocks -b 512 /dev/sdd 72047570 72047540 72047552 72047553 72047554 72047555 72047556 72047557 72047558 72047559 72047560
Finally, debugfs
is useful for finding which files are on those blocks.
Explanation:
- First, find the logical filesystem block number by computing (physical sector - partition start sector) * (physical sector size / filesystem block size). In my case, this would be (72047552 − 2048) * (512 / 4096) = 9005688. Since there are 9 contiguous sectors affected, the bad area stretches into block 9005689 as well.
- Use
testb
to see whether there is actually anything there. If not, then no data is lost. - Use
icheck
to find the inode corresponding to those blocks. Luckily (?), both bad blocks are associated with the same inode here. - Finally, use
ncheck
to find the pathname(s) associated with the inode.
# debugfs /dev/sdd1 debugfs 1.43.5 (04-Aug-2017) debugfs: testb 9005688 Block 9005688 marked in use debugfs: testb 9005689 Block 9005689 marked in use debugfs: icheck 9005688 Block Inode number 9005688 105518423 debugfs: icheck 9005689 Block Inode number 9005689 105518423 debugfs: ncheck 105518423 Inode Pathname 105518423 /drz/rdiff-backup/artanis/var/lib/pgsql/data/base/21595/26720
Here, it was just a backup file, so once I swap out the hard drive or reallocate the sector, the next backup cycle will fix the lost data.
Using 7-Zip to create AES-256 encrypted zip files from the command line
Sunday, March 11, 2018
The default encryption method used by 7-Zip for Zip files, and the only method supported by InfoZip, is ZipCrypto, which is generally pretty terrible. To create AES-256 encrypted archives using 7-Zip, use:
$ 7z a -tzip -mem=AES256 -p super-secret.zip super-secret.txt
To verify that it worked, use:
$ 7za l -slt super-secret.zip
Notice that all the filenames within the archive are visible; I don't think 7-Zip supports Zip header encryption. You can do it the ghetto way by simply putting a Zip inside an encrypted Zip to hide the filenames in the inner one.
Dual-booting Fedora 25 and Windows 10 on Thinkpad T560 with UEFI
Tuesday, February 28, 2017
I just bought a new Lenovo Thinkpad T560. It runs both Windows 10 and Fedora 25 quite well, but I had a lot of trouble getting the system to dual-boot. In hopes of helping somebody, here are the steps I had to take to get it to work. I'm assuming you're starting with a new laptop with Windows 10 installed, since that's what it comes with from the factory.
- Before doing anything else, open Power Options in Windows and turn off "fast startup" so that Windows fully shuts down.
- Open Disk Management and shrink the Windows partition, leaving enough space for the Linux installation. Luckily, the Windows 10 Disk Management tool makes this very easy and very fast — it wasn't so in the past!
- Head over to a different machine already running Fedora, and grab the netinst iso image. I got mine from here; see the links on the right-hand side of the page.
- Create USB boot media using
livecd-iso-to-disk
:# livecd-iso-to-disk Fedora-netinst-x86_64-25-1.3.iso /dev/sdb1
- The Thinkpad has UEFI Secure Boot enabled by default, which prevents it from booting off of this USB drive. Reboot the laptop and enter the Thinkpad setup utility by hitting F1 at the boot screen. Disable Secure Boot, and enable UEFI + Legacy boot. Save and exit the setup utility.
- At the boot screen again, hit F12 to enter the boot menu and choose the USB drive.
- Follow the Fedora installer like normal, taking care of course to only create partitions in the free space and not overwrite any existing partitions. Since this is a laptop with one drive, I didn't see any need for LVM and I chose the "standard" partitioning + encryption.
- After Fedora setup finished, I wasn't able to boot off the hard drive at all, not even into windows. So, boot off of the USB drive again. Choose the recovery option. It will find the new Fedora installation.
chroot
into your system:# chroot /mnt/sysimage
- Fedora's installer seems to have detected the system as a BIOS (not EFI) booting system. Let's fix that. Open
parted
:# parted /dev/sda
- At the
parted
prompt, toggle thepmbr_boot
flag off:(parted) toggle disk_toggle pmbr_boot (parted) q
You can verify this worked by issuingprint
. The "Disk Flags" list should not containpmbr_boot
. At this point, Windows should be bootable if you hit F12 at the startup screen and choose the Windows Boot Manager, but Fedora still won't boot. - Locate your EFI partition, which is the one with type
EFI System
. Mine is/dev/sda1
, and yours probably is too.# fdisk -l Disk /dev/sda: 477 GiB, 512110190592 bytes, 1000215216 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: (a UUID here) Device Start End Sectors Size Type /dev/sda1 2048 534527 532480 260M EFI System /dev/sda2 534528 567295 32768 16M Microsoft reserved /dev/sda3 567296 205367295 204800000 97.7G Microsoft basic data ...
- Get the EFI partition's UUID:
# blkid /dev/sda1
- Add a line to
/etc/fstab
so it always gets mounted:UUID=abcd-0123 /boot/efi vfat defaults 0 2
- Mount it:
# mount -a
You should see some files in /boot/efi now. - Install stuff needed for EFI boot. Remember, you are still booted into the USB recovery system, but these will install into the right place since the system is
chroot
'ed.# dnf install grub2-efi grub2-efi-modules shim
- Because the system is still booted off of the USB drive,
grub2-mkconfig
doesn't think the system is an EFI system at this point. We'll need to manually copy the GRUB config file:# cp /boot/grub2/grub.cfg /boot/efi/EFI/fedora/grub.cfg
- Open
/boot/efi/EFI/fedora/grub.cfg
(e.g. withvi
) and search/replacelinux16
andinitrd16
withlinuxefi
andinitrdefi
. - Now reboot and hit F12 at the boot screen to open the boot menu. Choose to boot off of the hard drive. You should get the GRUB menu and Fedora should boot!
- To be sure that this will survive kernel updates, once you've successfully booted into Fedora, try this and make sure it still boots:
# grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg
- Go back into the Thinkpad setup utility, switch it to boot from UEFI only, and re-enable Secure Boot.
At this point, Fedora should show up in the boot menu that appears when you hit F12, and it can be set as the default using the setup utility. To boot Windows, either choose the Windows Boot Manager from the boot menu, or choose Windows from the GRUB menu. Either way works.
Aside from this painful series of steps which almost made me want to give up and just wipe Windows off of it completely, the T560 actually works really well with Fedora. I guess Linux isn't what it was 15 years ago — wifi, audio, suspend, OpenGL, display brightness, everything just works fine out of the box. No screwing with wpa_supplicant
, xorg.conf
, or any of that! And the battery life on this machine is magnificent!