Add Lets Encrypt for Webmin/Virtualmin Panel itself

Are you getting a “challenge did not pass” error? Can’t find a matching hostname for your Webmin panel to generate Let’s Encrypt certificates to?

The trick is that Lets Encrypt only works with Apache. Most Webmin/Virtualmin panels are running on miniserv Perl scripts on port 10000. We must follow the following steps:

1. Create a virtual server (or sub server) for the matching domain in Virtualmin.
2. Go to Webmin > Webmin Configuration > SSL Encryption > Let’s Encrypt
3. Request Certificate for “Apache virtual host matching hostname”. Set renewal for 12 months!

Reload problems for Raspberry Pi as a Digital Signage solution

There are quite a few tutorials out there to turn Raspberry Pi into Digital Signage or Kiosk solutions. I’ve used elalemanyo GitHub gist to run Chromium browser for my Digital Signage needs. It provides lots of options and combines various sources of information. There are options for other browsers on that page.

The gist does not address a way to automatically reload/recover from page/chrome/connection errors. You can use the following two extensions to recover from these problems:

1. Oh No You Didn’t extension will automatically reload tabs that have crashed (Aw Snap! He’s Dead Jim etc.). Handy for Kiosk mode where access is limited. This helps recover from memory related problems (quite often the Raspberry Pi’s are limited by memory or CPU).

2. Autorefresh on Error will auto-refresh a page after 60 seconds if page did not load. This is handy for any webpage that changes often and needs to be refreshed. And for places where the WiFi coverage may not be ideal, especially on boot.

Install latest Chromium on Raspberry Pi B+

Update on December 26, 2016: Latest Raspbian can install Chromium Browser by just doing: apt-get install chromium-browser. No need to download debs or run gdebi. Making this post obsolete.

Installing the latest Chromium browser on Raspberry Pi B models is a hassle. Although the following tricks work on the other Raspberry Pi models. I wanted to target Raspberry Pi B+ with this tutorial since it requires some special steps.

  1. Download the latest .deb package files:
  2. Ensure you have gdebi:
    sudo apt-get install gdebi-core
  3. Install the packages one by one (install libgcrypt11 first!!!):
    sudo gdebi libgcrypt11
    sudo gdebi chromium-codecs-ffmpeg-extra
    sudo gdebi chromium-browser
    
  4. Run:
    chromium-browser

Kodi’s Pulsar errors starting up

If you are seeing the following Pulsar errors, check the possible solutions:

Problem: Trying to launch Pulsar

21:31:27 T:2440  NOTICE: [plugin.video.pulsar] 2015-02-03 21:31:27  INFO  btservice        Starting DHT...
21:31:27 T:2440  NOTICE: [plugin.video.pulsar] 2015-02-03 21:31:27  INFO  btservice        Starting LSD...
21:31:27 T:2440  NOTICE: [plugin.video.pulsar] 2015-02-03 21:31:27  INFO  btservice        Starting UPNP...
21:31:27 T:2440  NOTICE: [plugin.video.pulsar] 2015-02-03 21:31:27  INFO  btservice        Starting NATPMP...
21:31:27 T:4984  NOTICE: Thread LanguageInvoker start, auto delete: false
21:31:27 T:4984  NOTICE: -->Python Interpreter Initialized<--
21:31:27 T:4984  NOTICE: http://localhost:65251/
21:31:28 T:4984   ERROR: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                             - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                            Error Type: <class 'urllib2.URLError'>
                                            Error Contents: <urlopen error [Errno 10061] No connection could be made because the target machine actively refused it>
                                            Traceback (most recent call last):
                                              File "C:\Users\TVComputer\AppData\Roaming\Kodi\addons\plugin.video.pulsar\navigation.py", line 6, in <module>
                                                navigation.run()
                                              File "C:\Users\TVComputer\AppData\Roaming\Kodi\addons\plugin.video.pulsar\resources\site-packages\pulsar\navigation.py", line 72, in run
                                                data = _json(url)
                                              File "C:\Users\TVComputer\AppData\Roaming\Kodi\addons\plugin.video.pulsar\resources\site-packages\pulsar\navigation.py", line 40, in _json
                                                with closing(urllib2.urlopen(url)) as response:
                                              File "C:\Program Files (x86)\Kodi\system\python\Lib\urllib2.py", line 127, in urlopen
                                                return _opener.open(url, data, timeout)
                                              File "C:\Program Files (x86)\Kodi\system\python\Lib\urllib2.py", line 404, in open
                                                response = self._open(req, data)
                                              File "C:\Program Files (x86)\Kodi\system\python\Lib\urllib2.py", line 422, in _open
                                                '_open', req)
                                              File "C:\Program Files (x86)\Kodi\system\python\Lib\urllib2.py", line 382, in _call_chain
                                                result = func(*args)
                                              File "C:\Program Files (x86)\Kodi\system\python\Lib\urllib2.py", line 1214, in http_open
                                                return self.do_open(httplib.HTTPConnection, req)
                                              File "C:\Program Files (x86)\Kodi\system\python\Lib\urllib2.py", line 1184, in do_open
                                                raise URLError(err)
                                            URLError: <urlopen error [Errno 10061] No connection could be made because the target machine actively refused it>
                                            -->End of Python script error report<--
21:31:29 T:4548   ERROR: XFILE::CDirectory::GetDirectory - Error getting plugin://plugin.video.pulsar/
21:31:29 T:4548   ERROR: CGUIMediaWindow::GetDirectory(plugin://plugin.video.pulsar/) failed
21:31:29 T:764  NOTICE: Thread BackgroundLoader start, auto delete: false

Solution
Wait for Pulsar to start up. It usually takes about 1-2 minutes for it to load after Kodi has been enabled.

Problem: Deleting missing stale files

17:08:01 T:139736123422464 NOTICE: [plugin.video.pulsar] Deleting stale files set(['C:\\path\\to\\file\\filename.mp4'])

Solution

  1. Go to %APPDATA%\Kodi\ directory.
  2. Delete the cache directory.
  3. Delete the userdata\addon_data\plugin.video.pulsar\cache directory

MySQL CSV import skipping rows

I was experiencing problems with loading CSV files into MySQL tables. I noticed that it was skipping every other row in most cases. I was using the following SQL code to load the CSV file:

LOAD DATA LOCAL INFILE 'test.csv'
INTO TABLE games
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 0 LINES;

Solution:
It turns out that the CSV file that I was loading had line endings that my MySQL client did not recognize. I found out that the CSV file was coming from a Windows machine while I was trying to load it onto a Mac OSX machine’s MySQL client/server.

To fix this, I ran the following conversion command, which removed the Windows specific line endings:

cat test.csv | tr -d '\r' > "test.fixed.csv"

I verified that this fixed the file by opening it up in TextWrangler and noting the line ending style on the bottom status bar.

mDNSResponder and discoveryd hanging in Yosemite: CPU spikes at 100%

Since updating to Yosemite, I have been seeing 100% CPU spikes. You will see message (like the ones pasted below) from the Console.app. For some reason, discoveryd and mDNSResponder start going wild. Most of the instructions found online tell us to restart /System/Library/LaunchDaemons/com.apple.mDNSResponder.plist using launchctl. However in Yosemite, there is no com.apple.mDNSResponder.plist.

Solution
You must restart discoveryd like the following:

sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.discoveryd.plist
sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.discoveryd.plist

Console error messages

dnssd_clientstub ConnectToServer: connect() failed path:/var/run/mDNSResponder Socket:70 Err:-1 Errno:61 Connection refused
1/23/15 2:20:59.358 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 2
1/23/15 2:20:59.477 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect() failed path:/var/run/mDNSResponder Socket:26 Err:-1 Errno:61 Connection refused
1/23/15 2:20:59.478 PM Slack[16482]: tcp_connection_host_start 1929 DNSServiceCreateConnection failed: -65563
1/23/15 2:20:59.508 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 1
1/23/15 2:20:59.609 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 2
1/23/15 2:20:59.679 PM Google Chrome[15689]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 2
1/23/15 2:20:59.755 PM awacsd[265]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 2
1/23/15 2:20:59.774 PM coreaudiod[444]: 2015-01-23 02:20:59.774313 PM [BonjourBrowser] Retrying Bonjour start for _raop._tcp. after failure
1/23/15 2:20:59.775 PM coreaudiod[444]: 2015-01-23 02:20:59.774751 PM [BonjourBrowser] Retrying Bonjour start for _airplay._tcp. after failure
1/23/15 2:20:59.775 PM coreaudiod[444]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 1
1/23/15 2:20:59.775 PM coreaudiod[444]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 1
1/23/15 2:21:00.015 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect() failed path:/var/run/mDNSResponder Socket:15 Err:-1 Errno:61 Connection refused
1/23/15 2:21:00.015 PM Slack[16482]: tcp_connection_host_start 1930 DNSServiceCreateConnection failed: -65563
1/23/15 2:21:00.300 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 2
1/23/15 2:21:00.360 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 3
1/23/15 2:21:00.513 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 2
1/23/15 2:21:00.614 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 3
1/23/15 2:21:00.666 PM sharingd[460]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 1
1/23/15 2:21:00.685 PM Google Chrome[15689]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 3
1/23/15 2:21:00.829 PM awacsd[265]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 3
1/23/15 2:21:00.841 PM coreaudiod[444]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 2
1/23/15 2:21:00.841 PM coreaudiod[444]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 2
1/23/15 2:21:01.117 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 1
1/23/15 2:21:01.177 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 1
1/23/15 2:21:01.301 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 3
1/23/15 2:21:01.365 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect() failed path:/var/run/mDNSResponder Socket:34 Err:-1 Errno:61 Connection refused
1/23/15 2:21:01.366 PM Slack[16482]: tcp_connection_host_start 1932 DNSServiceCreateConnection failed: -65563
1/23/15 2:21:01.366 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 1
1/23/15 2:21:01.469 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 1
1/23/15 2:21:01.519 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 3
1/23/15 2:21:01.615 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect() failed path:/var/run/mDNSResponder Socket:27 Err:-1 Errno:61 Connection refused
1/23/15 2:21:01.615 PM Slack[16482]: tcp_connection_host_start 1931 DNSServiceCreateConnection failed: -65563
1/23/15 2:21:01.670 PM sharingd[460]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 2
1/23/15 2:21:01.686 PM Google Chrome[15689]: dnssd_clientstub ConnectToServer: connect() failed path:/var/run/mDNSResponder Socket:54 Err:-1 Errno:61 Connection refused
1/23/15 2:21:01.687 PM Google Chrome[15689]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 1
1/23/15 2:21:01.883 PM awacsd[265]: dnssd_clientstub ConnectToServer: connect() failed path:/var/run/mDNSResponder Socket:11 Err:-1 Errno:61 Connection refused
1/23/15 2:21:01.883 PM coreaudiod[444]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 3
1/23/15 2:21:01.883 PM coreaudiod[444]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 3
1/23/15 2:21:01.883 PM awacsd[265]: RRMetaClientItem: DNSServiceQueryRecord: -65563
1/23/15 2:21:01.884 PM awacsd[265]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 1
1/23/15 2:21:02.122 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 2
1/23/15 2:21:02.180 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 2
1/23/15 2:21:02.307 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect() failed path:/var/run/mDNSResponder Socket:30 Err:-1 Errno:61 Connection refused
1/23/15 2:21:02.307 PM Slack[16482]: tcp_connection_host_start 1933 DNSServiceCreateConnection failed: -65563
1/23/15 2:21:02.308 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 1
1/23/15 2:21:02.369 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 2
1/23/15 2:21:02.475 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 2
1/23/15 2:21:02.522 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect() failed path:/var/run/mDNSResponder Socket:26 Err:-1 Errno:61 Connection refused
1/23/15 2:21:02.522 PM Slack[16482]: tcp_connection_host_start 1934 DNSServiceCreateConnection failed: -65563
1/23/15 2:21:02.522 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 1
1/23/15 2:21:02.674 PM sharingd[460]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 3
1/23/15 2:21:02.691 PM Google Chrome[15689]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 2
1/23/15 2:21:02.938 PM awacsd[265]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 2
1/23/15 2:21:02.938 PM coreaudiod[444]: dnssd_clientstub ConnectToServer: connect() failed path:/var/run/mDNSResponder Socket:5 Err:-1 Errno:61 Connection refused
1/23/15 2:21:02.938 PM coreaudiod[444]: dnssd_clientstub ConnectToServer: connect() failed path:/var/run/mDNSResponder Socket:6 Err:-1 Errno:61 Connection refused
1/23/15 2:21:02.938 PM coreaudiod[444]: 2015-01-23 02:21:02.938188 PM [BonjourBrowser] ### Start browse for _raop._tcp. on if <any>, flags 0x7000000000100000 failed: -65563/0xFFFEFFE5 kDNSServiceErr_ServiceNotRunning
1/23/15 2:21:02.939 PM coreaudiod[444]: 2015-01-23 02:21:02.938754 PM [BonjourBrowser] ### Start browse for _airplay._tcp. on if <any>, flags 0x7000000000100000 failed: -65563/0xFFFEFFE5 kDNSServiceErr_ServiceNotRunning
1/23/15 2:21:02.939 PM coreaudiod[444]: 2015-01-23 02:21:02.939078 PM [BonjourBrowser] ### Browse for _raop._tcp. failed, retrying in 7349 ms: -65563/0xFFFEFFE5 kDNSServiceErr_ServiceNotRunning
1/23/15 2:21:02.939 PM coreaudiod[444]: 2015-01-23 02:21:02.939416 PM [BonjourBrowser] ### Browse for _airplay._tcp. failed, retrying in 7349 ms: -65563/0xFFFEFFE5 kDNSServiceErr_ServiceNotRunning
1/23/15 2:21:03.128 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 3
1/23/15 2:21:03.186 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 3
1/23/15 2:21:03.313 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 2
1/23/15 2:21:03.374 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 3
1/23/15 2:21:03.478 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 3
1/23/15 2:21:03.523 PM Slack[16482]: dnssd_clientstub ConnectToServer: connect()-> No of tries: 2

EXT4 repair on Mac OSX

My NAS WD drive recently became corrupt. It had trouble saving large files and running them. My router constantly complained about not being able to properly run Samba/swap on it.

Not having a Linux OS handy, I had a tall task to repair an EXT4 filesystem on Mac OSX 10.10.

I first tried the open source solutions: fuse-ext2, osxfuse, and ext2fuse (read-only). While these tools work great in allowing access to ext4. They do not include any disk repair tools. e2fsprogs from Homebrew also does not include the fsck utilities.

I also tried using the commercial ExtFS by Paragon. It integrates directly with Disk Utility and is able to verify/check the disk for errors. While it reported the errors properly, it had trouble repairing the filesystem and reported my ext4 partition as ext3. It failed to repair reporting: “Error: Disk Utility can’t repair this disk. Back up as many of your files as possible, reformat the disk, and restore your backed-up files.” Also, ExtFS was very buggy as in it consistently froze the System Preferences and Finder (while ejecting and mounting).

Finally, I installed VirtualBox with an Ubuntu VM and added the USB drive from settings. I had to umount the drive from inside Ubuntu and running fsck -y was a breeze. Now ExtFS by Paragon also reports that the Disk is good.

VirtualBox Images CentOS keyboard settings problem

I was using readymade CentOS images for VirtualBox from VirtualBoxes CentOS images. I noticed that some of the images come with weird keyboard settings, such that you can’t use the pipe (passing information from command to command), dash (for command parameters), @ (email sign), etc keys. In other words, when you try to type these keys, other keys would appears and vice versa. All of these keys are crucial to running any commands on the CentOS server.

The problem is that the VirtualBox image stores the keyboard setting that was used when it was created. Since this server image was built in Italy, and I am located in the US, there were obviously key differences.

The obvious solution is to change it to your preferred keyboard layout. There are 2 ways of doing it:

  1. Easier way: Install system-config-keyboard through yum. The caveat is this will install 75+ other bloat packages that you probably don’t need. However the command system-config-keyboard would configure your keyboard for you.
    yum install system-config-keyboard
  2. Advanced way: Use
    loadkeys

    command to change the keys temporarily and then make a change to

    /etc/sysconfig/keyboard

    to make a permanent change (after restart).

    loadkeys us # or your own locale
    vim /etc/sysconfig/keyboard # replace KEYTABLE value to your locale

PHP misreporting existence of directory or files

Recently I was working with a WordPress site where I was deleting a network. When the network was deleted, the root site was also deleted (or vice versa). And when the root site deleted, it deleted its own media directory and the parent network’s media directory as well. This worked out OK in most cases (because the root site and root network are one-in-the-same in a multisite network). However in some of my workplace’s custom code, we hooked into both network and site deletion. We tried to delete some extra folders attached with the network and site. Some folders were shared, so once they were deleted, they should’ve shown up as missing. However the folders were showing up as existing on second try and when deleting were giving out errors. So…

PHP has something called stat cache to make file operations faster. It caches information anytime you make a call to functions like “stat(), lstat(), file_exists(), is_writable(), is_readable(), is_executable(), is_file(), is_dir(), is_link(), filectime(), fileatime(), filemtime(), fileinode(), filegroup(), fileowner(), filesize(), filetype(), and fileperms().” Coincidentally, PHP was misreporting the existence of files and folders when they were clearly deleted and glob() would be confirming that fact. In the end, we had to add calls to clearstatcache() to avoid this trap.

I wish there was a way to disable the stat caching for a session, because calling this function every time gets to be a pain.

mysqldump and .my.cnf tip to avoid “ignoring option ‘–databases'” error

Don’t like how the mysqldump command keeps reading the .my.cnf and outputting the following warning/error:
mysqldump: ignoring option '--databases' due to invalid value 'dbname'

It’s been reported as a bug to mysql devs, but they keep saying it is by design, and working as intended. Here’s how you should structure the .my.cnf to avoid it:


[client]
user=user1
password=pw1
[mysql]
database=dbname

This way the “mysqldump dbname” doesn’t return that hideous error message and “mysql dbname” also works like a charm.