Category Archives: Helpful Tips

Helpful Tips

MySQL: Drop all tables with prefix

1 Comment

I use the mysql command line tool to administrate my MySQL. I needed a quick way to drop all tables with the following prefix.

SELECT CONCAT('DROP TABLE `', TABLE_SCHEMA, '`.`', TABLE_NAME, '`;') FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME LIKE 'prefix%' and TABLE_SCHEMA = 'dbname';

Then all you have to do is take a text editor and replace all the “|” signs around the result.

PS: As you can see the query can be modified to get all tables with suffix, or all tables in a schema, or TRUNCATE all tables in the criteria.

Helpful Tips

Yum install rpm files manually

Leave a comment

I love yum on CentOS/Red Hat/Fedora. It saves so much of my time installing software on servers. Sometimes, the need comes when I cannot find the package inside the usual “safe” repositories, such as RHEL repos and RPMForge. So I search it and find it on pbone, rpmfind, or epel packages. I have always tried installing with rpm, but often end up with dependencies problems.

Example:

# rpm -Uvh icecast-2.3.2-4.el5.x86_64.rpm 
warning: icecast-2.3.2-4.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 217521f6
error: Failed dependencies:
        libogg.so.0()(64bit) is needed by icecast-2.3.2-4.el5.x86_64
        libspeex.so.1()(64bit) is needed by icecast-2.3.2-4.el5.x86_64
        libtheora.so.0()(64bit) is needed by icecast-2.3.2-4.el5.x86_64
        libtheora.so.0(libtheora.so.1.0)(64bit) is needed by icecast-2.3.2-4.el5.x86_64
        libvorbis.so.0()(64bit) is needed by icecast-2.3.2-4.el5.x86_64

Then I would have to go hunt down each of those dependencies. Too much time wasted. If only you could use yum to resolve those “resolvable” dependencies (using rpmforge/rhel repositories). Well you can:

# yum localinstall icecast-2.3.2-4.el5.x86_64.rpm
    However, there are two caveats:

  1. Sometimes yum can’t find the dependencies, so you must search those out. (obvious)
  2. You probably haven’t imported the gpgcheck for the rpm package that you are trying to download.
    Example:

    warning: rpmts_HdrFromFdno: Header V3 DSA signature: NOKEY, key ID 217521f6
Public key for icecast-2.3.2-4.el5.x86_64.rpm is not installed

    So you can either import it, or you can ignore the gpg check by doing this

    # yum --nogpgcheck localinstall icecast-2.3.2-4.el5.x86_64.rpm
Helpful Tips

Installing VirtualMin – “Error: Missing Dependency: libGeoIP.so.1 is needed by package proftpd”

Leave a comment

On my Red Hat 5.5 box (similar to CentOS), I was trying to install Virtualmin.

I got two errors while installing.

#1.
-This system is not registered with RHN.
RHN support will be disabled.
\Error: Missing Dependency: ruby-rdoc is needed by package rubygems-0.9.2-1.el5.noarch (virtualmin)
Error: Missing Dependency: perl(XML::Parser) is needed by package perl-XML-Simple-2.14-4.fc6.noarch (virtualmin)
Error: Missing Dependency: libdistcache.so.1()(64bit) is needed by package 2:mod_ssl-2.2.3-43.3.vm.x86_64 (virtualmin)
Error: Missing Dependency: libaprutil-1.so.0()(64bit) is needed by package 1:httpd-2.2.3-43.3.vm.x86_64 (virtualmin)
Error: Missing Dependency: libnal.so.1()(64bit) is needed by package 2:mod_ssl-2.2.3-43.3.vm.x86_64 (virtualmin)
This simply meant that I had to run rhn_register to register using the Red Hat login/password that I had received. Because you cannot run yum/update without registering with RHN, unlike CentOS which can update without registering. Simple enough.

#2.
Error: Missing Dependency: libGeoIP.so.1 is needed by package proftpd
This happens because apparently Virtualmin updated to proftpd from 1.3.2 to 1.3.3 without testing properly (source: Virtualmin Forums). To fix, I installed RPMForge and ran install script again.

Helpful Tips

Django on CentOS Python 2.6 VirtualEnv Using GeekyMedia RPMs

Leave a comment

Django on centos geekymedia

for setuptools (easy_install):

wget http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.egg#md5=bfa92100bd772d5a213eedd356d64086
easy_install *setuptools*

use it to install pip:
easy_install pip

download MySQLdb and install by:
python26 setup.py build
python26 setup.py install

download virtualenv
mkdir ~/.virtualenvs

add to .bashrc
VIRTUALENVWRAPPER_PYTHON=/usr/bin/python26
source /usr/bin/virtualenvwrapper.sh

initialize virtualenv
mkdir dev
virtualenv dev

Start virtualenv for current session
source dev/bin/activate

now install packages, they will go inside virtualenv (since we are activated)
pip install django
pip install south
pip install pil

Create django project and app
cd dev/
django-admin.py startproject myproj
cd myproj
python manage.py startapp polls

Helpful Tips

Install Python 2.6 on CentOS 5.x

1 Comment

Steps to success:

  1. Download all the Python 2.6 rpm for your CentOS (i386 or x86_64) from geekymedia.
  2. Install tcl, tk, tix (required dependencies): yum -y install tcl tk tix
  3. Install the geekymedia rpms.

    Note that the python26-libs-2.6-geekymedia1.*.rpm and python26-2.6-geekymedia1.*.rpm must be installed together like this rpm -Uvh python26-libs-2.6-geekymedia1.*.rpm python26-2.6-geekymedia1.*.rpm. Similarly, *tools* and *tkinter* rpm must be installed together like this rpm -Uvh *tools*rpm *tkinter*rpm.

Helpful Tips

Yum update stops resolving dependencies for clamav-server

1 Comment

On my CentOS/VirtualMin server configured for RPMForge, I wanted to run yum update, but I was getting the following errors while resolving dependencies for the clam* packages (you can list them by typing: rpm -qa | grep clam.

clamav-server-sysv-0.96-1.vm.el5.i386 from installed has depsolving problems
--> Missing Dependency: clamav-server = 0.96-1.vm.el5 is needed by package clamav-server-sysv-0.96-1.vm.el5.i386 (installed)
Error: Missing Dependency: clamav-server = 0.96-1.vm.el5 is needed by package clamav-server-sysv-0.96-1.vm.el5.i386 (installed)

I’ve had this problem on other servers where I had simply ignored it. But it was getting too much. So I researched and found the following solution working perfectly.

Solution

  1. Remove the clam rpms without removing the dependencies (virtualmin-base) by typing: rpm --nodeps -e clamav-0.96-1.vm.el5 and rpm --nodeps -e clamav-server-sysv-0.96-1.vm.el5.
  2. Run yum update

And voila, I get the following:

Dependencies Resolved

=====================================================================================================================================================================================================
 Package                                                  Arch                                    Version                                            Repository                                 Size
=====================================================================================================================================================================================================
Installing:
 clamav                                                   i386                                    0.96.4-1.el5.rf                                    rpmforge                                  2.2 M
     replacing  clamav-lib.i386 0.96-1.vm.el5

 clamav-db                                                i386                                    0.96.4-1.el5.rf                                    rpmforge                                   25 M
     replacing  clamav-update.i386 0.96-1.vm.el5

 clamd                                                    i386                                    0.96.4-1.el5.rf                                    rpmforge                                  233 k
     replacing  clamav-server.i386 0.96-1.vm.el5

Another solution: I have had servers that had conflicts even on the yum update, because maybe dependent packages came from conflicting repos. Specifically, clamav dependencies of rpmforge conflicted with clamav dependencies of virtualmin. To solve this, I ran: yum --disablerepo=* --enablerepo=virtualmin install clamav clamav-filesystem clamav-data clamav-lib

Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package clamav.x86_64 0:0.96.5-1.vm.el5 set to be updated
---> Package clamav-data.x86_64 0:0.96.5-1.vm.el5 set to be updated
---> Package clamav-filesystem.x86_64 0:0.96.5-1.vm.el5 set to be updated
---> Package clamav-lib.x86_64 0:0.96.5-1.vm.el5 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

===========================================================================================================================================
 Package                               Arch                       Version                             Repository                      Size
===========================================================================================================================================
Installing:
 clamav                                x86_64                     0.96.5-1.vm.el5                     virtualmin                     934 k
 clamav-data                           x86_64                     0.96.5-1.vm.el5                     virtualmin                      25 M
 clamav-filesystem                     x86_64                     0.96.5-1.vm.el5                     virtualmin                      18 k
 clamav-lib                            x86_64                     0.96.5-1.vm.el5                     virtualmin                     4.4 M

Transaction Summary
===========================================================================================================================================
Install       4 Package(s)
Upgrade       0 Package(s)

Total download size: 31 M
Is this ok [y/N]: y
Downloading Packages:
(1/4): clamav-filesystem-0.96.5-1.vm.el5.x86_64.rpm                                                                 |  18 kB     00:00     
(2/4): clamav-0.96.5-1.vm.el5.x86_64.rpm             (0%) 17% [=======                                   ]  0.0 B/s | 160 kB     --:-- ETA 
(3/4): clamav-lib-0.96.5-1.vm.el5.x86_64.rpm         (3%)  3% [=-                                        ]  0.0 B/s | 168 kB     --:-- ETA 
(4/4): clamav-data-0.96.5-1.vm.el5.x86_64.rpm       (17%)  0% [                                          ]  0.0 B/s |  40 kB     --:-- ETA 
-------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                      914 kB/s |  31 MB     00:34     
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing     : clamav-filesystem                                                                                                   1/4 
  Installing     : clamav-data                                                                                                         2/4 
  Installing     : clamav-lib                                                                                                          3/4 
  Installing     : clamav                                                                                                              4/4 

Installed:
  clamav.x86_64 0:0.96.5-1.vm.el5            clamav-data.x86_64 0:0.96.5-1.vm.el5        clamav-filesystem.x86_64 0:0.96.5-1.vm.el5       
  clamav-lib.x86_64 0:0.96.5-1.vm.el5

Along with the above solution, you must remember to disable the conflicting repository. In this case, rpmforge. If rpmforge ever has a newer version of one of the clamav packages, and anyone runs yum update, the dependencies will again stop resolving (because rpmforge and virtualmin repositories do not have a symbiotic relationship). To disable rpmforge, edit /etc/yum.repos.d/rpmforge.repo and add/replace enabled=0 to the [rpmforge] section. Also note, that anytime you want to explicitly use rpmforge, you can do yum --disablerepo=* --enablerepo=rpmforge (or if you want to use it in conjunction with other enabled repositories, remove the –disablerepo flag).

Helpful Tips

Nginx with m4a/aac/mp4 seek support on CentOS 5

2 Comments

I was working on Sikh Sangeet website which serves a lot of static audio media files. For the past couple of months, we’ve been paying overage (i.e. bandwidth charges for surpassing limits). I either had the option of paying much more (hundreds) for a better server or keep paying the overage charges (usually less than a hundred). So, I wanted to convert all the listening audio to a smaller format, such that I could provide full quality mp3s for downloading and provide smaller format audio for listening. I ended up choosing H264’s aac-HE, because it is supported in flash, android and iphone.

Previously on our dedicated Sikh Sangeet server, I ran nginx because the server was very limited with hardware, yet needed to serve the huge files that were mp3s. I had the latest nginx rpm (0.6.39-5.el5) from epel setup, but to get mp4 support, I had to install nginx from source with mp4 support.

I will reproduce the steps in the hopes that if I or anybody needs to do this in the future, they can get a quick idea:

Install nero encoders (best encoders for aac).
Convert mp3 to wav: mplayer -vc null -vo null "orig_file.mp3" -ao pcm:fast:file="/tmp/out.wav"
Convert wav to aac/m4a: neroAacEnc -br 56000 -if /tmp/out.wav -of "new_file.m4a"
Tag the m4a file with proper info: neroAacTag "new_file.m4a" -meta:title="title" -meta:artist="artist" -meta:track="track" -meta:album="album" -meta:genre="genre"

Now we have a m4a (aac) file, but this file is still not interleaved like an mp4 should be for nginx to stream parts of it. We must now convert the m4a (aac) file to mp4.

Install MP4Box (best way to convert regular audio into mp4 containers).
Convert m4a to mp4: MP4Box -add new_file.m4a:sbr final_file.mp4

Now we have our audio part ready. Now we just need to setup nginx:
Use the compile/make instructions from the original creators of the mp4 streaming module for nginx.

Notes:

  • If using the latest nginx server, the configure statement will not work. For two reasons:
    1. The nginx source requires pcre, even if you have pcre and pcre-dev(el) packages installed. For some reason nginx ./configure wasn't picking them up, so I downloaded pcre from pcre ftp. And without running make/configure or anything on it, used the following ./configure statement for nginx: ./configure --add-module=$HOME/nginx_mod_h264_streaming-2.2.7 --sbin-path=/usr/local/sbin --with-debug --with-pcre=$HOME/pcre-8.10
    2. The nginx_mod_h264_streaming-2.2.7 module has an incompatibility with the latest nginx (confirmed with 0.8.53):
      nginx_mod_h264_streaming-2.2.7/src/ngx_http_streaming_module.c:158: error: ângx_http_request_tâ has no member named âzero_in_uriâ.
      So, you must patch the ngx_http_streaming_module.c file using this patch.
  • For Red Hat/CentOS/Fedora users: If you want to use service nginx start/stop/reload/etc, download this Nginx Init Script.

Once you have nginx ready serving regular files from your document root (public_html, www, etc). Then simply put this directive inside the server context:

location ~ \.mp4$ {
root /your/document/root/full/path/here;
mp4;
}

The mp4 line will take care of the final_file.mp4?start=25 variable (i.e. start at 25 seconds) that flash players send to the server and cut off the mp4 at the right place.

Helpful Tips

Vim’s powerful navigation

Leave a comment

I was reading about the Mac OSX Terminal app and somebody complained about vim being slow. A vim programmer shed some light on how to use vim’s amazing keyboard navigation:

Using Vim in Terminal.app, I can easily scroll about 5100 lines in 10 seconds, using ctrl-F or ctrl-B (scroll one page forward or back), if you’re trying to move a long distance in a hurry. I get the same speed in MacVim (which can update the screen much faster than Terminal.app), so I’m pretty certain that the limiting factor (at least in MacVim) is the keyboard repeat rate, not Vim’s ability to update the screen.

The only way I get roughly 120 lines in 10 seconds, is scrolling Vim (or MacVim) line-at-a-time by holding down “j” or the down arrow, and that’s clearly the default keyboard repeat rate (about 12 per second) limiting things.

You can easily adjust the keyboard repeat rate in System Preferences, in the Keyboard pane, by fiddling with “Key Repeat Rate” and “Delay Until Repeat”. But I’ve been using Vi/Vim/MacVim for a loooong time, and have never felt the need to have the keyboard repeat faster; the key (no pun intended) is to use Vim as it was intended…

If you want to move 3 lines down (or up) hit “j” (or “k”) 3 times (much faster than waiting for key repeat to kick in). If you want to move 10 or 20 lines down, then maybe hold down “j” (“k”) and let it repeat. But if you want to go further, that’s the wrong way: ctrl-D (ctrl-U) will take you half a screen Down (Up), ctrl-F (ctrl-B) will take you a full screen Forward/down (Back/up)… holding any of these down will get you somewhere fast. If you’re trying to get to somewhere in particular: “]]” (“[[“) will jump forward (backward) to the start of the next (previous) function (in C/Obj-C/Perl/Java/Javascript or any other language where “{” by itself at the start of a line indicates the start of a function), or use “/” (“?”) to search for some text and “n” to jump to subsequent occurrences until you reach the right one. Or use tags (see “:help tags”), build a tags file for your code, then you can use “:tag yourfunctionname” to jump to a function definition in any file, or sit on the function’s name in a call somewhere and hit ctrl-] to jump to the definition (and ctrl-T to jump back to the call). Or even more lazily, in Vim, if you sit on any identifier and hit “*” (“#”), Vim will select that identifier as search text and take you to the next (previous) occurrence, where you can use “n” to keep searching further — this is more than sufficient in smaller files. And turn on incremental searching and search highlighting (“:set incsearch hlsearch”, worth putting in your ~/.vimrc), then the search results will be more obvious on the screen.

Finally, nearly everything in Vim (and Vi before it) takes a repeat count argument. If what you want is really to jump 179 lines down, then type “179j” (and if you wanted to get to line 423, that’s “423G”). Use the full power of Vim as it was intended, and keyboard repeat rate will never be a limiting factor.

And if you’re using Vim on the Mac extensively, you owe it to yourself to check out MacVim — you can still start it from Terminal.app (an included helper script lets you type “mvim file1 [ file2… ]” at a command line prompt), but it’s much more Mac-like — cut’n’paste and drag’n’drop interact well with other programs, you get real scroll bars (drag those to move through a file really quickly), multiple editing windows, and you can use the mouse for selecting text, along with (practically) unlimited colors for color syntax highlighting (you do use that, right? “:help syntax” or just “:syn on” to switch it on), instead of the 16 colors you’re limited to in Terminal.app. And it’s fast.

Vim (and MacVim) is the ultimate programmer’s editor, if you take the time to really learn how to use it — it requires the smallest amount of hand movement to make changes (especially since one never has to reach for the mouse), it’s insanely customizable/programmable, and it has a very smartly laid out command set — yes, it has a steep learning curve, but for something that I use 10 hours a day, I’ll take easy/powerful to use over easy to learn any day.

Cheers,
Carl

Source: MacRumors

Helpful Tips

Javascript/JSON: Find Index in an Array of Objects

Leave a comment

JSON is one of my favorite data models notation (xml = :(). A lot of times, I get arrays of objects (hash tables) that I then have to search again an again to find the index of the object inside that array. Here’s a quick function to find it:


/*
function findIndexByKeyValue: finds "key" key inside "ob" object that equals "value" value
example: findIndexByKeyValue(students, 'name', "Jim");
object: students = [
   {name: 'John', age: 100, profession: 'Programmer'},
   {name: 'Jim', age: 50, profession: 'Carpenter'}
];
would find the index of "Jim" and return 1
*/

function findIndexByKeyValue(obj, key, value)
{
	for (var i = 0; i < obj.length; i++) {
		if (obj[i][key] == value) {
			return i;
		}
	}
	return null;
}