Posts Tagged ‘linux’

VirtualBox Images CentOS keyboard settings problem

I was using readymade CentOS images for VirtualBox from VirtualBoxes CentOS images. I noticed that some of the images come with weird keyboard settings, such that you can’t use the pipe (passing information from command to command), dash (for command parameters), @ (email sign), etc keys. In other words, when you try to type these keys, other keys would appears and vice versa. All of these keys are crucial to running any commands on the CentOS server.

The problem is that the VirtualBox image stores the keyboard setting that was used when it was created. Since this server image was built in Italy, and I am located in the US, there were obviously key differences.

The obvious solution is to change it to your preferred keyboard layout. There are 2 ways of doing it:

  1. Easier way: Install system-config-keyboard through yum. The caveat is this will install 75+ other bloat packages that you probably don’t need. However the command system-config-keyboard would configure your keyboard for you.
    yum install system-config-keyboard
  2. Advanced way: Use
    loadkeys

    command to change the keys temporarily and then make a change to

    /etc/sysconfig/keyboard

    to make a permanent change (after restart).

    loadkeys us # or your own locale
    vim /etc/sysconfig/keyboard # replace KEYTABLE value to your locale

rsyncd tips for TomatoUSB/DD-WRT

I wanted to integrate a NAS with rsyncd on a TomatoUSB router (equipped with ipkg and USB hard drive connect). I also wanted this NAS to be available from the outside, so I found that the instructions online were incomplete. If you’re having problems with it, please follow these two tips:

  1. rsync cannot connect to rsyncd from within the network. This is the default setup that everyone wants, so it should just work. The problem is that the command that everyone tells us to use with rsyncd profiles (rsyncd.conf: [profilename]) is wrong, even on dd-wrt tutorial. It is missing an extra semicolon, so the command should be:
    rsync file.ext user@server::profilename/optional/path

    (Notice how profiles need 2 semicolons)

  2. rsyncd is not accessible from outside the network. I haven’t seen instructions for these. To do this, one must do 2 things:
    • Add a rule to Port Forwarding section of the UI: forward TCP port 873 (default rsyncd port) to 192.168.1.1 (the router IP/gateway).
    • run the following command that adds a rule to the iptables firewall, inside Scripts > Firewall, or run it in the command line for a quick test (but it will disappear once you restart router):
      iptables -A INPUT -j ACCEPT -p tcp --dport 873
    • For OpenWRT routers, one would either use uci or add the above rule to /etc/storage/post_iptables_script.sh and do mtd_storage.sh save.

Command line python script to get context lines on a search string

grep -B and -B flags don’t work when grep is used on the command line with readline support. So I created this little script that does work on the command line. use -h flag to learn. Here’s how I’ve used it:

ls -al | ./printcontext.py -b 1 -a 1 -d test.txt

This finds the test.txt file and prints the 2 files around it.


#!/usr/bin/python
# print context when using a python script with readline support (command line piping)
# by inderpreetsingh.com

import sys, re
from optparse import OptionParser

def main():
    usage = "usage: %prog [options] needle"
    parser = OptionParser(usage)
    parser.add_option("-b", "--before", type="int", dest="before", default=0,
            help='Before context lines (a la grep)')
    parser.add_option("-a", "--after", type="int", dest="after", default=0,
            help='After context lines')
    parser.add_option("-d", "--debug", action="store_true", dest="debug", default=False,
            help='Debug information.')

    #Not implemented
    #parser.add_option("-o", "--output", type="string", dest="output")
    
    (options, args) = parser.parse_args()
    
    if len(args) != 1:
        parser.error("Specify what you want to search")
    
    needle = args[0]
    if options.debug:
        print "\nNeedle: %s\nBefore context lines: %s\nAfter context lines: %s\n" % (needle, options.before, options.after)
    

    lines = sys.stdin.readlines()
    lines = [x.strip() for x in lines]
    
    lastline = ''
    i = 0
    for line in lines:
        if needle in line:
#        if re.search(needle, line):
            first = max(0, i - options.before)
            last = min(len(lines), i + options.after + 1)
            
            if options.debug:
                print "Found '%s' on line %d, printing line %d to %d" % (needle, i, first, last)
                
            for println in lines[first:last]:
                print println
            print ""
        lastline = line
        i += 1

if __name__ == "__main__":
    main()

SSH private/public key auth not working

Problem: I can’t set up an automated login (passwordless with ssh agent) to one of my servers.

Tip: Best way to debug SSH problems is by using ssh -vvvv server. The extra verbosity flags will tell you exactly what is going on at each interaction.

Details:
I was receiving the following code:
debug1: Trying private key: /Users/inderpreetsingh/.ssh/id_rsa
debug1: PEM_read_PrivateKey failed
debug1: read PEM private key done: type
debug3: Not a RSA1 key file /Users/inderpreetsingh/.ssh/id_rsa.
debug1: read PEM private key done: type RSA
Identity added: /Users/inderpreetsingh/.ssh/id_rsa (/Users/inderpreetsingh/.ssh/id_rsa)
debug1: read PEM private key done: type RSA
debug3: sign_and_send_pubkey
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: publickey,password,hostbased

debug1: Trying private key: /Users/inderpreetsingh/.ssh/id_dsa
debug3: no such identity: /Users/inderpreetsingh/.ssh/id_dsa
debug2: we did not send a packet, disable method

debug3: authmethod_lookup password
debug3: remaining preferred: ,password
debug3: authmethod_is_enabled password
debug1: Next authentication method: password
inderpreetsingh@server's password:

Analysis: The errors are misleading. They seem to indicate that the identity file on our own machine is the culprit. But the problem was the .ssh directory and the authorized_keys file permissions. They may be too lax or too restrictive.

Fix: From your home directory, fire the following permissions:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

And for good measure, make sure you alone own the files:

chown username:username ~/.ssh
chown username:username ~/.ssh/authorized_keys

And passwordless SSH here I come.

Django on CentOS Python 2.6 VirtualEnv Using GeekyMedia RPMs

Django on centos geekymedia

for setuptools (easy_install):

wget http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.egg#md5=bfa92100bd772d5a213eedd356d64086
easy_install *setuptools*

use it to install pip:
easy_install pip

download MySQLdb and install by:
python26 setup.py build
python26 setup.py install

download virtualenv
mkdir ~/.virtualenvs

add to .bashrc
VIRTUALENVWRAPPER_PYTHON=/usr/bin/python26
source /usr/bin/virtualenvwrapper.sh

initialize virtualenv
mkdir dev
virtualenv dev

Start virtualenv for current session
source dev/bin/activate

now install packages, they will go inside virtualenv (since we are activated)
pip install django
pip install south
pip install pil

Create django project and app
cd dev/
django-admin.py startproject myproj
cd myproj
python manage.py startapp polls

Punjabi Chat iPhone app version 1

I have been running PunjabiJanta.com for over three years. At times, it has grown as a community and at times, it has shrunk in size. I rarely get time to work on the features of the website, so I felt like I owed the site something special, something the other Punjabi social networking websites lacked.

One of the most widely used features of Punjabi Janta is the PJ Chat. It has been the focus point of Punjabi Janta from the start. In fact I would go so far as to argue that it has been the cornerstone of PJ. So to propel that cornerstone, I have developed the Punjabi Chat iPhone app, which requires a PJ account to work.

Features:

  • Chat with users of PJ and on iPhone
  • Private message users
  • Smileys
  • Status: Away, BRB, Busy, Online
  • Rooms

Screenshots:

PS: You must have a activated (email authenticated) account on PJ for it to work. Please sign up now at PunjabiJanta.com.

Vim’s powerful navigation

I was reading about the Mac OSX Terminal app and somebody complained about vim being slow. A vim programmer shed some light on how to use vim’s amazing keyboard navigation:

Using Vim in Terminal.app, I can easily scroll about 5100 lines in 10 seconds, using ctrl-F or ctrl-B (scroll one page forward or back), if you’re trying to move a long distance in a hurry. I get the same speed in MacVim (which can update the screen much faster than Terminal.app), so I’m pretty certain that the limiting factor (at least in MacVim) is the keyboard repeat rate, not Vim’s ability to update the screen.

The only way I get roughly 120 lines in 10 seconds, is scrolling Vim (or MacVim) line-at-a-time by holding down “j” or the down arrow, and that’s clearly the default keyboard repeat rate (about 12 per second) limiting things.

You can easily adjust the keyboard repeat rate in System Preferences, in the Keyboard pane, by fiddling with “Key Repeat Rate” and “Delay Until Repeat”. But I’ve been using Vi/Vim/MacVim for a loooong time, and have never felt the need to have the keyboard repeat faster; the key (no pun intended) is to use Vim as it was intended…

If you want to move 3 lines down (or up) hit “j” (or “k”) 3 times (much faster than waiting for key repeat to kick in). If you want to move 10 or 20 lines down, then maybe hold down “j” (“k”) and let it repeat. But if you want to go further, that’s the wrong way: ctrl-D (ctrl-U) will take you half a screen Down (Up), ctrl-F (ctrl-B) will take you a full screen Forward/down (Back/up)… holding any of these down will get you somewhere fast. If you’re trying to get to somewhere in particular: “]]” (“[[") will jump forward (backward) to the start of the next (previous) function (in C/Obj-C/Perl/Java/Javascript or any other language where "{" by itself at the start of a line indicates the start of a function), or use "/" ("?") to search for some text and "n" to jump to subsequent occurrences until you reach the right one. Or use tags (see ":help tags"), build a tags file for your code, then you can use ":tag yourfunctionname" to jump to a function definition in any file, or sit on the function's name in a call somewhere and hit ctrl-] to jump to the definition (and ctrl-T to jump back to the call). Or even more lazily, in Vim, if you sit on any identifier and hit “*” (“#”), Vim will select that identifier as search text and take you to the next (previous) occurrence, where you can use “n” to keep searching further — this is more than sufficient in smaller files. And turn on incremental searching and search highlighting (“:set incsearch hlsearch”, worth putting in your ~/.vimrc), then the search results will be more obvious on the screen.

Finally, nearly everything in Vim (and Vi before it) takes a repeat count argument. If what you want is really to jump 179 lines down, then type “179j” (and if you wanted to get to line 423, that’s “423G”). Use the full power of Vim as it was intended, and keyboard repeat rate will never be a limiting factor.

And if you’re using Vim on the Mac extensively, you owe it to yourself to check out MacVim — you can still start it from Terminal.app (an included helper script lets you type “mvim file1 [ file2... ]” at a command line prompt), but it’s much more Mac-like — cut’n’paste and drag’n’drop interact well with other programs, you get real scroll bars (drag those to move through a file really quickly), multiple editing windows, and you can use the mouse for selecting text, along with (practically) unlimited colors for color syntax highlighting (you do use that, right? “:help syntax” or just “:syn on” to switch it on), instead of the 16 colors you’re limited to in Terminal.app. And it’s fast.

Vim (and MacVim) is the ultimate programmer’s editor, if you take the time to really learn how to use it — it requires the smallest amount of hand movement to make changes (especially since one never has to reach for the mouse), it’s insanely customizable/programmable, and it has a very smartly laid out command set — yes, it has a steep learning curve, but for something that I use 10 hours a day, I’ll take easy/powerful to use over easy to learn any day.

Cheers,
Carl

Source: MacRumors

Mplayer’s useless warnings at startup

Installed an mplayer rpm from RPMForge repository (mplayer.i386 1.0-0.40.svn20090711.el5.rf) to my CentOS 5 server. I was setting up a bash script to decode and encode several media files. Everytime mplayer was called from the script, I would get the following useless warnings:

MPlayer SVN-r29417-4.1.2 (C) 2000-2009 MPlayer Team
Can't open joystick device /dev/input/js0: No such file or directory
Can't init input joystick
mplayer: could not connect to socket
mplayer: No such file or directory
Failed to open LIRC support. You will not be able to use your remote control.

It was building up my logs. So I put the following two lines in my mplayer configuration file at /etc/mplayer/mplayer.conf:
nolirc=yes
nojoystick=yes

And like linux magic, the errors on each load disappear.

ls fails with Argument list too long

This problem usually occurs when ls is used with a wildcard, like this:

ls *test* | wc -l

Bash first expands the wildcard expression first, and when that list gets too large, the kernel cannot handle it (a limitation of the kernel). So to fix it, use the find command instead, like this:

find . -name "*test*" | wc -l

Helpful Linux Tips

Command line

Download all specified extension files from an html page:

wget -r -t1 -N -np -A.mp3 http://google.com/music/audio/

-np dont ascend to parent
-r recursive
-l1 level DONT NEED
-N timestamping
-nd no directories DONT NEED
-t 1 = tries
-H span across hosts

To remove quotas, edit /etc/fstab and remove grpquota,usrquota,
then execute the remount, replacing /home with the name:

mount -o remount /home

Convert unix timestamp to readable format in Bash

date -d @1280565192

Kill multiple processes using grep:

kill -9 `ps aux | grep perl | grep nobody | awk '{print $2}'`

Xargs: handle spaces and punctuation properly:

xargs -0

Using ack and sed, edit files in place

sed -i 's/replacestring/replacedwiththis/g' `~/bin/ack --php "searchstring" -l`

Rename doesn’t support renaming with a dash/hyphen, so we must use this forloop/mv hack:

for i in ./*foo*;do mv -- "$i" "${i//test test2/test - test2}";done

Find Command

COPY files less than 24 hours old to /some/other/directory

find . -type f -ctime -1 | xargs -I {} cp {} /some/other/directory

MOVE files less than 24 hours old to /some/other/directory

find . -type f -ctime -1 | xargs -I {} mv {} /some/other/directory

Scan files for certain text

find dir/ -name "*.txt" -exec grep -Hn "md5_func" {} \;

Find all directories and sub-directories that are empty.

find ./ -type d -empty

MySQL

Using mysql from command line, here’s how to save results to an outfile (in interactive mode):

SELECT * INTO outfile '/tmp/sql.out' FROM tablename WHERE condition  = '1';

Using mysql from command line, here’s how to save resultset to an outfile (using a sql file):

mysql database -u username -p < batch.sql > sql.out
Return top