Forum for freelance software developers

I got a new forum for freelancers. They call it FreelanceFest. You can go and post your problems there. Hopefully someone will answer soon. There are language specific categories and also freelance site based categories available.

Convert FLV to MP3 using Perl

Few days back, I found an interesting Perl module that converts FLV file to MP3. Here is the code:

use FLV::ToMP3;

$flv_filename = "/home/video/music.flv";
$mp3_filename = "test.mp3";

my $converter = FLV::ToMP3->new();
$converter->parse_flv($flv_filename);
$converter->save($mp3_filename);

Visit the following links for details:

• FLV::ToMP3
• flv2mp3

Send email using PHP

Sending emails using a PHP script / program is very easy. Just use the mail function: mail($to, $subject, $body);

Check mail function for details.

There can be some problems if it's not configured properly. Check the installation / configuration page for details.

Install Perl modules using cpan

Installation of Perl modules is very simple. Just use the command in the terminal:

cpan -install ModuleName

For example, to install LWP module, use the command:

cpan -install LWP

To install Mail::Sendmail module, use the command:

cpan -install Mail::Sendmail

Perl - return value of a function

Can you guess the output of the following Perl code?

$sum = add(3, 5);
print $sum."\n";

sub add
{
    $a = shift;
    $b = shift;
    
    return ($a + $b);
}

Yes, it prints 8.

Now what about the following code?

$sum = add(3, 5);
print $sum."\n";

sub add
{
    $a = shift;
    $b = shift;    
}

hmm... what should it print? It prints 5 ! A garbage value?
Not actually. I have found that in Perl if you don't return anything, it returns the value of the last variable in the function.

Now you can tell the output of the following code :-)

$sum = add(3, 5);
print $sum."\n";

sub add
{
    $a = shift;
    $b = shift;
    
    $c = 10;
}

yes, it's 10 !

Count number of elements in array - PHP, Perl, Python

We often need to count the number of elements in an array (or get the size of an array). Here is how to do it in PHP, Perl & Python:

PHP: $items = count($ara);

Perl: my $items = scalar(@ara);, you can also try: my $items = @ara;

Python: items = ara.__len__(). Note that in Python it's called list. List can be used as array.

CGItemp**** issue in file upload

Sometimes your file upload script produces temp files named CGItemp**** in the server (this happed to one of my Perl file upload script (cgi) running in a windows server).

So I did some search in the Internet about the problem. Many people faced the same problem and most of them choose a simple solution. That is to delete those files using a program or manually. After spending some more time, I found that those temp files are filehandles that are being recorded in the server. Though the CGI.pm file should have taken care of it (i.e. delete those files automatically).

I found that in my code I have opened a file handle which must be closed in order to get rid of this problem.

my $upload_file_handle = $q->upload("file_name");

when this variable (file handle) is not required any more, it should be closed.

So I added the line at the end of upload function:

close $upload_file_handle;

It solved the problem!

Objective-C for iPhone application development

iPhone is getting popular day-by-day. Naturally the demand for iPhone applications is increasing. So if you are ready to take the next big step, you can start learning how to develop software for iPhone.

The operating system used in iPhone is called iPhone OS (which is also the operating system for iPod). Unfortunately iPhone doesn't support Java like most of the smart phones. The programming language that can be used to write programs for iPhone is Objective-C. If you already know it, then you are one step ahead. Else you can start learning it. You can read this tutorial: http://developer.apple.com/documentation/Cocoa/Conceptual/ObjectiveC/ which is recommended by many.

For resources, you must visit iPhone Developer's Center: http://developer.apple.com/iphone/ and download the iPhone SDK.

Please share your experience with iPhone application development.

Wikipedia links: iPhone, Objective-C, iPhone OS

Create PDF file in Perl

Today I tried to create a PDF file using a program. I choose the language Perl to do this task, because there should be one or more useful module for pdf creation in Perl (I shall try to do the same task using PHP and Python later).

For PDF creation I found the module PDF::Create very useful. If you want to create and/or modify a pdf file, then you can try PDF::API2.

Here is an article on pdf creation from the book PDF Hack, that uses PDF::API2 to create a pdf file: Use Perl to create PDF. Note that I had to change the sample code (line 11) my $txt = $page->hybrid; to my $txt = $page->text;

Hope you can create PDFs smoothly. Please share your experience.

Non-interactive SSH in Linux

There are two machines, A and B in a network. Let the IP of A be 192.168.1.2, and IP of B be 192.168.1.8. You can access remotely from A to B using the command: ssh guest@192.168.1.8. It will ask you the password of the user guest in machine B. Now let me tell you a simple trick so that you don't need to enter the password any more. It's called non-interactive ssh.

Suppose your username in machine A is pypy. Open the terminal. Go to the directory /home/pypy/.ssh (cd /home/pypy/.ssh). Now enter the follwoing command:
ssh-keygen -t dsa
First it will ask you for a filename. Just press enter to keep it default (id_dsa) or you can enter any other name. Then it will ask you for a passphrase. Enter empty there (just press enter twice). Now you will see two files there named id_dsa (that has the private key) and id_dsa.pub (that contains the public key).

pypy@pypy-laptop:~/.ssh$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/pypy/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/pypy/.ssh/id_dsa.
Your public key has been saved in /home/pypy/.ssh/id_dsa.pub.
The key fingerprint is:
fc:37:69:9e:bb:15:e7:55:cd:b9:52:12:e6:49:b1:6c pypy@pypy-laptop

Key generation complete. Check the directory

pypy@pypy-laptop:~/.ssh$ ls -lt
total 12
-rw------- 1 pypy pypy 668 2008-06-20 23:54 id_dsa
-rw-r--r-- 1 pypy pypy 610 2008-06-20 23:54 id_dsa.pub

Now append the content of id_dsa.pub file to the file named authorized_keys2 in /home/guest/.ssh directory (if the file doesn't exist first create it) of machine B. You can do this using this command also:
cat /home/pypy/.ssh/id_dsa.pub | ssh guest@192.168.1.8 'cat - >> ~/.ssh/authorized_keys2'

(it will ask you for the password of user guest of machine B).

Now done. Try to access machine B from machine A
pypy@pypy-laptop:~/.ssh$ ssh guest@192.168.1.8

Hopefully it won't ask you for a password :-)

Perl script to display the content of a zip file

Here is a little script that shows the number of files in a zip file and the names of the files. Interesting thing is that, we don't need to unzip the file.

Here is the code

use Archive::Zip qw( :ERROR_CODES :CONSTANTS );

my $zip = Archive::Zip->new();

my $zip_file_name = 'scraper.zip';

$zip->read( $zip_file_name );

print "Number of files in the zip file: ".$zip->numberOfMembers()."\n";
print "List of files:\n";

my @members = $zip->memberNames();

#now print the name of the files
foreach (@members)
{
   print $_, "\n";
}

For details, please check http://search.cpan.org/~adamk/Archive-Zip-1.23/lib/Archive/Zip.pm

Your crawler freezes? Use BackgroundWorker in C#

When I wrote my first crawler with GUI in C#.NET (I used .NET so that I can create the GUI easily using Visual Studio) I got into a major problem. The GUI got frozen when I ran the crawler... I shared my problem with one of my friend who then told me about BackgroundWorker Thread. Then I got back home, explored BackgroundWorker, learned how to use it, and used it in my crawler and WOW! It worked fine then. So if you get into similar problem then you can also use BackgroundWorker.

You will find it in Toolbox > Component > BackgroundWorker. Just drag it and drop on your form. Right click on it (it's backgroundWorker1 until you change the name) to change it's properties and add events. Now just code the events...

You can go through this example in msdn http://msdn.microsoft.com/en-us/library/b2zk6580.aspx. It's a useful one I think.

Problem uploading large file in CGI script

Today I had a problem uploading large file using the CGI upload script I have written using Perl couple of day back. I was getting the response: "413 Request entity too large". I searched Google and found out that this problem can occur for mainly two reasons: if size of the file being uploaded is greater than the value of $CGI::POST_MAX in the upload script, and if the value of the apache (web server) directive LimitRequestBody is lower than the file size. But both cases were handled. But I was still getting the 413 error. Then I found that it was my local server which prevented me to upload large file size. the 413 was coming from that machine instead of the server where the script is hosted :(

How to send custom error message in http header from cgi script

Couple of days ago, while I was writing a CGI script in Perl, it was needed to send custom error message in http response header. I would like to share with you how I have done this.

First I have decided that for any kind of error in the code (for example, file not found, insecure path etc.) I shall return status 500 with the error message. Before this decision my code looked like this:

...
my $query = new CGI;
print $query->header(-type=>'text/plain');
...

So for any error in the script, I added the line:
print $query->header(-type=>'text/plain', -status=>"500 $err_str");
But it wasn't working. The return value (status line in the response) still '200 OK'. Then I found that query header can't be set twice (once you set it, you can't change it). Then I removed the first line print $query->header(-type=>'text/plain'); and modified the code in such a way that the header is printed once. In order to return error, I wrote a function and just called it in case of any error occurs (but I made sure the header is not printed before I call it). Here is the function:

sub catch_n_die
{

    my $err_str = shift;

    print $query->header(-type=>'text/plain', -status=>"500 $err_str");

    exit;

}

Perl script to check the number of instances of a program

To get the number of instances of a program running in your machine you can use the following Perl code:

my $program_name = 'abc.pl';
my $data = `ps ax | grep perl`;
my @ara = $data =~ /perl\s+$program_name/g;
my $count = @ara;
print "$count\n";

Sometime it might be useful for you to know at the start of a program whether it's already running. You can get the name of your current Perl script/program from $0. If the name of your Perl script is program.pl, then $0 will contain program.pl. You can try print $0;
Hope this tiny piece of code will be useful for you. :)

execute linux command using Perl, Python, PHP

Those who program in Linux often need to execute/run linux commands from programs/scripts.

Using python you can execute linux command is the following way:

import os
os.system('ls -lt')


If you use Perl, you can do it in the following ways:
1. system(mkdir $folder);
2. qw(mkdir $folder);
3. $temp = `mkdir $folder`; #back ticks
4. exec($command);

If you use back ticks or qw, $? will be set if any error occurs. So you can use it like this:

qw(rm $filename);
if($?) {
print STDERR "error deleting file\n";
}


There are also several options available to run linux commands using PHP. Take a look at here for detals: http://us2.php.net/manual/en/ref.exec.php

SQL command to dump data into file and load data from file

In order to migrate data, it is useful to dump data from database table to a file, and load data from file to a table.

To dump data for a specific range in a file, you can use the following SQL command:

SELECT $fields INTO OUTFILE '$dump_file_name' FIELDS TERMINATED BY "\t"
ENCLOSED BY '"' ESCAPED BY "\\" LINES TERMINATED BY '\n' FROM $table_name WHERE
id >= $start AND id < $end;

You can then load the file by using the following SQL command:

LOAD DATA INFILE '$dump_file_name' INTO TABLE $table_name FIELDS TERMINATED BY "\t"
ENCLOSED BY '"' ESCAPED BY "\\" LINES TERMINATED BY '\n' ($fields);

These two commands work fine with MySQL.

Use Firebug for High Speed Development

The Firebug extension for Mozilla Firefox allows the debugging, editing, and modifying of any website's CSS, HTML, DOM, and JavaScript, and provides other web development tools.

(from wikipedia)

Firebug is really useful to me in web spider/crawler development. It makes writing regular expression easier & faster as I can put the mouse on specific portion of the web page and get it's code using inspect. Another useful feature for me is I can see all the http requests and responses, which also helps me a lot in writing web spiders.

You can get Firebug from getfirebug.com or download from addons.mozilla.org

Speed up your web development using Firebug!

SimpleXML - simple way to parse XML in PHP

This morning I needed to write a PHP script that parses an XML file. I could do it easily with regular expressions - as I can play very well with regexp :). But I was looking for an easier & simpler way to parse XML using PHP, as I thought there must be a very simple way. And I was right, I found SimpleXML. SimpleXML makes XML processing really simple. Here I give links of some excellent (and simple!) resources of SimpleXML:

• SimpleXML: A very well written and comprehensive article/tutorial.
• SimpleXML Functions: SimpleXML functions list with good examples from php.net
• PHP XML SimpleXML: w3schools tutorial.

SimpleXML will definitely speed up your XML processing speed!

C# Timer Control

In C#.NET, in order to do a certain task repeatedly after specific time interval you have to use the Timer control. Today I had to use it for a small project (to repeatedly send request to a web service). I just used Google and found some nice and easy articles. I think it's worth sharing couple of them to save your google time :)

• C# Timer Demo
• Working with Timer Control in C#

And of course you can look at MSDN (though sometimes I find it time consuming to learn something from MSDN).

W3Schools - quick way to get started with www technologies

If you are a student, or just started your programming career or a professional, you may need to get started with a certain web technology (HTML, XHTML, CSS, JavaScript, PHP, XML, AJAX etc.). And if you are running in short of time, you need a high speed introduction to those technologies. W3Schools can give you a quick start. It has many short tutorials on the following ares:

• HTML
• XML
• Browser Scripting
• Server Scripting
• Multimedia
• Web Building

I found many of them very useful for me. Hope you also will find the tutorials useful.

UnxUtil - Run Unix / Linux command in Windows

Life sucks when you are asked to make your script run in Windows, that you have written for Linux and that uses lots of Unix or Linux specific system commands :(

But there is a nice tool name UnxUtil that you can install on your windows and run Unix / Linux commands in the command line. UnxUtil will speed up your development by saving hours.

You can download it from here. After download, just extract it in a directory and add the PATH in your ENVIRONMENT VARIABLE (PATH) (Start -> Settings -> Control Panel -> System -> Advanced -> Environment Variables).

Thats all!

Use Beautiful Soup for Screen Scraping in Python

What to speed up web crawler / spider development? Try Beautiful Soup, if you use Python.

From their website:

Beautiful Soup is a Python HTML/XML parser designed for quick turnaround projects like screen-scraping. Three features make it powerful:

1. Beautiful Soup won't choke if you give it bad markup. It yields a parse tree that makes approximately as much sense as your original document. This is usually good enough to collect the data you need and run away.
2. Beautiful Soup provides a few simple methods and Pythonic idioms for navigating, searching, and modifying a parse tree: a toolkit for dissecting a document and extracting what you need. You don't have to create a custom parser for each application.
3. Beautiful Soup automatically converts incoming documents to Unicode and outgoing documents to UTF-8. You don't have to think about encodings, unless the document doesn't specify an encoding and Beautiful Soup can't auto detect one. Then you just have to specify the original encoding.

You can download it from here: http://www.crummy.com/software/BeautifulSoup/

Ruby programmers can enjoy Rubyful Soup

Enjoy High Speed Spider development with Beautiful Soup!

Perl equivalent of UNIX / Linux commands

One afternoon I was asked by my boss to write a Perl script that does the same task as the 'egrep' command of Linux. I studied the manual, to understand what 'egrep' does and planned a solution. But next morning when I came to office, I searched google and found a the Perl equivalent of 'egrep' command. It saved me at least two hours. You can click this link for details.

Welcome to High Speed Software Development

Hi, welcome to my blog on software development. After creating love-python (where I discuss about Python only), I have felt the need of another blog where I can discuss about my development experience with other languages (Perl, Java, PHP ...) along with Python. I shall try to write here on various topics that will help you to speed up your development. Various resources that I use in my professional life (and also personal work) that help me to solve a problem quickly will be here. And some fun also. I mostly work in Linux platform, so you are going to find some *nix specific things here.

I would greatly appreciate if you feel free to post comments here.

Hope you will enjoy the blog. Welcome on board!