jozef.kutej.net

When I showed Benjamin (a college of mine) my TPF 2010Q3 grant proposal, we slipped to a discussion about deploying Perl software. Nice (and recent) list of different approaches can be found @modernperlbooks.com. To sum it up => TIMTOWTDI. Which is good, but none of those is perfect. The Perl+CPAN world is way too complex.

During out discussion with Benjamin I proposed an idea of shipping the application altogether with the OS. Insane? The base Debian system is ~190MB, all the rest is needed for the application. Then deployment will be a matter of running this system on a virtual machine, somewhere in the cloud or in a simple chroot. (btw any Linux distribution can have any other Linux distribution working in a chroot) The files will never clash, all the "machines" would be dedicated. No fear of putting files where they belong to.

dpkg-scanpackages: info: Wrote 2898 entries to output Packages file.
dpkg-scansources  unstable /dev/null > unstable/Sources
bzip2 -c9 unstable/Packages > unstable/Packages.bz2
gzip -c9 unstable/Packages > unstable/Packages.gz
bzip2 -c9 unstable/Sources  > unstable/Sources.bz2
gzip -c9 unstable/Sources  > unstable/Sources.gz
dpkg-scanpmpackages
processing unstable/Packages.bz2
apt-ftparchive -c=Release.conf release unstable > unstable/Release
rm unstable/Release.gpg
gpg -abs -o unstable/Release.gpg unstable/Release

real    11m14.629s
user    1m45.227s
sys     1m56.679s

The dpkg-scan* tools are probably not the best solution for such big repository having 2898 packages in one folder..., but given enough patience it works! :-)

Xacobeo a Perl GUI to visualize XML and perform XPath queries just entered Debian unstable - http://packages.debian.org/sid/xacobeo.

Who is inventing this things? For years we have this already, don't we? It is just called ad banners. ;-) The idea and message is there. You like this page/article/service? Then just click some random banner to say "Thank you!". Simple and doesn't cost anything to the users. :-)

It is amazing what happened to the Python thanks to just one company. The killer feature of that company is that they can handle and manage millions of servers and even give them for a public usage => "..., there are no servers to maintain: You just upload your application, and it's ready to serve your users.". I would like this to be true one day for Perl too!

I really do. In coming months (years?) I'll try to plug together « MiniCPANInject + SmokeTesting + ContinuousIntegration + Virtualization + Debian + Monitoring » to create an architecture for Debian-Perl hosting. Actually who cares about OS used, when the uploaded code just works, right?... A hosting that is aware that one server is not enough for everyone and will offer an architecture for development, user-acceptance, staging and production environments to start with a project from $day == 1.

I wanted to get some CPAN authors sample. Not everyone as there are too much (> 8000). So I've created this two on-liners to get the CPAN authors whose module is packaged for Debian:

apt-cache search perl | perl -lane 'next if $_ !~ m/^lib(.+?)-perl\s/; $m=$1; $m=~s/-/::/g; print $m;' > /tmp/deb-perl.txt

zcat ~/.cpan/sources/modules/02packages.details.txt.gz | perl -MIO::Any -lae 'my %x; BEGIN { %x=map { $_ => 1 } split "\n", IO::Any->slurp("/tmp/deb-perl.txt") }; while (<>) { @F=split /\s+/; $m = lc $F[0]; if ($x{$m}) { (undef,undef,$a)=split "/", $F[2]; print $a; }; }' | sort | uniq | xargs echo

And here is the result (842 out of 8270 total):

Here are some variants that I've seen so far.

1. Careful people that are afraid to spoil the system use:

   • /usr/local
   • /var/local
   • /opt

2. Crazy people have always an original place to hide files in:

   • /usr/local64
   • /usr/local/64
   • /usr/local/local
   • /shared
   • /corp
   • /data
   • ...

3. Linux distributions put files according to the Filesystem Hierarchy Standard.

Some years ago I belonged to the first group and I was putting everything into /usr/local to be sure that there is just my stuff. Today I belong to the second one... I use /data for my laptop. Which is not so great idea, but it is my laptop and no one ever will have to (won't even be able) to touch it, so it is my mess. Some of the other ones like /corp, /shared and /usr/local64 I have to use at $work, because it is ops decision or a historical reason (sniff, sniff).

I hope one day I'll further evolve to work according to the standards. Tiding-up my mess would be the easy part of the journey...

Some people say that specifications are useless. That they never reflect the reality, that by the time of being done they are out-dated and people should just rush out, start coding and start working. I found this nice quote:

That was my missing piece to the argumentation. May be the specs at the end are "not so accurate", but at the time of writing them, the idea has to be exercised, questioned, the missing gaps filled in, all details understood and the goal clearly defined.

So we have an SSL version of a quite decent search engine. This should, besides the client-server encryption which is obvious, also hide a referrer. I've tested it right away:

click from http://www.google.cz/

192.168.240.133 - - [25/May/2010:09:11:42 +0200] "GET / HTTP/1.0" 200 41844 "http://www.google.cz/url?sa=t&source=web&ct=res&cd=16&ved=0CFUQFjAP&url=http%3A%2F%2Fkutej.net%2F&rct=j&q=kutej&ei=iHj7S436JsGtOMfE5P8B&usg=AFQjCNEIRhHXy7b_DsiM6c28l0wx3vOcVg&sig2=_cYOxkj7loeHmqhdNH6cUA" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.9) Gecko/20100501 Iceweasel/3.5.9 (like Firefox/3.5.9)"

click from https://www.google.com/

192.168.240.133 - - [25/May/2010:09:12:21 +0200] "GET / HTTP/1.0" 200 41844 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.9) Gecko/20100501 Iceweasel/3.5.9 (like Firefox/3.5.9)"

"http://www.google.cz/url?sa=t&source=web&ct=res&cd=16&ved=0CFUQFjAP&url=http%3A%2F%2Fkutej.net%2F&rct=j&q=kutej&ei=iHj7S436JsGtOMfE5P8B&usg=AFQjCNEIRhHXy7b_DsiM6c28l0wx3vOcVg&sig2=_cYOxkj7loeHmqhdNH6cUA"
vs
"-"

So when hiding, use SSL! :-)

Another headache for webmasters, where are all this people coming from?

Joel shows that StackOverflow is (||was?) running on 2 servers (+1 backup) using C# and SQL Server. Impressive :-) seek to 24:56 to see the full performance+tech table.

There is also a Perl+Perl6 mentioned in this talk, seek to 38:00 to hear the #1 thing you should never do...