A week with KDE 4.4
This past week I decided to see what all the fuss was about KDE 4.4. Everyone seems to love it, so I thought it may be time to withdraw from my tilling window manager love-fest and embrace the floating window manager lifestyle, again.
It was a mistake.
The install was painless, a
sudo pacman -Sy kdestartkdexinitrcI played around with a couple plasmoids, but quickly realized that to do anything on the machine meant that they would be hidden, so stopped wasting time with them. Next, I had to figure out a good window/desktop management scheme to use with my dual monitors. I opted for Firefox and Thunderbird on desktop 4, coding terminals and Firefox (api docs, viewing websites I was working on, etc.) on desktop 1, and ‘random’ terminals and programs on desktop 2. Desktop 3 was used as a temporary/random application area.
I won’t bore you with the details of my week, but I had a chance to do everything from some serious “flow” programming, to server/system maintenance, to goofing off on the Internet. It was all painful.
To figure out why it was so painful was easy to spot, and it all came down to window maintenance. Maybe I’ve just used tilling window managers too much, but to have to keep alt+tabbing between windows and resizing them when wanting to see multiple programs on the screen was downright painful. To combat this I tried to run all programs full-screen, but this just made the alt+tab problem bigger.
Needless to say, I’m back to using XMonad. As soon as I switched back, I could feel myself breathe a sigh of relief …. I was back home.
Connecting with PHP to Sybase on WAMP Server
Sybase is one of those systems that can be sometimes finicky to setup on the client side. I was pleasantly surprised to see that PHP has support for Sybase. When you need to quickly prototype a new feature or give your user’s a quick dynamic report, PHP can be a great path to go.
It took a little research to get all the parts necessary, but the installation is very smooth.
- Install WAMP
- Install Sybase client (you want the open client; login/pass required)
- Update php.ini to: enable sybase extension (extension=php_sybase_ct.dll)
- specify location of sybase sql.ini file (sybase.interface_file = “c:\sybase\ini\sql.ini”)
To test the connection we can do a simle query against a table:
<?php
<html>
<head>
<title>Sybase Test</title>
</head>
<body>
Testing execution<br />
<?= print_sybase_version() ?>
</body>
</html>
<?php
function print_sybase_version(){
sybase_min_server_severity(20);
$connection = sybase_connect('<server_alias>', '<username>', '<password>');
if(!$connection){
echo "couldn't make a connection";
}else{
$sql = "select CUSTOMER_NAME, CUSTOMER_STATUS from CUSTOMER order by CUSTOMER_NAME";
$sql_result = sybase_query($sql, $connection);
print "<table border=\"1\">";
while($row = sybase_fetch_array($sql_result)){
$name = $row["CUSTOMER_NAME"];
$status = $row["CUSTOMER_STATUS"];
print "<tr><td>$name></td><td>$status</td></tr>";
}
print "</table>";
}
}
?>
WebService::Linode 301 Moved Permanently
For those who, like me, use the Perl module WebService::Linode to configure their DNS settings with Linode, you will have noticed that starting today we are seeing 301 Moved Permanently at … error message.
This is easily resolved by changing line 36 on your Base.pm module file from “https://api.linode.com/api/” to “https://api.linode.com”.
Depending on how you installed the Linode module the file will reside in different locations. For me, the location is: /usr/share/perl5/site_perl/5.10.1/WebService/Linode/Base.pm
It’s a shame that they changed this URL without more public communication.
Patch File: linode_base.patch
Encrypted LVM Hard Drive with Linux
While setting up a couple new hard drives I realized that the information on the internet to use LVM and encryption were either wrong or unnecessarily complex. Here’s the commands I use with a brief explanation:
The disk I will be using in this example is sdd. Replace this with the hard drive you are setting up.
modprobe dm-crypt
modprobe aes-i586
dd if=/dev/zero of=/dev/sdd #write zero's to the drive, to wipe out all previous data
lvm pvcreate /dev/sdd1 #create the physical volume
lvm vgcreate lvm /dev/sdd1 #create the volume group with the identified 'lvm'
lvm lvcreate -l 100%FREE -n home lvm #create a logical volume as big as the drive itself called 'home' , which will create the file /dev/lvm/home and /dev/lvm/lvm-home, within volume group 'lvm'
cryptsetup luksFormat -c aes-xts-plain -s 512 /dev/lvm/home #encrypt our new logical volume 'home' with 512bit aes
cryptsetup luksOpen /dev/lvm/home home #open our new encrypted logical volume with the name 'home', this will create the file /dev/mapper/home
mkfs.ext4 /dev/mapper/home #create an ext4 filesystem our encrypted lvm partition
That’s all there is to it. To enable automatic mounting see /etc/crypttab. Note that when booting you will need to activate the lvm partition with the command
lvchange -a y <lvm identifier>RubyGems with Ubuntu 8.04 in a VM Woes
I use the fantastic virtual hosting solution Linode for my primary website. In order to launch one of my new projects using Ruby and Rails, I was required to install the RubyGems software on this site.
Unfortunately there is a problem with the version of RubyGems (<v1.01) that comes with Ubuntu, when installing in a virtual machine with limited memory. This causes any gem command to max out the cpu and hang with the output: “Bulk updating Gem source index for: http://gems.rubyforge.org”.
To solve this, we need to install the latest version of gems manually.
The first step ensure that we have all the ruby essentials installed …
sudo aptitude install ruby build-essential libopenssl-ruby ruby1.8-dev… but not rubygems from the Ubuntu repository.
sudo apt-get remove rubygemsNow download and install the latest version of Rubygems. The version as of this time is 1.3.5
wget http://rubyforge.org..... #replace with actual link
tar xzvf rubygems-x.x.x.tgz
cd rubygems-x.x.x
sudo ruby setup.rbI would suggest at this point you link the gem command, and update gems
sudo ln -s /usr/bin/gem1.8 /usr/bin/gem
sudo gem update --systemSuccess! You should now be able to use gem to install our dependencies. For example, I needed rake, rails, and mongrel
sudo gem install rake
sudo gem install rails
sudo gem install mongrel
Ruby Histogram Chart
I’m working on a new project that involves creating dynamic charts of the number of steps and number of steps walked at a moderate pace. I’ve chosen Ruby on Rails as my language and framework, since it’s something I’ve wanted to learn for a while and it just “seemed right”.
When trying to create this charts, I wanted to do it in a native ruby library, as shelling out to a separate program is far from pragmatic. I was excited then to see that there is a ruby binding for gnuplot, the de facto Unix command line plotting tool. After many frustrating hours spent trying to convert my gnuplot syntax of a histogram into ruby_gnuplot, I finally figured it out. Hopefully others can save time and learn from my frustration.
Here’s the chart I’m trying to make:
To start off with gnuplot, let’s take a look at our data. For a steps.dat file I have the following:
date steps mod_steps
2009-08-09 1936 8178
2009-08-08 880 13844
2009-08-07 1424 15572
2009-08-06 1242 21451
2009-08-05 1666 14160
2009-08-04 2322 20286
2009-08-03 1442 16653
2009-08-02 478 15529
2009-08-01 2561 13055
2009-07-31 1012 1043
2009-07-30 1452 12383
2009-07-29 1283 15452
2009-07-28 1099 8858The following gnuplot script will create a rowstacked histogram of this data, and output it to ‘steps.png’
set terminal png transparent nocrop enhanced
set output 'steps.png'
set boxwidth 0.2 absolute
set style fill solid 1.00 border -1
set style histogram rowstacked
set style data histograms
set xtics border in scale 1,0.5 nomirror rotate -45 offset character 0, 0, 0
set title "Testing"
plot 'steps.dat' using 3:xtic(1) t 'moderate', '' using 2 t 'normal'Here is a function that, using the ruby_gnuplot gem, will create an identical chart. Note that instead of using a data file, we have three arrays (x, y1, and y2) which hold our x-axis, y1-axis, and y2-axis. The trick here, and where the endless hours of searching the documentation came, is whereas in straight gnuplot we can use one plot command, in ruby_gnuploot we need to split this up into two plot calls.
def genGnu
Gnuplot.open do |gp|
Gnuplot::Plot.new( gp ) do |plot|
plot.terminal “png transparent nocrop enhanced size 800 600”
plot.output “public/myPlot.png”
plot.boxwidth “0.2 absolute”
plot.style “fill solid 1.00 border -1”
plot.style “histogram rowstacked”
plot.style “data histograms”
plot.xtics “border in scale 1,0.5 nomirror rotate -45 offset character 0, 0, 0”
plot.title "Steps for " + @user.name
x = []
y1 = []
y2 = []
@steps = @steps.sort_by { |step| step[‘rec_date’] }
@steps.each do |s|
x.push(s.rec_date)
y1.push(s.steps – s.mod_steps)
y2.push(s.mod_steps)
end
plot.data << Gnuplot::DataSet.new( [x, y2] ) do |ds|
ds.using = “2:xtic(1) t ‘moderate’”
end
plot.data << Gnuplot::DataSet.new( [x, y1] ) do |ds|
ds.using = “2:xtic(1) t ‘normal’”
end
end
end
endTiling Window Managers and Productivity
On a recent walk whilst listening to one of my favorite podcast, the Stack Overflow Podcast, Joel and Jeff started talking about the productivity of dual monitors.
It is a [fairly] undisputed fact that having a second monitor makes any worker more productive. The reason for this is quite simple, more desktop real-estate means being able to see more applications at a time. This makes even more sense when you factor in all the “tabbing” between applications that you save, sometimes in my case this means tabbing multiple times because I had forgotten what I had just read.
Jeff made the point that often times he doesn’t bother with moving and resizing windows, and usually just brings it to the monitor he wants and then makes it full-screen. His point was that this makes him even more productive, because you no longer worry about placement of windows. I couldn’t agree more.
This is exactly the place where tiling window managers come in to play. A tiling window manager is one in which you do spend your time moving and resizing windows, as they are automatically placed for you. This allows you to focus on your work at hand, rather than tabbing to see all your windows, or moving and resizing to see two at once.
When you combine the power of a tiling window manager with dual screens, something magical happens. Now you have the increased desktop space in addition to no longer having to manage your windows. In my tiling window manager, XMonad, you treat each monitor as a separate “window” into your virtual desktops. By this I mean that you setup windows in a particular virtual desktop, and then have the ability to show that virtual desktop on whichever monitor you choose.
I cannot emphasize enough the benefits of both dual screens and a tiling window manager. Some good managers for Linux are:
Easy Email Disk Space Alerts
The other day I ran into an issue where a server of mine ran out of disk space. The server is part of a surveillance system I have and collects ~2 gigs of jpeg’s a day, and so can fill up every couple of months if I’m not careful. Not having an alert system finally caught up with me and I was greeted when I returned from work with a couple dozen failure e-mails from cron, indicating the system had run out of space.
Not to be caught in this situation again, I’ve added the below to my crontab. It’s a simple bash command that will e-mail me if my server gets over 80% utilized. I use the second line to send an SMS message to myself if the server becomes 90% or over utilized.
#Email when storage gets low
00 * * * * full=`/bin/df | /bin/egrep "(100\%|[89][0-9]\%)"` && echo $full | /usr/bin/mail -s "[$HOSTNAME] Warning: harddrive space issue" EMAIL_ADDRESS@PROVIDER.COM
00 * * * * full=`/bin/df | /bin/egrep "(100\%|9[0-9]\%)"` && echo $full | /usr/bin/mail -s "[$HOSTNAME] URGENT: harddrive space issue" EMAIL_ADDRESS@PROVIDER.COMAmazon.com Wishlist Tracker
Amazon.com is one of my favorite online retailers. Their prices are [usually] great, they have incredibly fast shipping (free 2-day with Amazon Prime membership), and have an enormous inventory of merchandise. Amazon has a nice feature when you leave things in your cart, in that they will tell you when you return whether your saved cart items have increased or decreased in price. While an awesome feature, the caveat is that you have to check their site multiple times a day to increase your chances of seeing some of the really good deals.
Solution? Amazon.com’s API. Amazon provides an API to their database called Amazon Web Services for free (registration required). This opens up a huge opportunity for some great tools. Take, for instance, an automatic price updater, which automatically tells you when something you’ve had your eye on is having a sale.
The program I wrote to do this runs in three parts. Using a public wishlist where I add my items to track (instead of my shopping cart), the first script loads all my wishlist items and find anything that was added, adding them to a mysql database.
#!/usr/bin/perl
use Net::Amazon;
use Net::Amazon::Request::Wishlist;
use DBI;
use Product;
sub createRecord{
my $rec = shift;
print "creating record for: " . $rec->getAsin() . "\n";
my $db='amazon';
my $host="localhost";
my $uid="USERNAME";
my $passwd="PASSWORD";
my $conn="dbi:mysql:$db:$host";
my $asin = $rec->getAsin();
my $name = $rec->getName();
my $desc = "";
# quotes will ruin our query, replace with underscore
$name =~ s/'//g;
my $dbh = DBI->connect($conn,$userid,$passwd);
my $query = "insert into products(asin,name,description) values('$asin','$name','$desc')";
my $sth = $dbh->prepare($query);
$sth->execute();
}
my $ua = Net::Amazon->new(token=>'YOUR_AMAZON_TOKEN');
my $req = Net::Amazon::Request::Wishlist->new(wishlist=>'YOUR_WISHLIST_ID');
my @products;
my @wishlist;
my $db='amazon';
my $host="localhost";
my $uid="USERNAME";
my $passwd="YOUR_PASSWORD";
my $conn="dbi:mysql:$db:$host";
my $dbh = DBI->connect($conn,$userid,$passwd);
my $query = "select asin,name,description from products";
my $sth = $dbh->prepare($query);
$sth->execute();
$sth->bind_columns(\$asin, \$name, \$description);
while($sth->fetch()){
#print "found: $asin | $name | $description\n";
push(@products, Product->new($asin, $name, $description));
}
my $resp = $ua->request($req);
foreach my $item ($resp->properties){
push(@wishlist, Product->new($item->ASIN(), $item->ProductName(),));
}
#foreach(@products){
# print "Found a product: " . $_->getAsin() . "\n";
#}
foreach my $wish (@wishlist){
#print "Found a wishlist: " . $_->getAsin() . "\n";
my $flag = 0;
foreach my $prod (@products){
if($wish->getAsin() eq $prod->getAsin()){
$flag = 1;
}
}
if($flag == 0){
createRecord($wish);
}
}
exit 0;
The second script can then run, which looks up all the items in the mysql database and finds their updated prices. If the price of an item has changed, it adds a new record to a table.
#!/usr/bin/perl
use Net::Amazon;
use Net::Amazon::Request::Wishlist;
use Net::Amazon::Request::ASIN;
use DBI;
use Product;
use Price;
## Create a record in the "prices" table
sub createRecord{
my $rec = shift;
#print "creating record for: " . $rec->getAsin() . "\n";
my $db='amazon';
my $host="localhost";
my $uid="USERNAME";
my $passwd="PASSWORD";
my $conn="dbi:mysql:$db:$host";
my $asin = $rec->getAsin();
my $price = $rec->getPrice();
my $dbh = DBI->connect($conn,$userid,$passwd);
my $query = "insert into prices(asin,price) values('$asin','$price')";
my $sth = $dbh->prepare($query);
$sth->execute();
}
## Get a Product details from amazon
sub getAmazonPrice{
my $ua = Net::Amazon->new(token=>'YOUR_AMAZON_TOKEN');
my $asin = shift;
my $req = Net::Amazon::Request::ASIN->new(asin=>$asin);
my $resp = $ua->request($req);
foreach my $item ($resp->properties){
my $tPrice = $item->OurPrice();
$tPrice =~ s/\$//g;
return $tPrice;
}
}
my @products;
my @amazonPrices;
my $db='amazon';
my $host="localhost";
my $uid="USERNAME";
my $passwd="PASSWORD";
my $conn="dbi:mysql:$db:$host";
my $dbh = DBI->connect($conn,$userid,$passwd);
my $query = "select asin,price from prices where id in (select max(id) from prices group by asin)";
my $query2 = "select distinct asin from products";
my @prodAsins;
my $sth2 = $dbh->prepare($query2);
my $prodAsin;
$sth2->execute();
$sth2->bind_columns(\$prodAsin);
while($sth2->fetch()){
push(@prodAsins, $prodAsin);
}
my $sth = $dbh->prepare($query);
$sth->execute();
$sth->bind_columns(\$asin, \$price);
while($sth->fetch()){
push(@products, Price->new($asin, $price));
}
foreach(@prodAsins){
push(@amazonPrices, Price->new($_, getAmazonPrice($_)));
}
foreach my $amazonAsin (@amazonPrices){
my $flag = 2;
foreach my $prod (@products){
if($amazonAsin->getAsin() eq $prod->getAsin()){
if($amazonAsin->getPrice() != $prod->getPrice()){
$flag = 1;
}else{
$flag = 0;
}
}
}
if($flag > 0){
print "Creating record for " . $amazonAsin->getAsin() . " | price is now " . $amazonAsin->getPrice() . "\n";
createRecord($amazonAsin);
}
}
exit 0;
The third piece is a php script which shows in a rudamentary way what the current status of your items are. Green highlighted items are currently at the lowest price you’ve seen, red are at the highest, and white have always been the same.
<html>
<head>
<title>A Little Bit of PHP</title>
<style type="text/css">
td {font-size: 8pt; font-family: Arial;}
</style>
</head>
<?php
mysql_connect('localhost', 'USERNAME', 'PASSWORD');
@mysql_select_db('amazon') or die("Unable to select database");
#$query = "call gen_report();";
$query = 'select * from products p left join ( select p.asin, p.price as min_price, max(p.creation_date) as min_date from (select asin,min(price) as price from prices group by asin) as x inner join prices as p on x.asin = p.asin and x.price = p.price group by asin ) as A on p.asin = A.asin left join ( select p.asin, p.price as max_price, max(p.creation_date) as max_date from (select asin,max(price) as price from prices group by asin) as x inner join prices as p on x.asin = p.asin and x.price = p.price group by asin) as B on A.asin = B.asin left join ( select p.asin, p.price as curr_price, p.creation_date as curr_date from prices p where id in (select max(id) from prices group by asin) ) as C on B.asin = C.asin';
$result = mysql_query($query);
mysql_close();
$num = mysql_numrows($result);
?>
<body>
<table border="1">
<tr>
<th>ASIN</th>
<th>NAME</th>
<th>LOWEST</th>
<th>DATE</th>
<th>HIGHEST</th>
<th>DATE</th>
<th>CURRENT</th>
<th>DATE</th>
</tr>
<?php
$i = 0;
while($i < $num){
$min_price = mysql_result($result, $i, "min_price");
$max_price = mysql_result($result, $i, "max_price");
$curr_price = mysql_result($result, $i, "curr_price");
if(($min_price == $curr_price) && ($min_price != $max_price)){
print '<tr style="background-color: lightgreen;">';
}else if(($max_price == $curr_price) && ($min_price != $max_price)){
print '<tr style="background-color: pink;">';
}else if(($max_price > $curr_price) && ($min_price < $curr_price)){
print '<tr style="background-color: #eeee44;">';
}else{
print "<tr>";
}
print "<td>" . mysql_result($result, $i, "asin") . "</td>";
print "<td>" . mysql_result($result, $i, "name") . "</td>";
print "<td>" . $min_price . "</td>";
print "<td style=\"background-color: #eeeeee;\">" . mysql_result($result, $i, "min_date") . "</td>";
print "<td>" . $max_price . "</td>";
print "<td style=\"background-color: #eeeeee;\">" . mysql_result($result, $i, "max_date") . "</td>";
print "<td>" . $curr_price . "</td>";
print "<td style=\"background-color: #eeeeee;\">" . mysql_result($result, $i, "curr_date") . "</td>";
print "</tr>\n";
$i++;
}
mysql_free_result($result); //we're done using the results, so set it free
?>
</table>
</body>
</html>
To set the scripts up on a schedule, just add them to your cron. The below example looks for new items in your wishlist every hour on the hour, and new prices five minutes later.
00 * * * * /home/jon/amazon/updateProductsFromWishlist.pl
05 * * * * /home/jon/amazon/updatePrices.pl
I’ve uploaded all the source code. I plan to add more features as time permits, and depending on Amazon’s license I hope to open a full site where user’s can sign up for automated alerts.
Monitor Network Usage With VNSTAT
With the recent threats from ISP’s lately to implement bandwidth caps on their customers, I began questioning how much bandwidth my family uses in a typical month.
I spent a few hours on google trying to find a program which could do just that, and give me both raw statistics as well as summations at the hourly, weekly, and most importantly monthly increments.
Having not found a suitable program, I began writing my own. While struggling with the 32-bit limit set on the netdev stats counter, however, I came across Crunchbang Linux, and found a post on their blog about vnstat. VNStat is a program that gives you everything you want from a network bandwidth usage standpoint, including the aforementioned statistics.
Even better, when coupled with the vnstat php frontend, seen above, you can show all the users of your network what your utilization is.
Of course, to set this up for a network this machine has to either be setup as your network’s router, or can do some changes with your default gateway if that is not possible.
Digital Drip ... not just a blank page anymore!
Hi Everyone! I’m just getting setup here with Wordpress, so please don’t mind the mess while I get things organized. Subscribe to the rss feed so you don’t miss a beat!