Archive

Archive for February, 2010

New web server, WP optimization

February 28th, 2010

Tonight, more or less on a whim, I moved my blog from my older (dual 1GHz Pentium III Coppermine, 1GB RAM, 10k RPM SCSI disks, Compaq Proliant DL360 G1, OpenSuSE 10.2 32-bit) web server to my newer one (dual 1.4GHz Pentium III, 2GB RAM, 10k RPM SCSI disks, HP Proliant DL360 G2, CentOS 5.3 32-bit). I did some profiling with ab (ApacheBench), and just moving from one server to the other got some serious performance gains (I was profiling with runs of 1000 requests total, 10 concurrent requests). I also added the W3 Total Cache Wordpress plugin, which got the numbers to look even better!

As a side note, this was all done pretty quickly (moving the database and tarball for the vhost, installing the plugin, changing DNS), so please give me a heads-up if you experience any problems.

The numbers are rather impressive:

 Total Time(s)RPSAvg. Connection Time (ms)
Old Server1192.252838.7511,893
New Server569.1211757.095,667
Default W3tc Config23.75442,098.44237
Tuned W3tc12.28181,428.76122

All tests were performed on my workstation, a Dell Precision 470, two dual-core Xeons at 2.8 GHz, 2GB RAM, 16GB swap, OpenSuSE 11.1 64-bit. This was on the same LAN and subnet as the servers, with the workstation connected via a 1Gbps copper Ethernet link and the web-serving interfaces of the servers connected via 100Mbps (There’s a trunk in between, from the gigabit aggregation switch to the 100Mbps distribution switch).

Projects , , ,

My Android Apps

February 28th, 2010

The past 48 hours has been very eventful in my Android world. Thanks to the instructions on rootyourdroid.info (hey, it’s an expensive phone and locked hardware – I’m not fooling around the way I would with something more common), I rooted my droid and did a few minor hacks. Though, I must say, it pains me to see all of the post-rooting instructions based on access via adb, instead of using a terminal emulator on the phone. Also, last night, my mother (definitely not a technical person) for a Droid – and loves it! (My dad got one a month or two ago).

Anyway, in the last 24 hours, I’ve gone app-crazy. I thought I’d share some of my findings here. Unfortunately, while a few of the app/file managers out there do dump a list of applications, I can’t find one that dumps a list including the package names (which are required to create an effective link to the app). If any of you know of one, please enlighten me. For now, I’ll construct this list by hand (using the list from ASTRO 2.2.4), and maybe write an app to do it in the future.

(Note: These links all use the market:// URI scheme, so they’re only useful if clicked on an Android device with the Market app.)

  • AndFTP (1.3) – A good FTP/SFTP client for Android, includes pubkey-based authentication and host storage.
  • Android Battery Dog (0.1.2) – An app that runs as a service and collects detailed statistics on battery usage, including temperature, charge percent, voltage, discharge rate, battery technology and external power status. While it can display a graph or formatted data, its’ real shining point is the ability to export timestamped semicolon-delimited data files for external graphing and analysis.
  • Any Cut (1.0) – Allows you to create shortcuts on your home screen to almost anything, including Android OS Acitvities, direct calls or direct text messages.
  • ASTRO (2.2.4) – File manager that allows copying of files on the device (both internal memory and SD card), image and file viewing, listing/management of apps (including finding out the package name of an App), reading of tar and tgz files, etc.
  • Battery Widget (1.5.2, by mippin) – Very simple widget for the home screen that takes up one square and shows current battery level. When clicked, provides shortcuts to settings screens for display, GPS, WiFi and Bluetooth.
  • Compass (1.1) – Simple compass app. Shows a compass which seems to be accurate, current lat/long and current street address. Has extensive settings and some nice skins for the compass.
  • ConnectBot (1.6.2) – A very good SSH client for Android. Allows storage of multiple hosts, pubkey-based authentication (with a master password), etc. Unfortunately, doesn’t seem to have any way (that I can find) to enter certain characters, such as tab and pipe (|).
  • Dolphin Browser (2.5.0) – An alternate browser for Android. I haven’t used it extensively yet, but it shows multiple tabs at the top of the screen like Firefox (easier to switch between tabs than the stock browser’s Menu -> Windows) and supports iPhone-like multitouch on the Droid.
  • drocap2 (2.07) – Screen capture program (requires root). Allows you to trigger a capture from the notifications bar and stores captures on the SD card.
  • DroidLight (3.0) – Nice twist on the usual flashlight app. By Motorola, this app triggers the camera’s flash LED in a steady burn mode, providing very good light output. Probably a real battery killer.
  • FoxyRing (1.12) – This was an ANdroid Developer Challenge winner and, among other things, it claims to monitor ambient sound levels and adjust your ringer volume to match them. Unfortunately, due to the overly restrictive End User License Agreement (EULA), specifically the strong provisions against reverse engineering and redistribution, I was forced to uninstall the app before even trying it.
  • GPS Status (3.0.3) – Very nice app. Provides a display like a real GPS, showing the location and status of various satellites (in a rotating compass), heading and orientation, number of fixes, estimated error (DoP), signal strength graph for stelites, speed, altitude, pitch/tilt of phone, magnetic field, acceleration, coordinates and time of last fix.
  • iPerf (1.07) – An iPerf client for Android that seems to work fine. How cool! Seems to be a wrapper around the binary, lets you specify CLI arguments, shows console output.
  • Meebo IM (22) – A simple, good, multi-protocol IM application.
  • Metal Detector (1.2-RELEASE) – Maybe not that useful, but way cool. The Droid (and perhaps other phones?) uses the compass to detect magnetic fields to trigger the modes for car dock and multimedia dock. This turns it into a metal detector. Wonderful cool-ness factor.
  • Nagroid (0.0.7) – A Nagios watcher for Android. Can be configured with only one URL, but can do HTTP Basic Auth and handle self-signed SSL certs. Options to hide everything that’s OK, and show only unhandled (un-acknowledged) problems. Also can start a service to poll and alert at regular intervals. Only down side is that it only handles one Nagios URL.
  • Network Discovery (0.2.7.1) – Intersting little app that I haven’t played around with much. Does port scans of IPs and runs a “network discovery” of the LAN, though it doesn’t say whether it is active (ping/port scanning) or passive (ARP). Displays info on devices (IP, MAC address, decodes MAC manufacturer name from address) and a button to run a port scan.
  • OSMonitor (1.1.0) – Good process monitor for Android – shows running processes. load from each process, total CPU usage, network information for all NICs (WiFi, BT, cellular/PPP), active TCP connections, battery status, storage status (of ALL filesystems), and internal log.
  • Ping (1.5.3) – Simple ping app. Lets you enter an IP and select how many pings to send out. Shows console output.
  • Shazam (1.3) – Yup, same thing that was the killer app for iPhone.
  • SMS Backup & Restore (2.1) – Allows backup and restore of SMS data to/from SD card, as an XML file. Good for Droid users who experience the disappearing SMS bug.
  • Speed Test (1.7.0) – A simple speed test app for Android from speedtest.net. Not sure how accurate it is, but it does upload and download tests over WiFi or cellular/PPP.
  • Spirit Level Plus (1.2) – Simple but cool. Spirit level for the phone, using the builtin accelerometer. Seems relatively accurate.
  • StopWatch (1.07) – AWFUL. It’s a stopwatch app, but I could not get it to stop displaying stuff in the notifications bar.
  • Terminal Emulator (1.0.4) – Terminal emulator for Android. It feels so wonderful to be able to pull up an app, pop open the keyboard, and type “su” on my phone. On the down side, once again, I can’t figure out how to enter the pipe or tab characters, and I don’t know what shell the phone has on it.
  • Wifi Analyzer (2.2.9) – REALLY COOL. Vaguely WiSpy like, but I doubt it’s accurate. Shows a graph of spectrum utilization with SSIDs and signal strength, a time-based graph of signal strength per SSID, a simple list of APs with channel number, BSSID, frequency, signal strength and encryption, and a simple “signal meter”. Looks like it could be pretty useful.
  • WifiScanner (1.7) – Simple WiFi scanner app. Shows all detected WiFi APs along with SSID, BSSID/MAC, signal level, channel and encryption.

android , ,

On FIle Sharing, DRM, and customer choice

February 27th, 2010

There’s been a lot of buzz over the past few years about DRM, file sharing, “intellectual property theft”, etc. A lot of that has been the two extreme sides – the media industry and their “have it our way” attitude, and the extremists who feel that everything digital should be freely shareable by everyone. I don’t fall into either of those categories, and I don’t think the majority of people do either.

First, let’s look at a bit of history. In my early childhood (1990’s), cassettes were giving way to CDs, and VHS tapes were the norm for videos. You could go to any corner store and buy a blank cassette tape or VHS tape, and it was widely known that people recorded TV shows or copied audio or video tapes. To cope with this, a portion of the purchase price of every blank tape was distributed among media companies and artists, to compensate them for the copies being made. It seemed that everyone was happy about this – nobody was trying to ban the sale of blank tapes, and my neighborhood video rental store never made me sign a contract promising not to copy a rented tape. There seemed to be a balance between the need for profit and what consumers wanted to do.

That all changed when the world went digital – first audio CDs, then movies on DVD. It requires mention that almost all of the problems faced by the media industry (namely “piracy” and file sharing) were brought by the industry itself. I vividly remember, over a period of a mere two years or so, the transition from VHS to DVD. I remember going to the video rental store (we were late adopters, nobody in my family had a standalone DVD player) and being told that new releases were no longer coming out on VHS. We had to buy a DVD player. This was a format that was pushed on consumers by the movie industry, and was pushed hard and fast. While everyone talked of the quality benefits, it was obvious that distributors were in love with the format’s cheap and quick reproduction. I simply do not believe that the movie industry was unaware (especially given the proliferation of DVD drives in computers) that this cheap reproduction was as easily available to consumers as it was to them. If they were unaware, we must ask how their million-dollar-a-year technical teams never mentioned it. You can’t have your cake and eat it too. The movie industry chose to convert to a format that’s easily copied. The movie industry chose to convert to a format that could be easily read – and copied – on any home computer. They should be forced to accept that choice, and the effect that anyone with a computer can duplicate or share their products. If they didn’t want people to do this, they should have stuck with VHS, or gone to a higher-quality tape format.

But, I digress. The main point that I want to make is about consumer choice, and how that effects purchasing (and sharing) habits.

In my parent’s generation, and those before it, customers voiced their choice through making a purchase or not making a purchase. If they didn’t like a car salesman’s attitude, they’d buy the car from someone else. If they didn’t like the terms of a warranty, they’d buy their washing machine from Sears instead of the local store. If they didn’t like their phone company, they’d switch.

My generation, in the digital age, was faced with a different choice – buy or share. The recording and movie industries more or less made this choice for us. They wouldn’t let us buy how we wanted to, so we made the other choice.

This choice required a bit of a tangent to explain. The industry wants us to think of file sharing as stealing. When sharing digital files, they want us to think of the fact that the file is duplicated (i.e. my friend now has it, but I still have it too). This is simply a side-effect of how digital systems work. Whether right or wrong, whether antiquated or not, in most human minds the concept of stealing is inextricably linked to physical property. Walking into a library and walking out with a book that you didn’t check out is clearly stealing. However, most people wouldn’t think the same thing of photocopying some pages from the book. Most people wouldn’t think of photocopying a newspaper article and mailing it to their friend as stealing. How many people, in the day of audio cassettes, thought of it as “stealing” when they copied a tape for their friend? I’d guess that, for the vast majority of people, file sharing is much more closely associated with these actions than walking out of a record store with a CD.

My personal theory is that a large amount of file sharing (of copyrighted material) would stop if the movie industry would let people buy the way they want.

There was a time, a few years ago, when I got almost all of my music through peer-to-peer file sharing (though, unlike many, I didn’t allow uploads). I never thought much of it – I shared lots of things with my friends, why not music? Then RIAA started their PR and lawsuit campaigns. They started suing college kids for sharing music – and suing them for a lot more than even the cost of the CDs they’d “stolen” (and that’s ignoring the fact that they just “stole” the information on the CDs, so the actual cost should have been lower, less the physical media and distribution costs). So, I heard what the recording industry was telling me: we don’t like you. I stopped downloading music, and I also stopped buying it. For about 3 1/2 years, I listened to what I already had on CD, or the radio, but nothing new.

Then there was iTunes. You could buy whatever music you wanted, usually for less than $1. But you had to use their software, which didn’t run on Linux. And if you wanted to listen to it away from your computer, you had to use an iPod. And you couldn’t burn it to CD, so it wouldn’t work with the older stereo in my car.

Finally, the industry woke up. Amazon came out with their MP3 store, where I could buy individual songs or complete albums, as standard (non-DRMed) MP3 files, that I could listen to on my cell phone, any of my computers, or burn to CD and play in my car. And I’ve been hooked ever since – I get all of my music for a low price, in a standard unrestricted format. I can burn it to CD for my car, put it on my computers at home and at work, put it on my laptop, put it on my phone. Thanks to 1-click ordering and instant downloads, I probably spend more on music now than I did when I had to go to a store to buy CDs. And why? Because I have choice. Because, finally, they’ll sell music to me the way I want it – and I buy it.

I don’t know of any source of unbiased statistics, but I’d venture a guess that since various stores have begun selling DRM-free music online, the volume of peer-to-peer sharing of copyrighted music files has gone down.

But it seems that the movie industry hasn’t woken up to this, the MPAA hasn’t taken a lesson from RIAA. While options are starting to appear – NetFlix streaming and others – they still haven’t made the realization that customers will continue to choose “other” until offered the choice they want. I still can’t buy and download movies on Linux, and since I use MythTV for my home theater, it’s no use to get a NetFlix box. Until offered what they want – a download of an unencumbered, DRM-free movie file, or full DVD image, people will keep sharing movies, and will keep renting them and ripping full-resolution copies.

Finally, it’s worth mention that the secret Anti-Counterfitting Trade Agreement (ACTA) is obviously tilted in the favor of content producers, and has a number of chilling provisions for the Internet. Most importantly, it seeks to reverse previous law and hold ISPs liable for infringement by their customers. Firstly, and I say this with all my heart, this is wrong. Until publishers start successfully suing Xerox for every copy of a page of a book ever made, don’t try and hold ISPs responsible for what their customers do. But more importantly, this is braindead – we should know by now that copyright holders can’t win the cat-and-mouse game. We saw it with p2p and random ports, etc. Trying to detect transmission of infringing material is impossible. Once a new method is invented, it will be bypassed. No matter how many millions the media industry spends on trying to detect violations, there’s simply more people working on the other side, and they’re probably smarter and better motivated as well. If the media industry pushes for ISPs to use deep packet inspection (DPI) technology, the users will just turn to PKI and encryption to hide their data. If ISPs just look at traffic patterns, the users will accept slower download times and shape their traffic to look like web browsing.

If the media industry really wants to stop file sharing of their content (instead of just benefiting from lawsuits) the solution is simple – let consumers buy it the way they want.

Ideas and Rants , , , , , , ,

Links for Tuesday, February 23, 2010

February 23rd, 2010

A Cartoon about Me

February 22nd, 2010

Just found this today on one of my all-time favorite sites, XKCD. I swear it’s about me:

XKCD number 705

Miscellaneous Geek Stuff , ,

Parsing Nagios status.dat in PHP

February 21st, 2010

If you’re just looking for the script or PHP module, you can get them via Subversion at: http://svn.jasonantman.com/nagios-xml/.

A while ago (back in late 2008), I wrote a PHP script that parses the Nagios status.dat file into an associative array. My original use was to output XML which was then read by another script on another server and used for a small custom GUI. It’s a very simple PHP script that just takes the path of the status.dat file (which, obviously, must be readable by the user running the script).

At that time, I was using Nagios v2. Since then, I’ve moved to Nagios v3, and have updated the script to include the ability to parse v3 status.dat files, as well as a function to detect the version of a status file. I also refactored the code so that the parsing functions are all contained in a single file (statusXML.php.inc) which is safe to include in other scripts. The actual statusXML.php file now just includes examples of how to call all of the functions and output XML (though it is equally useful to output the serialized array, or use it directly).

Since I posted my script online, two people have been kind enough to send back their modifications:

Both of these generous contributions have been included in my Subversion repository as of the current revision, 5. Unfortunately, due to my delay in putting my Nagios3 code into svn, both of these contributions are Nagios v2 only.

As time permits, I plan on merging Artur’s changes into the current version of statusXML.php.inc. Unfortunately, C isn’t one of my strong points, but I plan on also updating Whitham’s PHP module code to work with Nagios3 as soon as possible.

Stay tuned for updates, and thanks to both gentlemen for contributing their work. I’m always interested in hearing how people are using my code, and how they are making it better.

Also: While I added this project to Nagios Exchange, and plan on adding it to Monitoring Exchange, I don’t always keep those sites up to date (I can’t access Nagios Exchange right now, and who knows if I’ll have time to update it tomorrow). I strongly recommend directly checking out from Subversion at http://svn.jasonantman.com/nagios-xml/ or taking a look at the code through ViewVC at http://viewvc.jasonantman.com/cgi-bin/viewvc.cgi/nagios-xml/.

Projects , ,

Thunderbird SSL “Domain Name Mismatch”

February 12th, 2010

Some of my servers that are only for internal/personal use have SSL certs with a mismatched hostname. The cert for my mail server is issued for the CNAME used for my mail server, not the actual hostname. Of course, this means that Thunderbird gives me some annoying errors because they’re worried:

Domain Name Mismatch in Thunderbird

Luckily, there’s an add-on called “Remember Mismatched Domains” that adds a simple “remember this decision” check box, much like the one now found in Firefox. Problem solved!

Thunderbird Domain Name Mismatch after add-on

Tech HowTos , , , ,

pnp4nagios, CentOS 5.3 and pcre

February 11th, 2010

I started testing out the pnp4nagios tool to incorporate graphs of performance data into Nagios. Despite what Klein and Sellens suggest (p. 57), I really don’t want separate tools for monitoring and trending. Cactialready handles UPS metrics, switch ports, router traffic, etc. For everything else – system load, etc. – I see no reason to have two checks run rather than just one (Nagios).

There was a CentOS package for the older pnp4nagios 0.4.x, but I opted to build and install the new 0.6.x from source. Unfortunately, I hit one snag – it requires PCRE compiled with support for Unicode properties, and I couldn’t find any package for CentOS compiled with that option. So, with a simple edit of the %configure macro in the SPEC file, I built one. Unfortunately, I wasn’t working in a real build environment – just on one of my web servers – so I only built the .i386 version, but you can feel free to build from the source rpm.

Tech HowTos , , , , ,

Apache2 – list Name-Based Virtual Hosts

February 11th, 2010

Here’s a little tidbit that I never knew until I had an Apache2 name-based virtual host problem: httpd -S lists the vhosts that are being served by Apache, and how they were parsed from the config files.

The output on one of my servers looks something like:

[root@web2 vhosts.d]# httpd -S
VirtualHost configuration:
wildcard NameVirtualHosts and _default_ servers:
_default_:443          web2.jasonantman.com (/etc/httpd/vhosts.d/ssl-host.conf:7)
*:80                   is a NameVirtualHost
         default server www.jasonantman.com (/etc/httpd/vhosts.d/000-default.conf:1)
         port 80 namevhost www.jasonantman.com (/etc/httpd/vhosts.d/000-default.conf:1)
         port 80 namevhost rackman.jasonantman.com (/etc/httpd/vhosts.d/rackman.jasonantman.com.conf:1)
         port 80 namevhost whatismyip.jasonantman.com (/etc/httpd/vhosts.d/whatismyip.jasonantman.com.conf:1)
Syntax OK

This is quite useful in debugging vhost problems, especially those pesky times when a request that should go to a specific vhost is being served by the default (in my case at this time, I had two ServerName directives instead of a ServerName and a ServerAlias).

Uncategorized , ,

Using nmap to quickly ping all hosts in an address range

February 8th, 2010

At $WORK, the subnet we use for some of our workstations and test boxes was only recently setup with DHCP. Previously, we’d used IP-by-Whiteboard in the office. As a result, most of the recent machines use DHCP, but there are a few older ones still around using static addresses. I recently had to add a new machine, so I had to go through the process of finding out which IPs are in use and which aren’t (since some aren’t in DHCP).

I decided to be good and update DHCP with records for all machines in the subnet, whether they’re actually using DHCP or not. There’s a quick way to do this with nmap using the options for ping scan (-sP) and always resolve DNS (-R):

nmap -sP -R 172.16.43.129-159

Host ar01-hill-hill.example.com (172.16.43.129) appears to be up.
MAC Address: 00:11:BC:7D:28:0A (Cisco Systems)
Host ccf-hill019-1.example.com (172.16.43.130) appears to be up.
MAC Address: 00:00:AA:63:54:BB (Xerox)
Host ccf-hill019-2.example.com (172.16.43.131) appears to be down.
Host ccf-hill019-3.example.com (172.16.43.132) appears to be down.
Host ccf-hill019-4.example.com (172.16.43.133) appears to be down.
Host ccf-hill019-5.example.com (172.16.43.134) appears to be down.
Host ccf-hill019-6.example.com (172.16.43.135) appears to be down.
Host ccf-hill019-7.example.com (172.16.43.136) appears to be up.
Host speakeasy.example.com (172.16.43.137) appears to be up.
MAC Address: 00:17:A4:13:EB:57 (Global Data Services)
Host ccf-hill019-9.example.com (172.16.43.138) appears to be up.
MAC Address: 00:17:A4:13:E8:17 (Global Data Services)
Host ccf-hill019-10.example.com (172.16.43.139) appears to be down.
Host testmac01.example.com (172.16.43.140) appears to be down.
Host ccf-hill019-12.example.com (172.16.43.141) appears to be down.
Host ccf-hill019-13.example.com (172.16.43.142) appears to be up.
MAC Address: 00:0D:29:59:58:00 (Cisco)
Host ccf-hill019-14.example.com (172.16.43.143) appears to be down.
Host ccf-hill019-15.example.com (172.16.43.144) appears to be down.
Host ccf-hill019-16.example.com (172.16.43.145) appears to be down.
Host ccf-hill019-17.example.com (172.16.43.146) appears to be down.
Host ccf-hill019-18.example.com (172.16.43.147) appears to be up.
MAC Address: 00:1E:C2:0D:C1:98 (Unknown)
Host ccf-hill019-19.example.com (172.16.43.148) appears to be down.
Host ccf-hill019-20.example.com (172.16.43.149) appears to be down.
Host ccf-hill019-21.example.com (172.16.43.150) appears to be down.
Host lordkris.example.com (172.16.43.151) appears to be down.
Host ccf-hill019-23.example.com (172.16.43.152) appears to be down.
Host ccf-hill019-24.example.com (172.16.43.153) appears to be down.
Host ccf-hill019-25.example.com (172.16.43.154) appears to be down.
Host ccf-hill019-26.example.com (172.16.43.155) appears to be down.
Host ccf-hill019-27.example.com (172.16.43.156) appears to be down.
Host ccf-hill019-28.example.com (172.16.43.157) appears to be down.
Host ccf-hill019-29.example.com (172.16.43.158) appears to be down.
Host ccf-hill019-30.example.com (172.16.43.159) appears to be down.
Nmap finished: 31 IP addresses (7 hosts up) scanned in 0.892 seconds

As you can see, the results also (very usefully) include MAC addresses, so it’s pretty easy to update DHCP as needed.

Tech HowTos , , ,