Index bloat can be a major pain in heavy OLTP databases. With Reindex being a blocking operation in Postgres, you can not reasonably reclaim index bloat in a large database by hand without going offline. Thus enter concurrentReindex.php. This PHP cli script will concurrently reindex all non-unique, non-system catalog indexes in a given database. I have only tested this against a Postgres 8.2 database, but it should work in theory against 8.2 or higher. Feedback, enhancements and patches welcome.
Another strike for Polaroid
Called up today to inquire to the status of the replacement TV and found out that a supervisor put the replacement on hold as of April 21st due to needing secondary confirmation from the technician beyond what they put in the computer. That was the last status update. I received no call from them. I couldn't imagine running a business the way that they seem to. I think the next step is figuring out how to best get the word out to never buy from Polaroid and documenting all of the calls, exchanges, broken promises and lack of support I have received in fixing my flat screen TV that was under warranty when it went bad.
Follow-up on PDO and libpq Bug
A while back i blogged about a bug with PDO and prepared statements. After digging a little it appears that as of libpq.so.4 (PostgreSQL 8.0 tree) native prepared statement support was added. If PDO detects you have libpq.so.4 at compile time it will use the libpq prepare instead of its own internal prepare. What then happens is random prepared statements fail with heavy use. Again the only way around this right now is to use libpq.so.3 from PostgreSQL 7.4. Now to figure out who to bug about it. I have a feeling the PDO folks will point to libpq and the libpq maintainers will point to PDO.
Zend Studio 6
I'll admit, I was not happy with the jump from Zend Development Environment 5.5 to Zend Studio for Eclipse 6.0. I've tried other Eclipse based IDE implementations, most notably, Aptana. I found them to be slow, buggy and not very reliable. Eclipse is complicated and the interface designers clearly shouldn't be designing interfaces. I have to give Eclipse its due though, it is very flexible and robust.
I voiced my concern about Zend Studio 6 to my contacts at Zend, including the product manager for Zend Studio. To me it was too far of a departure from ZDE. From a business perspective I understand why they discontinued ZDE and moved to Zend Studio, money. Why pay programmers to maintain the core development environment when you can pay them to enhance an already popular and extendable Eclipse? That being said, as a developer I feel alienated and left out in the cold.
I decided about two months ago to give Zend Studio a fair shake. Not the load it up, poke around in it, find what I disliked and move on evaluation I initially did. What I found since doing so is I like some things about it, but the negative aspects far outweigh the positive.
Eclipse and Zend Studio on top of it is slow, unstable and prone to lockups. The remote development support, mounting remote filesystems over SSH or SFTP is unreliable and unusable. Remote projects and the various Zend Studio specific add-ons feel more like a kludge after-thought than a core part of the overall experience. The frequent lockups, instability and time it takes to work on remote filesystems, in comparison to ZDE 5.5 has made me switch back. Even the random errors in ZDE due to the Leopard upgrade are far less impacting than the issues with Zend Studio for Eclipse.
I do have to hand it to Zend Studio though, and it's more credit to Eclipse than to Zend; The XML editing in Eclipse is hands down better than ZDE.
Polaroid follow-up
Polaroid Warranty Service
Polaroid customer support for warranty service is a joke.
Nearly two months ago, which was two months before the warranty expired for my FLM3732 LCD television, the television stopped turning on. One would think it would be a simple process to get Polaroid to honor their warranty on their products, but thus far, it has not as easy as it should be.
I’m on my 15th call into the support and have faxed the receipt to them 6 times (with confirmation from the machine). They have yet to call me to inform me of any status and they do not have any info on the receipts that I have faxed, each time I call. They have verified the fax number is correct several times. To top it off, each time when you send a fax, they ask you to wait two business days before calling back. They won’t verify it has been received and they insist they must wait until the receipt has been entered into the system before doing anything.
It seems at this rate I’ll never have the unit repaired and should just ditch Polaroid and swear them off for good.
php 5.2.x, PDO, libpq bug
I am working on writing up a test scenario to submit the bug report, but I thought I'd let people know of a PDO+libpq issue I've recently found. The problem manifests itself in prepared statements. Using libpq for PostgreSQL 8.0 or higher, under heavy use, PDO will somehow drop prepared statements before PostgreSQL has set them up, so in the logs, you will see lots of entries like the following:
ERROR: prepared statement "pdo_pgsql_stmt_b6dc71e8" does not exist
The quick fix is to install libpq for PostgreSQL 7.4 and recompile PHP against that. I'll update more when I have additional information.
The Best Spam I've Ever Received
Do I need to say any more?
The Order of The Illuminati under the jurisdiction of the all Seeing Eye, Master Nicholas Brenner has after series of secret deliberations selected you to be an optional beneficiary of our 2008 foundation laying grants and also an optional opening at the round table of the Order of The Illuminati. These grants are issued every year around the world in accordance with the objectives of the Order of The Illuminati as stated by Thomas Paine in 1810 which is to ensure the continuous freedom of man and to enhance mans living conditions. We will also advice that these funds which amount to US$2,000,000.00(two million united states dollars) be used to better the lot of man through your own initiative and also we will go further to inform that the open slot to join the Order is optional, you can decline the offer. In order to claim your grant, contact the Grand Lodge Office secretary David P. Roberts. Email: illuminate@katamail.com Dr Paul Winsdor, Order of The Illuminati The ultimate goal of the Order of The Illuminati has always been to replace the destructive forces of monarchism, nationalism, religion with the productive dynamics of business. Toward that goal, Neo-Tech/Zonpower adds · free-market dynamics for societal decisions plus. · the essentialness of working-class individuals. For, such working-class individuals are disconnected from the elite class, which is intellectually indoctrinated into closed-circle visions of nature. Thus, non indoctrinated workers have retained their childhood capacities for fully integrated honesty -- for wide-scope viewings of nature. While vision-controlled elites, specialists, and philosophers -- including most Objectivists -- have lost their capacities for fully integrated honesty. The main objective of the Order of The Illuminati is making good men better, making strong men powerful. Order of The Illuminati.
PrivatePaste.com Updates
After much neglect I finally sat down today and updated PrivatePaste.com. The space bug in the empty textarea box for pastes is now fixed. File Uploads are now enabled, secure, and can expire. I've added a copy view which also attempts to copy to the clipbboard. In addition I added a download as text link for content. I also did some long needed database maintenance and added some indexes which should speed the site up a bit. If you’ve used it in the past, you should find the new features useful. If you haven’t used it, stop by and check it out!
Graphing Last.fm
I’ve been on last.fm for a few years, since it was called audioscrobbler. I’ve enjoyed seeing what musical listening trends I have and like the whole concept. I was pointed to LastGraph today, a service that goes and gets your last.fm data for a given range and makes a very cool graph of it. Here’s an example for December 2006-March 2007 for me:
Here’s a link to my entire Last.fm history, which I found interesting to check out. Do you have as last.fm account that you actively use? If so, drop your graph in the comments!
Shameless Self-Promotion
Check out my photography prints for sale at deviantart.com. All proceeds go to my Canon 40D fund!
Nighttime Photos
Last night I was inspired by the crisp, clear night and the tree next to my home so I broke out my camera and dusted off the cobwebs. The results are what I think are some neat pictures:
PL/Proxy: A Few Lessons Learned
After using PL/Proxy in production for a bit and working on some scaling and partitioning projects, I’ve come up with a few observations:
- The PL/PgSQL based configuration is flexible enough to not be considered a hack, but the lack of real in depth documentation leaves one to discover why on their own.
- If you have an environment, like mine, where you’re constantly expanding, building out new infrastructure and have the potential to add new pgBouncer servers on your DB tier for PL/Proxy, Do not use the CONNECT methodology. Instead invest the time to write your configuration functions and use the CLUSTER, RUN ON method instead.
- Plan for more time in debugging PL/PgSQL functions. While generally it’s just replacing the body and language in your functions, it does take more organization and testing to make sure your new PL/PgSQL functions and PL/Proxy counterparts work flawlessly in production.
- Single database machine development environments shouldn’t prevent you from testing PL/Proxy functionality. Setup pgBouncer and various databases to mimic your production environment. We’ve skipped this step in development and paid the price during rollout.
- Even with Marko’s suggestions we’ve yet to find a good way to handle the client_encoding issue. Unfortunately what this means is as we scale, we’re still stuck in SQL_ASCII land and the job to convert to UTF-8 just gets bigger and bigger. Note my dirty hack of using a wrapper function kicks out log messages that normal PLProxy based functions do not, which clutter up our Postgres logs.
- It’s solid. PL/Proxy is still everything I thought it was when I rolled it out and I’ve been very happy with the reliability and scaling it buys me.
I’m looking forward to Marko’s next release of PLProxy which I'm guessing will hit sometime in December. If I read his notes right, it may address my client_encoding issue.
Are we really paying these guys?
VMWare Fusion - Strike 1?
After upgrading to Leopard by doing clean reinstalls on all of my boxes, I wanted to get Parallels back up and running. To my surprise, the Parallels installer doesn’t seem to want to install Parallels. At about 2% of the installation it freezes up. In discussing this with a friend, a suggestion of VMWare Fusion was made and I decide to give it a try. VMWare Fusion installed great, and seems to start ok, but alas, when I went to go convert my Parallels disk image, I found that there was no native functionality in VMWare Fusion to do this.
After some brief googling, I found that VMWare has a Parallels converter application. I merrily went on my way to download it, realizing I’d probably have to reactivate windows and such. It was when I got to the download page that it struck me. The VMWare application to convert one OS X based virtualization image format to their own... was windows based. Yup, to convert a Parallels image to VMWare Fusion image, you have to run a windows based conversion tool. Now I ask you: if you’re looking to steal users away from a competing product, doesn’t it make sense to make the tools which make it possible to do so, in the native operating system your competing product runs in?



