I’ve always liked CherryPy. I thought its routing method was a very novel take on the application routing problem. But I never used it too much because it was too loosely coupled for my purposes at the time.

Fast forward to a few weeks ago when I hear about circuits.web, part of the circuits framework. Event driven, modeled after CherryPy, asynchronous, web-socket support, and loosely coupled — which I need this time.

I’ve spent the better part of three weeks working with circuits.web, and I can say it’s been a resounding disappointment.

First of all, documentation is non-existent. And when I say non-existent, I mean non-existent. The core circuits framework is poorly documented at best. The circuits.web component isn’t documented at all. The closest thing to documentation is a collection of asinine examples that have almost no merit for real life applications, if they even work at all.

Second, I can’t exactly tell if the project is still being worked on. When I first started, I naively looked at the dates on the recent commits to the whole circuits project. The circuits.web portion hasn’t been committed to since August. WebSockets were made a candidate recommendation in December. Nothing in circuit.web’s implementation needs to be changed? That makes me suspicious.

Lastly, the Component architecture seemed like a good idea, but it seems to be thrown together in no logical way. Case in point?

Components handle events. Controllers inherit from Components. Controllers can’t handle events. Or if they can, I couldn’t figure out a way. Who knows why — there’s no documentation.

I was trying to create an Authentication component for an admin controller. There are of course, no examples of this. The authentication examples — that is the ACL, HTTP Auth, and HTTP Auth using /etc/passwd — all use an authentication scheme that is site wide.

What I needed was a way to pick specific actions that would require authorization. I figured out a way by accident, and within 3 seconds found a way around it due to the “channel” member of the Controller.

The channel in circuits.web, I assume, specifies the routing for the dispatcher to find the object. So your Admin Controller object might have a channel of “/admin”. The problem I found, using a Component for forcing authentication, I could bypass the Auth Component by going to, “http://localhost/admin/“. Note the trailing slash.

Change the channel to “/admin/” or “admin” and get a 404 on all admin actions — but the auth component works.

And that’s where I’m at. I should have went with CherryPy on this project, but no, I guess I felt like being disappointed by Python web frameworks again.

Wow. Here I was thinking I’d have time to update this site more, but then I suddenly became busy at work somehow. Anyway, here we go with the thoughts:

Anniversary

This site turned 14 on Dec 4th. I swear I didn’t forget this year. I was going to post on the 5th, but I couldn’t think of anything besides, “The site is 14. Yay.” … and I still can’t think of anything better. So… yay.

Drupal

I no longer have confidence in Drupal. Not that I ever had much, but after spending 6 months working with 5 or 6 Drupal 7 sites, I don’t know why anyone uses it or continues to develop it. There are a few reasons to list, but the long and short of it is the database.

Drupal’s “Field” feature is important and necessary for nearly every project, and I use it quite a bit. On one project, I had at least 30 unique fields for different content types. The problem is that for each field, two database tables are created. Yes, not just one table, but two.

So my project that has 30 unique fields now has 60 extra tables. Sixty. This is in addition to the numerous tables that are created by default and the different modules. This creates unbelievably long queries with so many joins that use so much memory and are so slow. I don’t know where that decision came from, but it feels like it was a bolted on solution. Drupal 7 breaks compatibility with previous versions in almost every regard, so why not just rewrite the storage paradigm?

So which CMS then?

I don’t have an answer. I started toying around with several Django based CMSes, Plone, and other Python solutions. All of them have their drawbacks, but mostly, they are far too elaborate and massive for the type of projects I do.

So I’m going to roll my own. I’m aware of the, “why reinvent the wheel,” argument, but if you’re never fully satisfied with the wheels available, why settle?

Currently, I’m thinking about a Python system with ZODB as a storage engine. But…

Python library developers make me nervous

The way I see it, there are 3 levels of  developers for most languages:

1. The core developer — works on the language and/or standard library
2. The library/extension developer — creates projects to ease pains caused by the stdlib, to speed development along and to add specific functionality not provided by the stdlib. E.g., Django, Pyramid, SQLAlchemy, Zope, etc.
3. The application developer — works on projects that integrate various libraries to create something for end-users. E.g., Me.

I’ve been working with Python since 2005 or 2006 — not a very long time. Yet I’ve picked so many libraries that have been superseded, barely updated, or flat out abandoned. To name a few, Spyced, Aquarium, Webware/WebKit, Pylons, Cheetah Templates, ZSI, PyXML, Glashammer, Google AppEngine, mod_python and who knows how many others.

Over the past 6 months, I’ve really had the Flask and Jinja projects grow on me. So much so that I started working on the aforementioned CMS with them. But over the same period of time, Armin Ronacher – the creator of Flask, Werkzeug and Jinja  – has written some things on Python 3 and WSGI that make me worry that I once again bet on the wrong horse.

This isn’t meant to be a dig at him at all; I love the work that he’s done. I just can’t believe my luck. And it’s not just a Python thing. Javascript, C, Assembly, PHP…I’ve been picking libraries this way since I the days when I was starting a project with WinG just before DirectX started to break.

This is why I do not gamble or own stocks.

Ding Dong, the wicked jQuery plugin site is dead

jQuery is getting a new plugin site due to an “accident” on the existing site. I use “accident” loosely, because it was limping along, gasping for air, on life support, and mostly brain-dead since it’s inception. (Do note that the old plugin site ran Drupal)

The new plugin site will be based on github or something.

The point is I plan to have both of my plugins up there at some point.

And that’s about it

I’ll be back before 6 months… maybe :)

Earlier today, I went looking for a plugin similar to hoverIntent but worked for other events. Turns out there wasn’t one (or at least I couldn’t find one), so I wrote one. Here it is:

Delayed Event Plugin page

Zipped source and example

Enjoy. I hope it’s useful for other people.

Oh, I should mention… As with the Timer plugin, I have no interest in maintenance or support. So, unless there’s a major security problem or giant leaking of memory, I don’t really care. I didn’t put a lot of thought in to designing an API or anything. You should feel free to modify the code as you see fit.

I haven’t really had any down time at work lately, and when I get home, I barely have any energy to think about programming. So this site has kind of fallen to the wayside. But, I do have some work related things I can blather on about.

Django is still a pleasure to work with

I developed a site a little over a year ago using Django (1.2) and just completed some updates. Even after updating this code multiple times, it hasn’t become messy. For content oriented sites, Django rocks. Plain and simple. For things more complex, core Django would probably stand up. For the admin tool, I’m not so sure. I couldn’t see it being good in every situation — if you have to customize it too much, you might be better off writing your own admin interface.

I’ve been managing the database schema with South — which is probably one of the best programs I’ve ever used. Everything just works. I haven’t run in to an issue where an automatic update didn’t work, but then again, this site isn’t the most complicated site.

Lots of people don’t understand Pyramid

I have my own personal gripes with Pyramid. But over the last few months, I’ve seen a lot of posts looking for help understanding the “new” aspects of Pyramid: namely, the repoze/traversal/zope portions of it. The typical response is: ignore the parts you don’t understand. So the Pyramid developers spent all this effort, all this time, to improve their framework, write documentation about how it’s better and how it’s different and how the complaints users have about it are unfounded, to say ignore key parts about it and to even go so far as to write a wrapper to make it more like the old framework.

Forget understanding the framework… I just don’t understand its developers.

Do framework developers not understand framework users?

I found this article a few weeks ago — a run down of a presentation by Russell Keith-Magee at Djangocon. My particular interest was in the following quote:

Microframeworks. How on earth can an april fools joke like Flask get actual traction? Turn into a popular framework? Django is lots and lots smaller than zope, but these new ones are even smaller. What is small? What is micro? Could we adopt some? Can we become more attractive? We should think about this.

It’s simple. Flask is better documented than 95.2187% of all the web frameworks available for Python. On the other hand, Django is documented better than 99% of all of them. When it comes to web programming, there’s no such thing as a silver bullet. Django is > 6MB. Some people think that’s too big. Some people don’t like Django’s ORM and/or template system, and rather than spend effort changing it, would prefer to start with something that doesn’t force these things upon you.

Maybe instead of trying to figure out how it gained traction and how to apply “the marketing” of a project to your own, maybe try to figure out why users like it.

In conclusion, WTF?

Apparently, there is a pretty serious bug with Drupal 7.2. Also, a fix has been found and applied in SVN/CVS/Whatever. But, they are waiting for 7.3 I guess to roll out this fix?

So I’ve been toying with NLTK and generating text. I’ve written a plugin for the crappy-irc-bot project which uses bash.org as a source. I wrote something similar, which uses one of the built-in NLTK sources to generate dummy text in house. I got sick of using “Lorem ipsum…”

And yesterday, I adapted it to use some of the wild stuff that Charlie Sheen has been saying and feed it in to a Twitter account, charliesheenbot.

Right now, it’s just using a Trigram based generator, so it usually doesn’t make a lot of sense. I tried using a grammar based generator at one point, but it was even worse. Grammatically, it was valid. But as far as looking like something a human being would say, no. Turns out that this is one of the more hilarious parts of the English language. Here are two of my favorite articles on grammatically correct but meaningless sentences:

http://en.wikipedia.org/wiki/Buffalo_buffalo_Buffalo_buffalo_buffalo_buffalo_Buffalo_buffalo

http://en.wikipedia.org/wiki/James_while_John_had_had_had_had_had_had_had_had_had_had_had_a_better_effect_on_the_teacher

Seems like a good grammar (that is, a natural language grammar, for use with NLTK) would be able to generate meaningful sentences, but I have no idea how to do that with NLTK, or with anything.

Things like MegaHAL or Cleverbot, if I understand correctly, use a neural net to learn sentence structure based on user input. That’s like research project stuff. I don’t really have the drive to get something like that done. The only reason Charlie Sheenbot exists is because NLTK has all of the stuff needed for it already done.

I’m open to suggestions on improvement though. In the meantime, enjoy, I guess.

The other day, I was inspired to make a quick and stupid HTML5 app. It’s a soundboard that plays clips from this video: http://www.youtube.com/watch?v=TugslL45aXk

Angry Ibex Soundboard

I wanted to test out the HTML Audio element. It works best in Firefox, Opera, or Chrome. Oddly, Firefox doesn’t support MP3 format. I assume due to the posturing over codec licensing and such. All seem to support WAV format, although I don’t know why FLAC support is non-existent.

Anyway, enjoy I guess.

Seeing as how I’m pretty unable to focus on anything these days, I have run up a collection of thoughts that have made their way through my damned brain.

Anniversary
This site turned 13 on 12/4/2010. Yes, I’m now in my terrible teens. Next thing you know, I’ll hate everything and complain about it. Oh wait.

Servers
I like Ubuntu as a server more than I like Debian. Specifically because they aren’t stuck on Python 2.5 and PHP 5.2. Which brings me to my next thought…

Python on Linux sucks
I’m not the first to point this out. It’s true though. Specifically, I’m referring to CentOS, which ships with Python 2.4. When I first started programming in Python, oh 5 years ago, I used Python 2.4. Having experienced installing a “side-by-side” version, which has to be done from source, I would rather be stabbed in the eye than do it again.

That kind of crap makes it hard to get mod_wsgi working
And as mod_wsgi takes over in popularity, fcgi deployment examples start to become scarce.

I actually tried out Pyramid
After all of my brain hemorrhaging when it was announced, I decided it may be worth a shot. Boy was I wrong. And by “wrong” I mean “right” (that I would hate it). It seems to be to unclean for me. I read a giant tutorial and ended up with something similar to Django (except it runs on Paste and you can use ZCML to configure it). Or so it seemed. I wasn’t sure where to put model definitions, or where to put what used to be “controller” code. I’m going to stick with Flask or Bottle.

nginx is cool. I guess?
I’ve been setting up tons of VMs lately. On one I installed nginx, php-fpm and xcache to test out Drupal 7. It runs with great speed. There’s no per-directory or per-user configuration for nginx, so it’s pretty much useless as an Apache replacement. Chances of me ever using it outside of a VM are very slim.

Tried out Redis, MongoDB and CouchDB
Redis isn’t related to the other two, but all 3 are something I’ll never get to use in real life.

That’s about it, I think. I did make a pretty neet web app recently, but I’m unsure how I can host it. Just something else to think about over the next 3 months.

So in a rare moment of downtime today, I decided to check out the Pylons mailing list since I’m working on a *huge* project in Pylons. Somehow I missed that Pylons is merging with repoze.bfg.

What does it mean? According to Ben Bangert, it will provide, “a way to easily extend a Pylons project.” What does extending mean here? I have no idea. Apparently, in the few Pylons apps I wrote, I’ve never said, “Gee this is great, but I need Pylons to do shit that it doesn’t need to do.” I guess?

The only thing I can think of is similar to Django’s reusable apps. If I was that worried about, I’d use Django.

Anyway, what does it mean to me? Another rewrite. I started using Pylons at 0.9.4. I wrote a project for it. Then 0.9.5 came out and I rewrote it for that. Then 0.9.6 came out and I rewrote parts again.

So now I’m in the middle of one of the biggest projects I’ve ever done — and I’ve staked my reputation on Pylons — and I’m finding out that I will need to rewrite it if I plan on using the next version of Pylons.

No. I’m not doing that. The Pylons 1.0 codebase isn’t going anywhere (it seems). But it won’t grow anymore either.

I will continue to write my application for Pylons. I don’t plan on using Pyramid. I don’t care how good the architecture is. I’m not worried that it uses parts of Zope. I don’t care how simple the hello world program is. I don’t care about extensibility. I don’t care that it’s “better” than micro-frameworks because they did something “wrong” or “evil” in their design.

I don’t hate Zope, I don’t have any bias against one web framework or another, and nothing in the Defense Document applies to me. I just… don’t… care.

I’m glad the Pylons devs will get around this wall they’ve run in to. Well… until the next wall and the next rewrite, the next merge, the next whatever. I’m staying away.

I have 2 draft posts written, one of which sings the praises of SQLAlchemy, the other describes some of the pitfalls I’ve run in to with Pylons, but I can’t really think of a way to finish them. So I’ll just summarize them here:

Pylons

1. Documentation still not very good. I have 8 separate sites that I use for reference.
2. Avoid @validate. It is only useful if you don’t want custom validation error messages from FormEncode.
3. If you want orphan control in your models, know that you can define the cascade rules in your ForeignKey fields. Defining them in the call to relationship() will not handle ‘delete-orphan’ unless you use forward referencing.

SQLAlchemy

1. Setting up relationships is so simple, it’s almost criminal. Even many-to-many.
2. No really, it almost seems like witchcraft.
3. Eager loading of relationships is probably the coolest thing ever.

When I wrote raw queries in apps, I hated using joins because it blew up my result table — horizontally and vertically. When I started using ORMs I couldn’t use joins anymore, but I sometimes hated the fact that I knew a million queries were getting sent. SQLAlchemy addresses this through spells and incantations. I know some kind of post-processing is going on because not only can you load an entire hierarchy of objects, but all in one query… with joins. SQLAlchemy maintains the hierarchy and no additional queries will be run at reference time. Witches, I tell ya.

Just a quick thought on SQLAlchemy today.

Everyone that reads this blog surely knows that I think it is the champion of database libraries despite the fact that time and again they have ditched API backward-compatibility between minor releases (Yes open-source world, 0.x.y is still a minor release). It infuriates me to no end, because I am usually effected and I have to nearly rewrite my application…. or blow it off and stay with an older version until the world itself stops turning.

Anyway, I’ve used SA for many types of applications, but using it in web apps always bothers me. Why? Use of the word session.

Sessions, in web applications, almost universally refer to the user’s “session” — their persistent data, stored on the webserver. Sessions, in SA-land, are roughly the equivalent of a database connection, but they’re not exactly the same. It handles every aspect of communicating with the database server, in a very smart and efficient manner, I might add.

When developing web applications using SA, it can get confusing. Pylons can initiate a project with some code to start your database Session if you choose to do so. And if you plan on using SA in your Pylons application, why wouldn’t you do this? By default, the variable for the database session is Session. Note the capital letter. session is different — that’s where your persistent user-data goes.

It’s simple enough with Pylons to end the confusion and change the variable. I like using dbs (database session). But, now the Pylons documentation and user snippets will be different from your app — which kinda adds some of that confusion back. To make things worse, Pylons’ docs usually refer to the 3rd party module docs — and in SA’s case, all of the examples in the documentation use the variable session.

Obviously this isn’t a major problem. But imagine it’s the first thing in the morning, you were up late and you haven’t gotten your caffeine fix. This could be catastrophic.