So this past week, I decided to try the circuits.web framework again. The framework’s author had contacted me about my ranting post that I made last year after a bad time trying to use it.

I honestly couldn’t even remember what the project was that made me want to try circuits, but anyway, I went ahead and recreated (roughly) the situation in the post.

Here’s that code: http://circuits.codepad.org/qJvRYhDX

I was hoping that with the newer release, I’d be able to figure it out. But still no luck. Fortunately, James Mills, the author of circuits stepped in to help out. Here’s his solution: http://codepad.org/N9Nsx7qi

I really don’t see how I could have figured it out on my own — even going through the source. Circuits is so different that it takes a solid understanding of how it works on a fundamental level. And while I think it’s an interesting project, I really am not interested in getting that in-depth with it. Well, at least not now.

I apologize to James for the hot-headed post and I’m hoping to keep cicuits on my radar in the future.

Here’s my yearly biannual update part 2.

So I’ve been working on a new site. It’s been difficult due to a lot of different reasons, but:

0. I’m absolutely staying away from WordPress and PHP.

1. I started writing my own blog engine and I decided about 80% through it that I’d rather use something that already exists and has a userbase and all that. I do, however plan to continue working on it.

2. I looked at a system called blohg which sounded interesting, but I soon realized I’d lose all of my old entries.

3. I settled on Mezzanine, but just haven’t had time to get a theme together and publish the site.

The decision to move away from WordPress is part of an epic rant that I want to save for the publishing of the new site. I was originally hoping to have it launch today, but since I can’t, this is as good a place as any…

Happy 15th anniversary to my humble slice of the internet. :)

Recently, I was reading the subreddits for Python and Django and came across a post entitled, Flasky Goodness (or Why Django Sucks). Then there were some rebuttals, comments, fanboys and trolls. Anyone who’s been on the internet for more than 5 minutes sees these kinds of posts and arguments everywhere. Microkernel vs Monolithic, Emacs vs Vim, Language vs Language, PHP vs Sense, and on and on…

I don’t get it. They never end and they’re more or less pointless.

My response to all of them: everything sucks. It’s just that something suck less than others. The rest is just personal preference.

If everything was good and great with computers, we’d all still be programming in Fortran or COBOL or something. Today’s languages aren’t perfect, and in 20 years we’ll all be wondering how we ever did anything in Javascript or Ruby or Python.

Of course Django sucks. Know what sucked more before that? Subway, Webware, and the other frameworks that have been discontinued. When I first started with Python, I tried several times to work with the Myghty framework. Then Pylons came along and I realized how much trying to set up Myghty sucked. Even though the early versions of Pylons used Myghty for templating, it was still better than trying to use just straight Myghty.

I like Django. It’s been great for some projects. I also like Flask — it’s nice and easy and small and fast. But Flask also sucks. I can make quite a mess of Flask projects if I’m not careful. I can also make Flask projects so complicated that they basically mirror Pylons or Django set ups… which sucks.

OK so I don’t think that every single piece of everything sucks. But if there wasn’t some kind of dissatisfaction with our tools, we’d never get better tools. Just remember that everything we’re using now is, ultimately, temporary.

Wow. Here I was thinking I’d have time to update this site more, but then I suddenly became busy at work somehow. Anyway, here we go with the thoughts:

Anniversary

This site turned 14 on Dec 4th. I swear I didn’t forget this year. I was going to post on the 5th, but I couldn’t think of anything besides, “The site is 14. Yay.” … and I still can’t think of anything better. So… yay.

Drupal

I no longer have confidence in Drupal. Not that I ever had much, but after spending 6 months working with 5 or 6 Drupal 7 sites, I don’t know why anyone uses it or continues to develop it. There are a few reasons to list, but the long and short of it is the database.

Drupal’s “Field” feature is important and necessary for nearly every project, and I use it quite a bit. On one project, I had at least 30 unique fields for different content types. The problem is that for each field, two database tables are created. Yes, not just one table, but two.

So my project that has 30 unique fields now has 60 extra tables. Sixty. This is in addition to the numerous tables that are created by default and the different modules. This creates unbelievably long queries with so many joins that use so much memory and are so slow. I don’t know where that decision came from, but it feels like it was a bolted on solution. Drupal 7 breaks compatibility with previous versions in almost every regard, so why not just rewrite the storage paradigm?

So which CMS then?

I don’t have an answer. I started toying around with several Django based CMSes, Plone, and other Python solutions. All of them have their drawbacks, but mostly, they are far too elaborate and massive for the type of projects I do.

So I’m going to roll my own. I’m aware of the, “why reinvent the wheel,” argument, but if you’re never fully satisfied with the wheels available, why settle?

Currently, I’m thinking about a Python system with ZODB as a storage engine. But…

Python library developers make me nervous

The way I see it, there are 3 levels of  developers for most languages:

1. The core developer — works on the language and/or standard library
2. The library/extension developer — creates projects to ease pains caused by the stdlib, to speed development along and to add specific functionality not provided by the stdlib. E.g., Django, Pyramid, SQLAlchemy, Zope, etc.
3. The application developer — works on projects that integrate various libraries to create something for end-users. E.g., Me.

I’ve been working with Python since 2005 or 2006 — not a very long time. Yet I’ve picked so many libraries that have been superseded, barely updated, or flat out abandoned. To name a few, Spyced, Aquarium, Webware/WebKit, Pylons, Cheetah Templates, ZSI, PyXML, Glashammer, Google AppEngine, mod_python and who knows how many others.

Over the past 6 months, I’ve really had the Flask and Jinja projects grow on me. So much so that I started working on the aforementioned CMS with them. But over the same period of time, Armin Ronacher – the creator of Flask, Werkzeug and Jinja  – has written some things on Python 3 and WSGI that make me worry that I once again bet on the wrong horse.

This isn’t meant to be a dig at him at all; I love the work that he’s done. I just can’t believe my luck. And it’s not just a Python thing. Javascript, C, Assembly, PHP…I’ve been picking libraries this way since I the days when I was starting a project with WinG just before DirectX started to break.

This is why I do not gamble or own stocks.

Ding Dong, the wicked jQuery plugin site is dead

jQuery is getting a new plugin site due to an “accident” on the existing site. I use “accident” loosely, because it was limping along, gasping for air, on life support, and mostly brain-dead since it’s inception. (Do note that the old plugin site ran Drupal)

The new plugin site will be based on github or something.

The point is I plan to have both of my plugins up there at some point.

And that’s about it

I’ll be back before 6 months… maybe :)

I haven’t really had any down time at work lately, and when I get home, I barely have any energy to think about programming. So this site has kind of fallen to the wayside. But, I do have some work related things I can blather on about.

Django is still a pleasure to work with

I developed a site a little over a year ago using Django (1.2) and just completed some updates. Even after updating this code multiple times, it hasn’t become messy. For content oriented sites, Django rocks. Plain and simple. For things more complex, core Django would probably stand up. For the admin tool, I’m not so sure. I couldn’t see it being good in every situation — if you have to customize it too much, you might be better off writing your own admin interface.

I’ve been managing the database schema with South — which is probably one of the best programs I’ve ever used. Everything just works. I haven’t run in to an issue where an automatic update didn’t work, but then again, this site isn’t the most complicated site.

Lots of people don’t understand Pyramid

I have my own personal gripes with Pyramid. But over the last few months, I’ve seen a lot of posts looking for help understanding the “new” aspects of Pyramid: namely, the repoze/traversal/zope portions of it. The typical response is: ignore the parts you don’t understand. So the Pyramid developers spent all this effort, all this time, to improve their framework, write documentation about how it’s better and how it’s different and how the complaints users have about it are unfounded, to say ignore key parts about it and to even go so far as to write a wrapper to make it more like the old framework.

Forget understanding the framework… I just don’t understand its developers.

Do framework developers not understand framework users?

I found this article a few weeks ago — a run down of a presentation by Russell Keith-Magee at Djangocon. My particular interest was in the following quote:

Microframeworks. How on earth can an april fools joke like Flask get actual traction? Turn into a popular framework? Django is lots and lots smaller than zope, but these new ones are even smaller. What is small? What is micro? Could we adopt some? Can we become more attractive? We should think about this.

It’s simple. Flask is better documented than 95.2187% of all the web frameworks available for Python. On the other hand, Django is documented better than 99% of all of them. When it comes to web programming, there’s no such thing as a silver bullet. Django is > 6MB. Some people think that’s too big. Some people don’t like Django’s ORM and/or template system, and rather than spend effort changing it, would prefer to start with something that doesn’t force these things upon you.

Maybe instead of trying to figure out how it gained traction and how to apply “the marketing” of a project to your own, maybe try to figure out why users like it.

In conclusion, WTF?

Apparently, there is a pretty serious bug with Drupal 7.2. Also, a fix has been found and applied in SVN/CVS/Whatever. But, they are waiting for 7.3 I guess to roll out this fix?

Seeing as how I’m pretty unable to focus on anything these days, I have run up a collection of thoughts that have made their way through my damned brain.

Anniversary
This site turned 13 on 12/4/2010. Yes, I’m now in my terrible teens. Next thing you know, I’ll hate everything and complain about it. Oh wait.

Servers
I like Ubuntu as a server more than I like Debian. Specifically because they aren’t stuck on Python 2.5 and PHP 5.2. Which brings me to my next thought…

Python on Linux sucks
I’m not the first to point this out. It’s true though. Specifically, I’m referring to CentOS, which ships with Python 2.4. When I first started programming in Python, oh 5 years ago, I used Python 2.4. Having experienced installing a “side-by-side” version, which has to be done from source, I would rather be stabbed in the eye than do it again.

That kind of crap makes it hard to get mod_wsgi working
And as mod_wsgi takes over in popularity, fcgi deployment examples start to become scarce.

I actually tried out Pyramid
After all of my brain hemorrhaging when it was announced, I decided it may be worth a shot. Boy was I wrong. And by “wrong” I mean “right” (that I would hate it). It seems to be to unclean for me. I read a giant tutorial and ended up with something similar to Django (except it runs on Paste and you can use ZCML to configure it). Or so it seemed. I wasn’t sure where to put model definitions, or where to put what used to be “controller” code. I’m going to stick with Flask or Bottle.

nginx is cool. I guess?
I’ve been setting up tons of VMs lately. On one I installed nginx, php-fpm and xcache to test out Drupal 7. It runs with great speed. There’s no per-directory or per-user configuration for nginx, so it’s pretty much useless as an Apache replacement. Chances of me ever using it outside of a VM are very slim.

Tried out Redis, MongoDB and CouchDB
Redis isn’t related to the other two, but all 3 are something I’ll never get to use in real life.

That’s about it, I think. I did make a pretty neet web app recently, but I’m unsure how I can host it. Just something else to think about over the next 3 months.

So in a rare moment of downtime today, I decided to check out the Pylons mailing list since I’m working on a *huge* project in Pylons. Somehow I missed that Pylons is merging with repoze.bfg.

What does it mean? According to Ben Bangert, it will provide, “a way to easily extend a Pylons project.” What does extending mean here? I have no idea. Apparently, in the few Pylons apps I wrote, I’ve never said, “Gee this is great, but I need Pylons to do shit that it doesn’t need to do.” I guess?

The only thing I can think of is similar to Django’s reusable apps. If I was that worried about, I’d use Django.

Anyway, what does it mean to me? Another rewrite. I started using Pylons at 0.9.4. I wrote a project for it. Then 0.9.5 came out and I rewrote it for that. Then 0.9.6 came out and I rewrote parts again.

So now I’m in the middle of one of the biggest projects I’ve ever done — and I’ve staked my reputation on Pylons — and I’m finding out that I will need to rewrite it if I plan on using the next version of Pylons.

No. I’m not doing that. The Pylons 1.0 codebase isn’t going anywhere (it seems). But it won’t grow anymore either.

I will continue to write my application for Pylons. I don’t plan on using Pyramid. I don’t care how good the architecture is. I’m not worried that it uses parts of Zope. I don’t care how simple the hello world program is. I don’t care about extensibility. I don’t care that it’s “better” than micro-frameworks because they did something “wrong” or “evil” in their design.

I don’t hate Zope, I don’t have any bias against one web framework or another, and nothing in the Defense Document applies to me. I just… don’t… care.

I’m glad the Pylons devs will get around this wall they’ve run in to. Well… until the next wall and the next rewrite, the next merge, the next whatever. I’m staying away.

Just a quick thought on SQLAlchemy today.

Everyone that reads this blog surely knows that I think it is the champion of database libraries despite the fact that time and again they have ditched API backward-compatibility between minor releases (Yes open-source world, 0.x.y is still a minor release). It infuriates me to no end, because I am usually effected and I have to nearly rewrite my application…. or blow it off and stay with an older version until the world itself stops turning.

Anyway, I’ve used SA for many types of applications, but using it in web apps always bothers me. Why? Use of the word session.

Sessions, in web applications, almost universally refer to the user’s “session” — their persistent data, stored on the webserver. Sessions, in SA-land, are roughly the equivalent of a database connection, but they’re not exactly the same. It handles every aspect of communicating with the database server, in a very smart and efficient manner, I might add.

When developing web applications using SA, it can get confusing. Pylons can initiate a project with some code to start your database Session if you choose to do so. And if you plan on using SA in your Pylons application, why wouldn’t you do this? By default, the variable for the database session is Session. Note the capital letter. session is different — that’s where your persistent user-data goes.

It’s simple enough with Pylons to end the confusion and change the variable. I like using dbs (database session). But, now the Pylons documentation and user snippets will be different from your app — which kinda adds some of that confusion back. To make things worse, Pylons’ docs usually refer to the 3rd party module docs — and in SA’s case, all of the examples in the documentation use the variable session.

Obviously this isn’t a major problem. But imagine it’s the first thing in the morning, you were up late and you haven’t gotten your caffeine fix. This could be catastrophic.

I haven’t given up on the projects that I’ve started recently. However, work has been pretty busy and looks to be getting busier. That means the jQuery Plugin Index, that I was in the midst of creating, has been stalled. I still *want* to do it. Very much so. However, I did notice that someone is working on PyPi for Google App Engine. If it gets fully implemented, I think it would be easy enough to fork that and adapt it.

In the meantime, my post on building python extensions in Windows has gotten a lot of attention. I hope it helps people out, but believe it or not, I still have trouble building certain extensions. It’s especially painful if the extension depends on a library that doesn’t have native Windows support. But hopefully, this cuts down on the need for running and maintaining a VM just so you can code fun stuff with Python.

I’ve touched on this a while back, but I never followed through with it. Having some down time at work, I’ve decided to jump in. I want to replace http://plugins.jquery.com.

There are numerous problems with it, and one of the reasons I got overwhelmed by this project originally is because I wanted to fix the site, rather than replace it. So now, I’ve wised up and decided to start from scratch without considering any aspect of how the site currently works.

Here are the problem areas, as I see them, in no particular order.

Browsing through plugins is ridiculously terrible

First, when you get to the page, you get a bunch of categories. Compare this with http://pypi.python.org. PyPi gives a tabular listing of 40 recently updated packages. For the Latest Releases, PJC (plugins.jquery.com) gives a full body of content for each item and goes on for a zillion pages.

Second, from the start page on PJC (with the category listing), the “Browse by Name” tab doesn’t work. The “Browse by Date” tab does work, but what date? The date the plugin was created, or the date of the last release? It turns out this is the same as the “Latest Releases” page, just the tab navigation at the top doesn’t disappear. The “All Plugins” link on the is the same as the “Browse by Name” tab and also doesn’t work.

Lastly, browsing plugins in a category gives a different layout from browsing by date. Why? It’s the same information, just sorted differently and filtered.

Searching is basically useless

Do you know why I’m surprised that people have actually used my timer plugin? Because I can’t even find it myself. Searching for “timer” yields 10 pages of results, and includes plain pages and issue tracker items.

I understand the appeal of having the bug tracker and plugin page tied together, but it’s terrible. A plugin like mine is so small that it doesn’t need a bug tracker. Not to mention that use of a bug tracker is annoying without the use of source control. The plugin author should bear the responsibility of setting up bug tracking, source control, etc. There are plenty of free sites to do that.

The search is easily bombed by adding keywords and tags (which are not moderated). So when I search for timer, the sixth result I get is for dualSlider — perfect for managing timeouts and intervals.

The Rating System

There’s no point to this. The “Top Rated” plugins all have 1-3 votes. Plugins with more votes should have more clout. But it doesn’t really matter anyway. It’s not a popularity contest.

This particular part of PJC will have no part whatsoever in my new project. If there will be any spotlighting of plugins, it will be done by moderators.

Other Data Formats

Right now, there are no RSS feeds for plugins at all. Each plugin should have its own release feed, as well as a feed for all latest releases.

Writing a plugin manager currently would involve screen scraping the existing plugin page to see if there have been any changes. Of course, you have to know the URL of the plugin because searching basically gets no where, and if by some chance you were able to search, you’d have to scrape the search page as well.

That’s why I want to have everything available as JSON. Plugin details, list of plugins by category, search results… the new site has to be highly query-able. PyPi uses XML-RPC to expose their API. JSONRPC might be an option for this, or XML-RPC, but I’ll cross that bridge when I come to it.

Categories

The Categories on PJC are terrible. Not in the way that they aren’t descriptive, but they just suck. They should be hierarchical. For example, “Widgets” and “Windows and Overlays” could fall under “User Interface.” Menus could as well.

I’m not sure how Navigation and Menus are different.

DOM should probably be a child of Utilities.

I don’t know what AJAX means for a category. If the plugin is an AJAX request helper, it should go under “Utilities” or “jQuery Extension.” If it’s something like an auto-complete widget, well it should go under Widgets.

The point is, that categories aren’t very helpful in there current state. I put my Timer plugin under jQuery Extensions, Javascript, and Utilities, leading me to believe that they could all be the same category. I don’t know why Javascript is a category actually, since jQuery encapsulates, rather than extends.

The New Site

I’ve already started. http://code.google.com/p/jqpi (the app page will be http://jquerypi.appspot.com)

Basically, I want to create PyPi for jQuery plugins. I figured using Google App Engine would be nice. Also, knowing my penchant for dragging out projects, I’m coding it for HTML5, since it will probably be widely supported by the time I’m finished.

There are a few things that I don’t know how to do with GAE though. Hierarchical categories, searching, optimization, JSONRPC or XML-RPC. I’ll figure it out eventually, but help is always appreciated. Create an issue, create a wiki page, send patches, join the project, anything. We shouldn’t have to suffer the damned plugins.jquery.com any more.