What you need to know about Drupal views
Every content management system needs its query builder—an application which creates customizable lists of content elements and present them in a similarly customisable way. Customization will be typically through an admin interface on the website, and the query builder can be as basic as a text box for SQL or as complex as a many-layered GUI across multiple webpages. Drupal, for its part, has Views. D5 has Views 1, D6 has Views 2, and it looks like the forthcoming Drupal 7 will have Views 3.
However, as the criteria for list-building get more and more complex, there will come a point where custom code will produce the exited result far more quickly and efficiently than Views. The problem for the developer is: while simple lists are obviously candidates for Views; and riotously tangled layouts are obviously candidates for your own modules and SQL; how do you cope with the middle ground? How can you tell from the outset that a particular problem will land you in a Views-only cul-de-sac?
Well, I contend that the individual developer needn’t worry, so long as he’s clued up about developing with, not outside, Views. I reckon that the middle ground can almost always be bridged by Views augmented with custom code; unless you’re completely convinced from the start that, say, the logic of what you want is not representable in a query builder, you can start with Views and tweak with customisations later.
But to be sure you can extricate yourself from any small to medium tiger traps that “Views+your spec” might drop you in, it’s wise to know in advance the full range of tools at your disposal. Here are five of what I consider the most important tricks for building a complex Drupal view.
- Views themeing Views has a very modular set of Drupal theme files. The markup and presentation of individual fields, rows of fields, and whole views of many rows can all be modified on a per-view, per-view-format or even cross-site basis. In a particular view’s admin GUI, you can see an entry marked “Theme: Information.” If you click on this, it will tell you what template files the view will try to use, and what files it’s currently using (in bold.) You can copy these files across from the module directory to your theme and rescan to use them; you can even rename them to be specific to the current view or formatting options, and when you rescan the directories, Views should spot the new files.
- Addon modules There are a number of extension modules for views which will very quickly improve view presentation or data flexibility. At the one end is Semantic Views, which allows you to present certain fields in your views with more semantic markup: titles can be h2 elements; contact details can be in address elements, etc. At the other end of the workflow, close to the database, is the Views Or plugin, which lets you swap the normal views filter AND logic (“all items must be published AND have type=blogpost”) for a more inclusive OR logic (“all items must have type=blogpost OR be less than three weeks old”). If you’re not sure whether there’s an addon module for you, ask on the #drupaluk IRC channel.
- Views as a backend Several third-party modules use views as an adjunct: that means that while the module itself is doing something apparently clever with your content, it’s secretly using a view to manage most of the hard work. If you want to maintain arbitrary admin-sortable lists of content, Nodequeue provides a nice interface for doing so. But behind each nodequeue sits… a Drupal view, and a plugin which ties the nodequeue database table to your nodes to produce the required results. Tagadelic, a module which produces tag clouds for e.g. your recent blogposts, has over the years moved from a standalone module (which still exists), to a Views style plugin, so you can build the initial tags view yourself and activate Tagadelic styling and popular-tags filtering once you’re done.
- Views hooks The workflow for views is like the workflow of Drupal itself in miniature: as such, it has its own set of views-specific hooks, including hook_views_pre_render and hook_views_pre_build. All the Views hooks are documented on drupalcontrib.org. So, learn the basic page-view flow, from a blogpost by Mr Views himself, Earl Miles. Pick it apart. See what each bit does in the code. You should eventually be able to find the views hook you want in the code, and be able to modify the view object, its compiled SQL, the results returned, and even the eventual output, all in your own modules.
- Writing your own plugins A Views plugin is less complicated than you might fear. There’s not a great deal of easy-to-find, easy-to-read documentation out there about Views plugins, but try not to let that put you off. A plugin consists basically of a module, a hook function (or two) and a PHP object: the rest is just detail and theming! While that sounds a bit glib, it has some merit: look at the plugins for such modules above as Semantic Views and Nodequeue, and see how they all relate back to an object in a .inc file and a template in a .tpl.php file. You can extend an existing plugin using standard PHP class syntax, and develop it piece by piece by overriding methods one at a time. Views plugins are incredibly powerful, more so than using configurable third-party modules or just interrupting the current view using a Views hook. They get your code right into the heart of a given view’s workflow and let you do (almost) whatever you want. Remember to clear caches if you get stuck: you might even need to resort to a “TRUNCATE cache” at your MySQL command line if your changes don’t seem to be having an effect.
If you’ve started building your listing page with Views, but the spec has ended up getting more complicated, try each of the five solutions above in turn. When the problem at hand suddenly gets a whole lot more complicated, escalate your solution’s complexity by moving another step down the list!
There’s obviously a lot more to developing with Views than a single blogpost could ever really summarize without turning into an extended essay. But there’s a lot more documentation out there—although of a pretty sprawling and hard-to-navigate sort—and the tips above should hopefully give you pointers both for where to start your extending of Views, and also what to google for when you get stuck. Have fun and good luck!
OGN18 in under two weeks’ time
The next Oxford Geek Night is on Wednesday 21 July, in less than two weeks’ time. I for one am really looking forward to it.
We managed out of sheer luck and cheek to bagsy a fantastic speaker for OGN18, the ever-Interesting Russell Davies. Russell was the organizer of the 2007 and 2008 Interesting conferences, and is involved in lots of fantastic projects, including Speechification and Newspaper Club. He’s also a writer for Wired, a speaker at many conferences including Lift 2010, and what one might frivolously call a futorologist or pundit. He’s going to talk about his experiences turning internets into print, and what he’s learned from doing it as part of projects like the Newspaper Club.
Along with our keynote speaker we’ve got half a dozen of the absolute best of local microslot volunteers. There’s talks on topics as far ranging as “designing backwards”, linked data, graphing 19th-century social networks, genomics, CSS and Rotacoo’s Spotify #fridaymix tape. A few new faces and a few established (and deservedly so) local faces. As always we’ll be putting video up on the site afterwards, so there’ll be a permanent record of our high-quality speakers.
Finally, we hope to have space for the Pitches – our sixty-second open mic slots that anyone can volunteer for, even on the night – and a book raffle. All told it should be a great excuse to saunter over to the Jericho Tavern in the July sun / sudden downpour (delete as applicable.) Hope to see all the other Oxford geeks there.
Feeds objects within feeds objects
We’ve been doing a lot of work with the Drupal Feeds module recently. The frontend is nice enough, although the sub-navigation was rendered almost illegible by our theme’s CSS. The online tutorials need work, and the admin navigation needs to be made a bit more robust to layout changes; but then it will be the de facto way for people to consume feeds on their Drupal sites.
The most recent work we’ve been doing involved custom integration with RSS feeds arriving effectively as PHP string variables containing all the XML. This is different from either a file on disk or a remote URL: in fact, we had a Python program creating the RSS file from us via a shell (which in turn, horribly, was hitting a remote Oracle database using cx_Oracle). Feeds was definitely up to the job in terms of power. In fact, it was quite a toolkit of useful functionality, which is Drupal code for “incredibly powerful but almost incomprehensible.
It’s not that the developer documentation for Feeds isn’t decent: it’s pretty good. But it’s limited in scope: it tells you roughly how to expose your own Feeds-like objects to the admin interface, but not really how all those objects interact. Most importantly, we wanted to know what happened on a cron run: this is the bedrock of how Feeds works on your site, after all.
I poked around a bit and this is what I discovered:
Here’s a summary of the above diagram to give you some idea of what’s going on.
- Drupal’s cron creates a FeedsScheduler object and passes it a “job”, which is all the configuration for a feed call, including any configuration that was attached originally to the particular node which defines the Feed. The scheduler creates a FeedsImporter and passes it the job; the importer then creates a FeedsSource and embeds itself in it as a parent. In each case, the method ::work() is called to create the child/helper object.
- The Source object is what now runs the three phases of feed consumption, via its parent Importer. The Source asks the Importer for the relevant Fetcher, Parser and Processor objects: for example, the HTTP Fetcher, the RSS Parser and the Node Processor objects are strung together to turn an RSS feed at a HTTP URL into a set of nodes, one per entry. Each of these have a relevant, verb-like named method: so ::fetch() for the Fetcher etc. The common currency is a FeedsBatch object, which gets passed around and needs to have methods that make it feel like a batch of feed objects.
- After the three phases have run, the Source calls hook_feeds_after_import() to do any tidying, then quits to the Importer, which quits to the Scheduler, which then runs its ::finished() method on the job, and the cron run for this particular feed is done.
When you build a new plugin, you need to implement hook_feeds_plugins() in a module and reference a class file: this class will be selectable in the admin interface for one of the three consumption phases, depending on what class it’s ultimately based on. You should therefore extend existing classes rather than start from scratch: there are abstract PHP classes in the feeds module directories, which give you skeleton “interfaces” which you can then flesh out with relevant functions. But what’s better is to extend e.g. the HTTP fetcher to fetch from a command on disk (which is what we did) or, say, extend the CSV parser to interrogate JSON.
Class hierarchies mean you don’t have to spend a lot of time reinventing the wheel or hacking existing modules until they become unupgradeable; instead you can take existing classes and tweak them through inheritance, experimenting as you develop.
A Drupal view serving multiple tabs
Drupal’s menu hierarchy is a big and complex beast. It acts as both the repository for registered menu callback handlers (and their associated permissions handlers) and as a way of building more mundane frontend menus for people to click round. It serves both static hierarchical side menus and also dynamic tabbed contextual menus: if you’re on a user’s profile page, there are “View” and “Edit” tabs, with “Account” and maybe “Profile” sub-tabs under “Edit”; yet this menu hierarchy doesn’t exist in any real sense. This complexity, and this slight disconnect between all the various bits of menu.inc and menu.module, means that menu often gets exposed to other modules in a counter-intuitive way.
Say you’ve got a listing of content (using Views, naturally). You want this listing to sit at /posts . But you also want an advanced search to sit at /posts/advanced , and you need tabs across the top of the page. Should be easy, right? Create a view, put it on a menu path, in a menu, then create a second view and, oh, I don’t know: put it as a menu tab? As a default menu tab? As a normal menu item under the menu item you just built? Or maybe build a page, and put that on the menu, then create two views as child menu tabs, or maybe one as the default menu tab, or….
After lots of fiddling, I realized you could take advantage of Views’ ability to clone displays within a single view, to solve this fairly straightforwardly.
- Start off by building your “Default” display—the initial display a view gives you—so that it matches the field and filter criteria you want to sit at /posts .
- Then create a “Page” display to handle the standard search. Set it to be a normal menu item, at path /posts . Check it’s on the menu hierarchy.
- Create a second “Page” display with exactly the same configuration. Under “Page settings”, give it the path ‘/posts/default’ and make it a “Default menu tab” with the parent as “Already exists.”
- You now have a tab to handle your default, non-advanced search, and a menu hierarchy entry to tie it to. Further tabs should now be straightforward.
- Your advanced search can now be created as a third “Page” display. As with #3 above, give it a path ‘/posts/advanced’ and make it a “Default menu tab” with the parent as “Already exists.”
OGN 17 on Wednesday
Oxford Geek Night 17 is happening, and it’s happening on Wednesday. That’s this Wednesday. So this week I’ve been running round the office like a headless chicken, and Lawrence has been helping to pick up the pieces. We’ve got posters printed off, and a Pitch sheet drawn up as well. That’s what we record the entries for The Pitch, our sixty-second open mic slots. We’ve still got a few spaces on that so you could tweet me on @jpstacey if you’re interested.
I’m really looking forward to our keynote talks on open-source licensing and user testing from Andrew Katz and Rebecca Gill, but we’ll also have a short announcement to make about the speaker for OGN18, scheduled for three months’ time on July 21st. We’ve got a great keynoter lined up for then but my lips are sealed until this Wednesday (note: lips not actually sealed.)
Along with Lawrence, Nick and Neal will be helping on the night of OGN17 with the tech and video, as usual. Together we’ll be like the four headless chickens of the apocalypse, so it promises to be quite a night.
Greenpeace UK supporter website is a Webby honoree
Torchbox’s Team Drupal has been working on Greenpeace UK’s website for a couple of years now. One of the major single pieces of work we’ve done is to redevelop what was originally a separate PHP application for their network of local active supporters. Greenpeace’s campaigning relies heavily on the enthusiasm of this network: promoting campaigns, raising awareness, even taking part in actions. So it made sense for them to want to move their supporters’ site into the main Drupal site, and take advantage of the permissions framework and content management there to help active members become active online, and also publicize their offline events as easily as possible.
We built Get Active using Organic Groups and a sizeable amount of custom module and theme development. It was quite an uphill struggle, as Drupal 5’s themeing isn’t particularly flexible compared to what’s demanded of websites these days. Also, OG is quite a walled-garden module, with a limited UI and (certainly before D6) very few easy inroads in terms of extending it. Still, we were very happy with the result and pleased that so many of Greenpeace’s supporters, quiet before on the separate site, have become enthusiastic contributors in their new home.
So it was an extra accolade, a welcome if completely unexpected surprise, to hear that the Get Active site was made a Webby honoree in the “Activism” category. Although I was a (decidedly hands-off) technical lead, the real work was done by Garrett, David B, Sam and the designers, who should be really proud of this recognition. Nice one, guys: have a drink on me. Well, on James. Right, James?
Postcode lookup must not suck
Recently my wife and I were trying to work out why she couldn’t submit her address details to a website, even though I could. As we watched her behaviour in filling out the form, we encountered error after error: or rather, exceptional circumstance after exceptional circumstance. And it was clear that very few of the circumstances had been considered, that error handling was the absolute bare minimum, that the form was set up to be almost a trial to use. The postcode lookup part of the process was probably the source of the most unhandled exceptions: difficult if not impossible for the power user to flow through; unwieldy for the standard user; of almost no benefit at all to the web newbie.
People still think of their workflows on the web like they’re workflows. Over here there’s the start; over there is the goal; somewhere in between there might be some intermediate stages, but ultimately you go from over here to more or less over there eventually. It comes as something as a shock to most people that their beautiful webform does not encompass a workflow: the web has holes all over it; the user is a ball bearing and your application is a pinboard:
Above we see a slightly clumsy metaphor for your web application. The end point of your own particular “workflow” isn’t even something visually obvious like the shark’s head. It’s, oh, let’s say that small metallic gate in the bottom right with the red doors (luckily for you) open. The best you can hope for is that the user caroms through your website hitting as few pins as they have to and ends up in one of a trillion end points, that they don’t close the browser, or reach a form error, or silently lose their submission, or navigate elsewhere in irritation.
In fact, it dawned on my wife first, and then gradually on me, that postcode lookups are not intended to directly benefit the user filling them in. Instead, they’re meant to force the user—remember that phrase—to provide a canonical address, and not the address. That is, the user comes to your site with an opinion about where they live and limited good will about your “product”, and the postcode lookup is a mechanism for forcing them to discard the former, while the application as a whole is trying hard to get them to keep hold of the former.
Good luck with that.
Richard Rutter figured out the dirty secret behind postcode lookups—that they’re not for the user—long before my wife and I. In order to mitigate this natural tension between forcing the user and keeping them happy, he’s done a sizeable chunk of work to condense the postcode lookup pattern here. Along with a quite lively and informed conversation in the comments, this post nails much of the core of the pattern that lookup needs. Much of what’s frankly miserable about using a postcode lookup is indeed tackled there, but there’s an important omissions that I think needs dealing with. Roughly speaking, that is: never force your users to pretend to be someone they’re not.
As an example, consider Spotify. Since inheriting a slightly ropey eMac, I’ve been able to listen to Spotify, and I like it. I think Spotify is a gamechanger in the field of streaming music. I’ve heard albums on Spotify that I would never have bought. And yet: I would never consider purchasing Spotify Premium. The obvious barrier for me is that I use Linux, and there are no native Linux binaries for the Spotify desktop client.
People keep telling me that Spotify runs really well on the Windows emulator WINE. I’m sure it does. But that misses a more fundamental point: if something wants me to enter into a relationship with it, commercial or otherwise, I should not have to pretend to be a demographic I’m not in order that the relationship can be properly fulfilling. I’m not a Windows user, and it’s an affront to a paying customer to expect them to make out that they’re a type of user that they’re not if they want to buy your stuff. More consisely: I don’t take offence at the interface requirements; I take offence at what they imply about the respect for my needs as a user.
With that in mind, consider the first decision block in Richard’s workflow:
does the user know their postcode?
As this is the first pin the user’s pinball hits, then this is the one that alters their final resting place the most. This is the critical pin. And what does it tell me, a user who knows his postcode but also knows his address, and doesn’t want to bother with lookup? It tells me that I have to pretend to not know my postcode if I want to be in a situation where I don’t have to put it in. I have to play games with the application, and mask my true intentions. No, the postcode lookup system must allow for users who simply do not want to fill out their postcode. Postcode lookups should therefore begin with a simple choice of two buttons by a traditional form label. The buttons should not make any assumption about the user’s reasons for choosing either route:
Address: [LOOK UP POSTCODE]
[JUST LET ME TYPE MY ADDRESS]
When you press the first button, both buttons should still remain on the page: the user might decide they wanted to press the second one after all. In fact, as we’re probably using AJAX here, the minimum necessary modification to the form is just to add a postcode box (and to move and maybe change that button) like this:
Address: [ postcode ] [LOOK IT UP]
[JUST LET ME TYPE MY ADDRESS]
You should still present the user with the ability to change their address. The “speed bump” of having to press the button is what works in your favour, is what gets you access to canonical data; anything more than a bump will run the risk of walloping the user right off your pinboard.
If at any stage the user clicks on the second button, the webform should then change to this view:
Address: [ Address, either as a set of textfields or as a textarea ]
[ postcode ] [LOOK IT UP INSTEAD]
For simplicity, I’m almost tempted to drop lookup button altogether at this point: the user has made their choice, after all. But you should never make it difficult for users to go back in a workflow, especially when (almost certainly) the browser back button will have been disabled by these shenanigans.
In comment 10 on his blogpost, Richard agrees with the idea of one big textarea for the address: possibly even including the postcode in that textarea! Again, the simplicity is appealing. You could do all sorts with this to make the user’s experience easier: regex matching behind the scenes would retrieve the postcode; the address could even be automatically split into lines rather than setting real estate aside for the user to split them up themselves. It sounds great, but it’s not an established pattern, and I think a lot of users—especially power users—would mistrust it. Better to go with Address 1, Address 2 etc. even though from a data perspective they’re a horror, slightly improved—but, again, made more complex for the user—by labelling the last address line as “town”. But this last detail is up to you. Do some A/B testing. See how it goes.
Richard’s workflow, with the addition of the basic prototypes above, permits us to move towards as usable a system as postcode lookup will ever be. Usability means the least number of pins on your pinboard, and exactly the right pins, the ones that nudge and tilt the user just enough. And so we end up with a system that still satisfies your original remit—to nudge the user towards using your shiny, expensive, time-consuming, postcode lookup service, with all its concomitant costs in development and maintenance—while catering for the users who simply do not want to, who will never want to, and who will actively object to your site giving them short shrift if they try not to use it.
I’ll make a prediction here, that the users who try not to submit a postcode will tend to be the users you want: the digital natives, the users in flow, the people who will buy ten things from a polished user interface without even stopping to think about it. When they reach that very first pin on the board, they briefly want to be your application’s friends on the web. Your application should consider itself honoured. The least it could do in return is be polite.
Save BBC 6Music and Asian Network
Everyone in a particular demographic will now know that the BBC director general, Mark Thompson, has announced a plan to close two of the most Reithian of radio stations, 6Music and the Asian Network. If you’re up in arms about these closures, you can still do something about it.
Kate Butler has an excellent eight-point plan. You don’t have to complete all eight points: this is not a competition. More specifically, J Hunt has a basic pro forma letter and two email addresses you can send it to. I recommend you actually send it to both, as you won’t get a reply from the “srconsultation” one.
In the spirit of not sending anything entirely pro forma, I’ve written my own as an open letter below (and of course emailed it to the relevant contacts). Feel free to rewrite for your own use.
Subject: Response to Mark Thompson’s announcements – NO to closing 6Music and Asian Network
To whom it may concern,
I would like to address proposals announced on March 2 2010 by the BBC director general Mark Thompson. These suggest the closure of BBC 6Music and the BBC Asian Network, among many other changes.
I am a loyal supporter of the BBC and of the licence fee. I believe that the BBC’s television, radio and multimedia outlet easily justifies the licence fee, and moreover that the two stations earmarked for closure in the proposals are exactly what the BBC is about and what make the BBC worth its fees.
Each station fulfils a remit that commercial broadcasting simply cannot and will not provide for. The stations cater for “sizeable minorities”, which commercial media ignores in favour of massive homogeneities. 6Music’s audience are committed, involved, interested listeners who get a lot out of the service, whereas a large commercial station (or even BBC1) simply does not have that audience buy-in. 6Music, the Asian Network and catering for other “sizeable minorities” constitute the most direct routes to the very heart of Reithian, socially active, life-changing public-service broadcasting. 6Music supports live music and niche genres, and provides an access to the BBC’s archive of recordings, thus offering a service not just for listeners but also for the UK’s arts and culture, for which no equivalent commercial alternative exists.
Having said that, I would certainly not recommend the closure of any other part of the BBC in the place of these stations: I am not writing to you to merely express a preference as to where the axe should fall (although I am myself an avid listener to 6Music.) In an ideal world I would rather there be no axe at all: through ringfenced experimentation and risk-taking the BBC has always fostered new talent and fulfilled social functions that again no commercial broadcaster would care about. Any closure at the BBC makes me, a BBC fan, unhappy.
Yet if cost-cutting is essential, then what’s the point in cutting such vital, heartland services for the sake of some £7m and £21m respectively? Such sweeping cuts make for great soundbites, but when BBC1 has a budget of £1400m, and £600m is being brought back into media production by the closure of online services, then it would surely represent far less of a wrench for £30m of savings to come from that £2000m total – around one percent of that total! – rather than getting rid of 100% of the 6Music and the Asian Network channels.
If the BBC can work with its audiences of “sizeable minorities”, rather than neglecting them when the time comes for cuts; if the return of £600m into programme-making can be visibly demonstrated to float 6Music, the Asian Network and even other digital channels; if the BBC Trust rejects the proposals to close these passionately loved stations; then it will find that the listeners to these stations will be active and reliable supporters of the BBC for years to come.
I therefore strongly urge the BBC Trust to reject these proposals, for the good of musicians, of the BBC and of the listening public.
Best regards,
J-P Stacey
Automated backups to S3
Simon Willison put together an excellent short-and-simple backup script over a year ago now, and I’ve used it intermittently to make backups. It takes files, whole directories, or the output of shell commands, and wraps it all up into a datestamped gzip file, before sending the whole thing up to an Amazon S3 account. It’s a nice little piece of kit, in other words.
I found that it worked mostly fine, although it was (a) fiddly to run as a cron and (b) threw occasional errors which I wanted to fix. These two things together meant I wanted to do some proper development on the script and keep track of it.
I wrote a wrapper to help setting up cronjobs, which means it can be scripted using config files in the user’s home directory and . I then decided on a whim to commit all of this to github , which is the online home for a community that’s been set up around the git versioning tool. As befits a DVCS tool, git has spawned a community which is multithreaded and agile and most of all very friendly, with all free repositories available to the public and everyone able to fork off everyone else.
The repository was still a bit empty, as out of respect for the original coder I didn’t commit backup_to_s3.py with everything else. Then, very kindly, Simon agreed to let me bundle his script with everything else, so you can now check out (clone) the “Configured S3 Backups” repository and follow the README to get scripted backups up and running fairly speedily.
I’d love for people to give it a try and offer some feedback. I definitely want to squash bugs and would also welcome new functionality. But the joy of git and github is that if I don’t want to implement any extras, then other programmers can go ahead and
In summary? Configured S3 Backups: give the script a try. Git: give DVCS a try. And github: give DVCS-based online community building a try. Message ends.
Change of plan at OGN16
There’s been a change in the speakers for Oxford Geek Night 16 on Wednesday 17 February. We’re really happy that Chris Thorpe will still be giving his keynote talk, about how social media is all about the social and not about the media. Alongside Chris’ talk will be a keynote from Oxford’s very own Garrett Coakley, who will be talking about how to run a successful online community.
Garrett’s well placed to give this talk, having been an administrator of evolt.org and instrumental in the now very successful Oxford Flickr group, which has recently exhibited in central Oxford. He’s also fresh from discussing these very issues at the Leeds GeekUp in 20/20 form, so it should be pretty exciting. If not, then I know where he lives.
Brillskills. OGN16 is shaping up to be pretty darn fine, though I do say so myself. Be there, or don’t, fool.
Most Popular Articles
Subscribe to RSS
If you enjoyed the post, make sure you subscribe to the RSS feed.
© 2009 TypeBased. All Rights Reserved.
Powered by WordPress. Design by 