Wednesday 1: Kevin Bridges, "Drupal 7 entities and fields"
Example module
Used to just live in CVS, now on d.o at project/example
Needs help
Install file
No longer need explicit hook-schema – fields API provides it
Revision control no longer needs to be self-rolled
node-example module defines and invokes 3 fields, bundles up a content type and then creates an instance of type node using that bundle.
Field definitions as a structured array
Cardinality is number permitted of this field
Field type – images in core
Define widgets as part of instances
Define displays as part of instances
hook-uninstall deletes fields and instances, and content type, and purges field data
Multiple node delete
All field delete
All field instances delete
Delete node type
Purge
Module
hook-node-info
Hook-menu
hook-help [which has always been of limited use outside of contrib.]
These all look very similar
["hook-help is incredibly useful for users" - really? Or just wishful thinking?]
NodeHook-form for node content form
hook-theme
D7 niceness
Hook-entity-info-alter
Extending the entity definition -our entity is “nodes”
Telling drupal it has custom settings
Hook-field-formatter-info
Say which field types that applies to
Safe values, like in CCK
Demo
[still bloody colons on form labels, so much that one of his labels has TWO colons]
Field formatters are massively simplified from CCK
Really early adopters at theexaminer.com
1. How do I prevent hook-schema being IMplicitly called? Does that happen
2. How do I change a widget later? Do I have to change in the .install file?
All configurable in the interface
3. What if I have the same field on several content types? What happens at purge?
You can still share fields across content types in D7. Purge only deletes unrefererenced data:it knows if data is still referenced.
4. Flexibility of e.g cardinality?
In interface
Tuesday 5: Florian Lorétan, "10 Reasons why you should be using features for your Drupal project"
A lot of blogposts talking about features recently. It’s a wave.
We forget about how good CCK is with its drag and drop
We also forget how hard Views is to understand, but how good it is when you get it
When we show our developers features, they know how problematic config in the database is.
What are features?
They’re little cute monsters which come out of eggs. Er, no.
1. Features puts configuration into code
What can be exported: exportables
views
imagecache presets
ctools panels
ctools open layers
rules
blocks (via context)
variables (via strongarm)
Feature versioning
Start with default feature in the code (like default views)
But tweak features – these tweaks are in the database. Feature will be marked as overridden
Non-exportables: features gives you an API
Content types
Roles
Permissions
Menus
Vocabularies
… Nodes, terms and users using UUIDs… experimental
These need to always be in the database, because there’s no core API to declare it as e.g. a default content type in the same way as you can declare a default view.
What can’t be exported
Input formats
Things that can only be identified using sequential numeric IDs (menu items)
2. Features lets you organize your project
Massive number of permissions
Modules
Views
Allows you to bundle N modules, M views and X permissions into one feature bundle. Features can have dependencies
Feature which is job listing plus job content type
3. Features is integrated with drush
drush features-*
4. Features can be version-controlled
5. Features mean you can work with other people
Features package up config
Feature gets shared; changes work; features get put back into the central repository; second dev gets a newer code version than her database – features puts the feature up for review.
6. Features make deployment easier
7. Features helps Drupal major version migration
We would want to separate out functionality and content
[this is never as easy as it looks, anyway! to the client, it's all HTML]
But a big problem is that there’s actually a separation of filesystem and database, with some in one and some in the other.
8. Features let you build Drupal products
9. Tracking changes your clients make
When you let your clients change things, they break.
But now, when they change things, they override the feature. You can always drop back.
[Have to invest time in ensuring all our own live-site config is checked in as a feature]
10. Features is the best way to put config into code
It’s also used by a LOT of people.
Tuesday 4: Jakob Persson, "A method for getting early estimates right"
Early estimating using prior experience
As you make better estimates the picture comes into focus, and what you find might surprise you.
Forecasts on uncertain specs
Early estimate usually by a single person; in-project done by the whole team
Effort and accuracy – small effort gives you decent accuracy
[We've found this: specs tend to be "fractal" and breaking them down just leads to more accurate small items which total up to very similar to the original estimate]
How made?
Spreadsheet
and dedicated developer
1. Timebox work beforehand
Based on top-level gut instinct guestimate – order of magnitude – assign time
2. Analyse requirements
Look for assumptions client made, and question them
3. Guestimate
Top-level
4. Extract features and write solutions
A deliverable with a proposed solution – not necessarly the MDF
Content type as a feature; modules as features
Cutting features [if the client will let you]
Wireframes good for extracting features
[but can contain a lot of assumptions]
5. Estimate features
Estimating needs to include:
development
unit tests
config
testing
fixing bugs
deploy
Estimation sheet
Can the client cut requirements? Negotiate high-estimate requirements with them
Use a scale [e.g. T-shirt sizes]
Always round up!
1,2,5hrs
3hrs for admins to be notified when links are added – round up to 5hrs
Order-of-magnitude estimates at the higher level e.g. 0.1,0.2,0.5hrs or 10,20,50hrs
You have to take testing and design into account
6. Uncertainty and errors
Large features – large uncertainty
Unknown features – possible error
Offer a pilot study
Chaotic development can worsen errors
inexperienced personnel
Poor coding practices
Requirements not investigated
Inexperienced personnel
Developers going too far
Constraints alongside features
Performance
Stability
Unfounded optimism – tend to underestimate by 20-30%
NO OFF-THE-CUFF estimates. Don’t give a number until you’re sure.
Cone of uncertainty
Reduce by proposing well-tested solutions you have used before
Research solutions by Googling, reading articles and blogposts
Prototype!
Avoid unknown third party integration
7. Final estimation
Degrees of experience
1-5 works fairly well as a scale
Put into the spreadsheet
Apply ranges based on degree of experience
Be careful with low experience – shot in the dark
Overhead time – admin / PM, testing, theming
Culture of awareness of time at the company
If unsure, ask an expert
“What do you do with early guesstimates?”
Be clear to the client: say what your assumptions are.
“How to avoid specification drift at the design phase and still encourage design creativity?
Get designers and developers to work closely together as early as possible. Include them in the estimation process. Team should share the estimate. Designers need early feedback from developers, and developers need to be constructive. Neither should restrict the other, but they need to compromise.
Tuesday 3: Poul-Henning Kamp, Varnish HTTP Cache Server
Usually writes operating systems, but writing
“Never argue with someone who buys ink by the barrel”
Content production not new. Not a lot has changed. Text, images, get it together and deliver and replicate. Now we do it with computers, not humans.
Content creation needs
input methods
flexibly typography
cross-referencing
composition rules
user-generated content
Used to be printing needed massive amounts of weight
Now with “the Heidelberg wing” much lighter.
Varnish delivers content fast and reliably.
100,000 requests per second
Setting goals
only a HTTP accelerator
CM feature set focus
accelerator != cache
Configuration files are hell
What you want is a process diagram of the system with controls laid out on it
So use… basically hooks! Event-driven process using Varnish Configuration Language (VCL)
Instantaneous changes
VCL rocks
compiled to C code – runs at full speed – no observable effect on performance
Load balancing with scripting based on client IP
Managing Varnish
CLI for real-time control
Varnish architecture – one binary running as two processes
Manager runs through C compiler, cacher gets a shared object
telnet :81 to configure in real-time
Performance and speed
not something you add: something you subtract
If you don’t have to text process all HTTP errors, don’t!
virtual memory, sendfile and other tricks
Traditional model with a bus and a CPU on it? Rubbish
These days you have CPUs with multiple cores and shared caches and disks and RAM and pagefiling
Modern model
Caches all the way down
Virtual Page Cache (formerly known as RAM)
Performance pricelist
Things the CPU can do
Things that involve protected memory
Mechanical operations – the real bad guys
Moving round in memory, you can sometimes fight with existing caches
Classical logging expensive
disk I/O
Instead use a shared memory segment, which gets written by the OS kernel
If the process crashes, the kernel writes the logfile. If the kernel crashes, you lose anyway.
Speed up – factor of 100
We should be able to deliver web content at line speeds.
The world’s largest FTP server in 1996 maxed out a 100MBit/s line. So we should be able to do that.
Where does my traffic come from? varnishtop
What is my most popular URL? varnishtop
At newspapers this is a good indicator of which page has the most scantily-clad women
Response-time histogram w/varnishhist
If you get hit by CNN or Reddit you will need a cache.
Real-time statistics via shared memory
X-ray vision does not slow down Varnish!
If squid runs slow, then when you’re fiddling with it you’re slowing it down.
Content management features
Instant purges via regexp
TTL/caching policy using VCL
Load mitigation also using VCL
Header washing – get rid of confusing headers
Understand Vary headers
Edge-side-includes
Purges
cache EVICTION based on exact criteria
understands vary: versions
Bans
cache PREVENTION
Prevent cache hit based on loose criteria
These have a cost
Extra features
including inline C code in VCL
ESI includes
SGML extensions to HTML
CMS system has to spit out ESI includes
This locks you into Varnish, unfortunately
You can do tricks with Javascript to try to unlock you
Lots more stuff
“How easy to configure for multiple vhosts?”
By default, configured on host header
“SSL”
SSL – we don’t do that. Do a lot of things very well, and don’t want to do anything, sort of, eugh. I can’t spot what Varnish can do that SSL itself already does. Would be convenient, but we have to deal with crypto-secrets, and be careful not to spill them. Can’t see it happening any time soon.
“Any change visible at the browser end?”
You’ll see a performance change, obviously… but chunk encoding is the only difference – and that’s an improvement. Some browsers start processing when they have their first chunk.
“Separate VCL files for different virtual hosts?”
I wouldn’t say it’s just because I’m lazy that I haven’t done that yet… a couple of wrinkles in the backend. Trunk now does it right, so we could put that in harmlessly.
“Compression?”
Is stuck in the queue. One of the requirements for 3.0, though.
“Logging?”
Varnish NCSA.
“Varnish 3.0 in terms of timing?”
… That’s a very interesting question, yes. If you set up a feature list, the entire economy could go bust and the release might not happen for five years.
“Shared caches across many servers?”
Generally quicker performance for server A to just ask server B to get it out of its cache, then cache it itself.
Varnish ESI layer in Drupal, for blocks and panels, is on its way.
Tuesday keynote: Dries Buytaert, "State of Drupal"
“Get ready to dance with the elephants” – that happened
Capgemini intranet – Royal Mail
95,000 employees
$9 billion
Been meeting a number of companies
Turner Broadcasting (Ted Turner)
Working with e.g. Lullabot so people are tangibly benefiting from them moving to Drupal
Problems: Config mgt, Code staging
Everyone has this
NY Stock Exchange
Pulled plug on expensive project with closed-source CMS
trying to hire 50 Drupal people
REI Systems – prompted by govt Drupal adoption, hiring lots of Drupal people
Drupal unstoppable (?)
Job trends exponential for Drupal [not the same as Google Trends...!]
Powers more than 1% of the web
Cockroach – hard to kill, but not necessarily relevant
Can aspire to become a bigger platform
Richness & reach
Linear extrapolation for D7 release date – September 15
Logarithmic – October 15
Feel good about D7. More stable already than D6.0 release – test framework
Horse’s hoofs – ever all off the ground? continuous photography by Muybridge answered the question, but then began the moving picture industry.
So OS CMS are changing the way people develop websites, and we never expected that.
Drupal 11 – 2020
d.o content will grow by a factor of 11
60,000 modules? Current module choosing solution will not scale
Module competition is healthy, but module duplication is a problem for the end user
Thirty slideshow solution modules already
But we’ll be running… 10% of the web?
Linux has 20% market share in servers
Firefox has 25%
It’s possible
The community is the solution.
Problems look easy in retrospect. The problems ahead are no harder to solve than the ones we already solved.
[Now standing room only in auditorium, give or take.]
10% market share influences the market.
Print magazine dedicated to Drupal?
Ad on TV? But TV and print might be dead in 2020…!
If we get that market share, we’ll learn how to promote Drupal.
Sex scandal for Drupal in 2020? For that to happen, we really need to move beyond the engineers….
Grow beyond technical people
amazon.com running Drupal?
cnn.com running Drupal?
Jeffrey Zeldman is promoting Drupal?
apple.com running Drupal?
I know Apple are overpriced, and overhyped, but I still buy them. Apple create an experience that people want to buy.
We need to talk about experience, and stop listing features.
100 Turner-like orgs that are using Drupal?
Drupal will be how Mum puts up an advert for her tuperware party?
More SAAS sites than self-hosted sites.
Distributions will be downloaded more than core.
25% of all Drupal modules will be written originally in Chinese. Hardly any of us here would be able to review the code.
Arabic web grew 1900% 2000-2009: English-speaking web only grew 240%. So…?
Weekly multi-day Drupal training in every major metropolis?
Drupal courses taught in universities?
Drupal certifications on resumes?
Managing media in Drupal becomes as easy as managing text
[THAT Wired graph...]
Delivering content to multiple platforms. iPads, wifi router admin software?
Integration
People will connect to people, sites to sites, data to data
Drupal adopted by enterprises – config mgt, code staging, content staging, workflows, content federation [?]
Example: large website, performance testing by IT team. Developer turned off page cache. IT people had no way to know. Want to be able to lock down those settings.
Ecosystem will look different – some really large Drupal shops (500 developers?)
“I know everyone important in X” will end
Some of the large 500+ person shops will contribute to PHP
Drupal Assoc has a physical office and 10 full-time employees
We have to still have young people in the community
We have to keep attracting good developers. We need a hacker culture.
We’ll have a lot of barely competent morons. We have to deal with them.
We can scale as a community. It’s possible. It’s all about the culture.
We need a culture of sharing and collaboration
of passion and believing (and issues with 200+ comments)
where people self-organise
where we innovate (support for Cassandra, RDFa)
of execution
of having fun
“Fixed release cycle?”
Shorter release cycle – D7 was too long – but doesn’t have the answer right now. Every model has its own challenges. Fixed cycle means no major architectural improvements as easily.
“Dries and Acquia in 2010?”
No intention of stopping. I never get tired of Drupal. I love what I do. As for Acquia, make successful company but be part of the community and the ecosystem.
“Will Drupal be OO?”
Will be a lot MORE OO. But let’s see. Ultimately it’s up to the community.
“When Drupal is no longer for developers, who wil ldevelop Drupal?”
I hope we all will. I will. I know at least a dozen people in this room who will…. You’d better.
“Human counterpart to automating testing of 60,000 modules?”
Part of the redesign. It’s a good idea, but it’s subject to gaming. But I’m a believer in automating what you can automate.
“Will Drupal still run on PHP in 2020?”
That’s a good question to ask Rasmus tomorrow!
Tuesday 1: Peter Wolanin and Jakob Suchy, "Drupal security for coders and themers"
Your own site – you have a duty not just to your data and your users, but also your casual visitors – untrusted sites rapidly get blacklisted by the Internet as a whole
APIs – if you use “known APIs” [what does that mean?] you have security “guarantee” [what does that mean too?] but not if you use other functions.
Resources – drupalsecurityreport.org Cracking Drupal (book recommended by webchick, tech ed by chx)
Across all apps XSS is the largest vuln overall; SQL inj is the next
In Drupal XSS is by far the most common vuln – themers should be most worried
Update, update, update – install module for D5
Have a consistent method for updating the site and stick to it
Email address on admin/reports/updates/settings
Email every day – you’d be blind to it
ALWAYS backup before update
Don’t use insecure tools – FTP, http. Use https for admin via securepages
Avoid Total Commander at all cost. It stores your FTP passwords in plain text, and there’s a virus which targets it.
Server security updates
XSS – user input is dangerous. Even if you trust the user, don’t trust the text.
Default format MUST be filtered. e.g. comments
check_url, check_plain etc. if you don’t know the format, then filter_xss at the very minimum.
Demo
Moderated comments just make it worse
Moderated comments means that an attacker knows the admin is going to go and moderate.
So Javascript can make a form POST and replace your admin user’s email address
Apache JIRA hack story
You can decrypt passwords with rainbow tables
Themers are the frontline – be careful
Preprocess functions have responsibility to escape. Modules have responsibility
So use $title in node.tpl.php and complain to module maintainers if the content is unsafe!
Theme functions in D6 can be unsafe
Use tpl.php if possible
XSRF
post IMG tags which run JS on a 3rd party site.
GET and POST are vulnerable
Use FORM tokens. Can’t be accessed by browser security model.
Confirmation forms for destructive actions, to prevent people joining groups!
drupal_confirmation_form() !
Security vulns in the past and risk levels
You’ll be in good company
If you have a security problem, don’t fix it. Don’t tell people: tell security@drupal.org
Arbitrary Code Execution using preg_replace with /e [!]
SQL injection – don’t even need to be able to write, to compromise the site. Just read.
Access Bypass – complex access privileges can bypass
Directory Traversal – put locations with “..” or just simply “/” or “\” (Windows) and retrieve e.g. /etc/passwd or code execution
- What do known APIs mean? What does that mean for a given contrib module? Is that just core? Because we get SA advisories for contrib modules, so that’s not clear.
- If you’ve got custom formats, what’s the most important filters within them for avoiding malicious code? To switch on – whitelist html? To switch off – PHP interpretation?
- Can we have that demo somewhere?
- Salt encryption in Drupal re: rainbow tables?
Unconference: Andreas Haugstrup, "Drupal for beginners"
Really Drupal tips. (Rich: turned into an Agony Aunt session!)
Theming the same content type differently in different locations is hard. But Display Suite is a replacement to Panels that can help.
Can use Panels for complex layouts, but not recommended as a complete replacement for blocks. Any alternative to Panels welcome.
Node Composite alternative to Panels. Easy to use.
Rich: Display suite + context nicer than Panels? Definitely need to try that!
Unconference: Mark Gazel, "Wordpress"
Wordpress began as a blogging platform. Best for single-person and small company sites. Can now manage a multi-user network of many sites.
Five-minute installation (but see Hagen’s talk) and easy upgrade. Also one-click plugin upgrading
Wordpress spinoffs – microblogging, bbPress, BuddyPress, Gravatar. WP not intended for large scalable websites (although it can do it) but it has more flexible uses.
“Wordpress community involvement? in Drupal sometimes hard to understand how features get into final release. Same?”
A bit! Lots of routes into core code. .org is plugins, themes, docs etc; .com is the SAAS and makes money for Matt Mulenweg’s company
What can D learn from W?
Hard to say. Don’t know D. But maybe: avoid bloat. We moved import tools back out of core because it was only going to be used once, and would otherwise be bloat.
“Good to have company driving it?”
It’s still Mullenweg and he cares about open source. Recent discussions with a non-OS’ed theme developer led to the developer agreeing to OS his code.
When Wordpress does one-click, it gets one-click right. Really right. Drupal site upgrades are a decent experience but not straightforward.
Avoiding bloat is basically about hard decisions. One person’s bloat is another’s killer feature.
Wordpress’ main strength is its ease of use, but that restricts what it can be used for. Drupal can learn a lot from it, but should never try to just be “Wordpress Plus”
Unconference: Christian Jul Jensen, "Typo3"
Typo3 features
Typoscript is NOT a scripting language. A configuration language. Learn a config language before using the CMS? thinks it’s worth it.
Template view- looks like RegEdit! Creating a hierarchical menu – using scripting. Also things like creating users.
List module, like basic Drupal views, but has multiple editing built in.
Templating module, assigning templates to the subpages of given pages. Flat HTML. Define structure in interface. Map fields to… tokens in the templates? This is like blocks in Drupal.
“Content in Joomla and Drupal is always a bunch of objects. In Typo3 it’s more like bundles of text, like Perl” – Hagen Graf
Wil be releases every six months, like Ubuntu r cycle. Next release is on stability, usability, simplicity. 4.x branch, trying to converge on new 5.x branch.
Typo3 Phoenix is a major, OO, MVC, DBO-abstracted rewrite with a new templating system. Live demo.
If someone could program Typoscript reliably, I’d trust them with Drupal module and theme development. So this still feels like a hobbyist CMS, where the dev is the main CMS user, even though it can probably power non-hobbyist websites fine.
Unconference: Hagen Graf, "Drupal, Joomla, Wordpress"
Joomla, Wordpress and Drupal
Hagen Graf (writes on all three)
What site is what platform? Hard to tell. Katy Perry, Atlantic Southeast Airlines, Schweizer Illustrierte, etc.
“Have to show us the source!” So does it matter?
Drupal has no showcasing, unlike J and W.
Try to download Drupal 7… Hard to find.
Huge number of modules
Google trends… all except W going down gently, W pretty flatlined.
All three: installation to a content type with an image.
Joomla – smooth, will even create db. Slight glitch removing installation directory. Sample data.
Wordpress – briefly unfriendly. Won’t create db for you. If you do create it, what now? When you’re done sample blogpost and page.
Drupal – don’t know what an install profile is, so whatever. I can add several databases? No db – big kaboom. But no obvious explanation. When you’re done no sample content but D7 gives you routes to add content.
SAAS for W and D, not J
Create content
Joomla – a lot of fields. Permissions on article – might not scale? Can’t put it on a menu here. If you’re a beginner, menu manager will put you off. Print/send – you can’t turn off. Media directory, a bit like iMCE.
WP media gallery is right there.
D7 complicated to set up a field for images. Nicely configurable but hard to find. Upload a big image. Wow, that’s a big image. How do I change its display? Display fields on the content type, well, people don’t always find that.
What if the client wants the image floated? Well, you can’t do that without editing the templates, which you can’t do in the browser.
D7 possible to install modules through the interface, even via URL. But e.g. WYSIWYG module – how do you get the third party editor? Answer: you can’t. WP plugins very nice e.g. upgrade akismet, find new plugins. Joomla has confusion of modules, plugins and components, and is just not there. Joomla can’t add comments…?
“How can we improve Drupal to get this audience?”
Agencies don’t always know how to incrementally improve a site. They change incrementally but they’re not improving it. Six months later it’s a mess. We need a core like Wordpress. It just works.
Too many agencies and products and programming. If you come from the outside you think, wow, it’s like Java!
“But if you extend”
Well, everything can be solved with ten modules and some tweaking and some code… But that’s not audience-ready. Joomla templates are strongly MVC. Customizing is easier. But Joomla has no clear competitor for views. W has fields on a per-article basis, but you can combine that with a plugin. Field to field dependence in drupal is possible but hard.
Generally D7 UX is less fun than it looks. Lots of configuration but frustrating to find. Not media-ready out of the box. But when you can find it, configuration is better to use than it used to be. When you use some of the whizzy features they can fall apart. Does they get user tested by non-programmers?
Most Popular Articles
Subscribe to RSS
If you enjoyed the post, make sure you subscribe to the RSS feed.
© 2009 TypeBased. All Rights Reserved.
Powered by WordPress. Design by 