DocPad Performance & Memory Details

docpad
discussion
Tags: #<Tag:0x00007f10ed12fde0> #<Tag:0x00007f10ed12f840>

#1

DocPad can be blazing fast or horribly slow. This document will go into the various facets of DocPad performance.

Current issues:

  • If you are performing queries inside your documents, it will be very slow. Solution is to use live collections inside your DocPad Config File.

  • If you have 1000+ files or a few very large files, DocPad will be quite slow. Solution is to use the Raw Plugin. #446

  • If you have 1000+ documents (not files), DocPad will get quite slow (1000 jade documents with a layout takes 3 minutes). No one solution. #936

Why can DocPad be slow?

  • Queries in Documents
    • If inside your layout or partial you are doing something like @getCollection('html').findAll(menu:true).models then you are doing a query on the entire html collection every time that layout is rendered. That is sure to be slow (especially times 1000x documents with that layout or partial).
      • The solution here is to make use of Query Engine’s Live Collections as much as possible. This means adding the following to your DocPad configuration file collections: menu: -> @getCollection('html').findAllLive(menu:true) then inside your template doing @getCollection('menu').models. Live collections only perform query comparisons when they need to (e.g. on changes or new models).
  • In Memory Database
    • To keep DocPad’s setup easy, DocPad utilises an in-memory database. That means DocPad keeps files and documents inside it’s memory, that includes their content. That means if you have a 64MB image inside src/static then that is now a 64MB image in memory. It adds up.
      • The solution here is unless you require the file in memory (e.g. for rendering or cross-document referencing) then the file should be inside the Raw Plugin’s src/raw directory instead.
  • Redundant Template Portions/Partials
    • If you say each one of your blog posts utilises a single layout, and that single layout utilises a partial, then for each blog post that is 3 renders that occur (one for the blog post, one for the layout, one for the partial). Times that by each blog post and it adds up.
      • The solution here is if your partial outputs the same content each time it will be rendered, you can add cacheable: true to its meta data, this means that it will now be cached after the first render of that generation and not rendered again (using the cached result from the first render for subsequent calls). This is especially useful if your layout contains other redundant templating, as they can be abstracted out into a cached partial! Awesome.
  • Document References
    • DocPad is intelligent… for the most part. It is able to detect when a document, like a blog post listing, or a stylesheet that is to be concatenated on render, references other documents. This allows DocPad to be able to keep blog post listings up to date by rendering them when a blog post changes. The problem is that while DocPad knows that certain documents references other documents, it does not yet know which documents that document references. This means when you modify a page, DocPad will also regenerate that blog post listing just to be safe. Yes this is dumb, but safe and slow is better than invalid incorrect outdated data.
      • There is no current solution for this.

Other common workarounds:

  • Keeping processing down:
    • Use the standalone: true flag - @todo document the consequences of this and the sideeffects of this
    • Use the referencesOthers: false flag - @todo document the consequences of this and the sideeffects of this

What DocPad has been doing about this:

Memory

Issues

  • #545 memory usage optimisations discussion
  • #527 stream not buffer - risks stuffing up content reading in general
  • #276 raw directory - no risks - completed
  • #528 forget content after it’s needed - risks stuffing up references inside dynamic documents
  • #629 render documents as we need them
  • #788 Get rid of Backbone Models for native javascript objects

Notes

  • watch memory and decrease parallel tasks is another option (requires #528 to be effective) but is silly compared to just doing #527
  • memory optimisations are essential for deployment to dotcloud

Performance

Issues

  • #508 performance optimisations discussion
  • #336 Keep references on which documents reference which
  • #535 suspend collections - closed due to negligible performance impact
  • #538 remove all traces of synchronous file system calls - completed
  • #590 importer speed optimisations
  • #788 Get rid of Backbone Models for native javascript objects
  • implement a “no streams” option in caterpillar

QueryEngine:

  • #24 specific (instead of generic) change listeners

Notes

  • performance improves with native streams over readable-stream inside caterpillar (until we drop 0.8 support, tough)

Migrated from #529


#2

I think I may of hit the jackpot in terms of why Docpad is generating an extreme amount of memory. I’ve been looking at a heap that was generated during renderCollectionAfter using the 1000 document test site I made.

Just a quick understanding of a Heap Snapshot (for those that dont know), it’s a Graph of Nodes and is similar to that of an Adjacency List. Each node has what is known as Edges, each Edge connects to a related Node in the Graph. Each node contains details like type, memory size used etc (i wont go in to the specifics). With this information we can build a tree starting from the root object (i.e. window or global).
Each Node has a distance calculated from the root of the tree, so if we have window.someobject.myProperty then myProperty is a distance of 3 and so on…applications can get quite deep in terms of distances but nothing like what I’m seeing here.

Whats happening in this heap is that there is a node->edge relationship starting at 24 in distance and spamming all the way down to 58000 in distance. Each of these nodes are 8192kb is shallow size, which means (58000-24 * 8192kb) = 460MB (roughly speaking).

So it looks like we have a massive Buffer cascading problem during rendering. This would totally explain why we’re seeing this mass of memory and yet only rendering 76kb of files.

Now looking at a heap snapshot during writeAfter their is about 1000 smalloc entries and their sizes have dropped to around 76 bytes and distances reduced to around 10. This is just after writing 1000 files so it makes sense here.

Over the next day or two I’m going to investigate where the cascade problem starts and hopefully we can nail it.

disclaimer: Having said all this I may have the memory figure wrong, but I’m sure the cascading issue must impact on performance


#3

Sorry I’ve found I got my memory figures wrong (it was late at night honest! haha), but there is still a cascading problem involving 58000 distant properties. It looks like the progress bar is generating the cascade and when putting docpadConfig.prompt = false it removed the cascade and knocked half the memory of the snapshot too.

I also dug in to the cpu profile flame graph and did a search for (ctrl+f) writereq to find a how it was getting called.
Here is one workflow cycle that ends up calling WriteReq


#4

The progress bar is to blame eh?


#5

@balupton looks that way


#6

@pflannery can you try the same test but with DocPad’s new #dev-progress branch: https://github.com/docpad/docpad/tree/dev-progressbar


#7

I will try this as soon as I can. I also found out the smalloc is new in iojs so not sure if this was an existing issue prior to moving to iojs.


#8

I’ve looked at https://github.com/docpad/docpad/tree/dev-progressbar it maxed around 11800 in distance…
I then re-installed https://github.com/docpad/docpad and it now seems I cant recreate 58000 in distance but now max’s at 11800 but this still seems way too high. I also updated to iojs 1.6.2 and it didn’t resolve this.

I’ve raised an issue in iojs hoping it might shed some light. Maybe a similar issue here too https://github.com/iojs/io.js/issues/1075


#9

Note on the status updates.