Understanding Post Processing

[slypknot]

I'm a convert from newznab to your new project - have been using Johnnyboy's scripts for many moons...and first let me say - this new project is IMPRESSIVE!  Let me say again, extremely impressive!  The reduce DB bloat, the clean parts, the speed at which it adds... if there was a 'contribute $$' I would click it!

One thing I've never gotten my mind around though is the post processing piece.  This seems to tail behind releases quite a bit - I'm running an i7 quad core / 16GB ram / 4 TB server / Percona with modified my.cnf (I dabble in DBA work) with Ubuntu 12.10 server.  And no matter what my post processing queue seems to hang @ around close to 200k in queue!  I've dedicated 20 threads to it at once and not seemed to be able to keep up with the backfilling/new indexing that is flooding in.

Guess my moronic question would be - What does Post Processing actually do and what does it provide?  The releases all seem to be present and as the post processing loops complete and recycle I do not notice any new releases being added. 

[zombiehoffa]

Donate here: http://nzedb.com/index.php?action=treasury

post processing does stuff like generate screen shots, associate imdb ids (something you need for couchpotato to work), categorize releases, etc. You probably want it. what is your load average like? You may be slowing yourself down by running so many pp threads if they are putting your cpu load way over maxed out. You may want to stop backfilling for a bit while you work your way through the backed up releases that need post processing...


Cheers