Author Topic: PreDB Request  (Read 18038 times)

Offline SCROFT

  • Junior Indexer
  • **
  • Posts: 14
  • Helpful: +0/-0
PreDB Request
« on: 2013-06-18, 07:35:16 am »
predb processing never finishes for me… to many rows I think as I imported that 2.5 million rows that was posted.

We need a way for it to do chunks at a time so it doesn't run for 4 days then when I update the site scripts it throws away everything it was trying to do.

Just my 2 cents.

Offline jonnyboy

  • Epic Indexer
  • *****
  • Posts: 1046
  • Helpful: +93/-1
  • Lazzy Trucker
Re: PreDB Request
« Reply #1 on: 2013-06-18, 07:39:48 am »
predb processing never finishes for me� to many rows I think as I imported that 2.5 million rows that was posted.

We need a way for it to do chunks at a time so it doesn't run for 4 days then when I update the site scripts it throws away everything it was trying to do.

Just my 2 cents.
What script is taking 4 days?

Offline SCROFT

  • Junior Indexer
  • **
  • Posts: 14
  • Helpful: +0/-0
Re: PreDB Request
« Reply #2 on: 2013-06-18, 08:03:32 am »
The preDB matching in tmux... never completes cause my DB and preDB is too large. if it tackled things in chunks like a 1000 releases matched to preDB at a time it could progress through them.

Offline jonnyboy

  • Epic Indexer
  • *****
  • Posts: 1046
  • Helpful: +93/-1
  • Lazzy Trucker
Re: PreDB Request
« Reply #3 on: 2013-06-18, 08:09:17 am »
Which pane? I do not run all of the scripts.

Offline SCROFT

  • Junior Indexer
  • **
  • Posts: 14
  • Helpful: +0/-0
Re: PreDB Request
« Reply #4 on: 2013-06-18, 08:25:38 am »
second page... third pane (bottom middle)
The pane says
"
Retrieving titles from preDB sources.
Matching up predb titles with release search names.
"

but never finishes

Offline jonnyboy

  • Epic Indexer
  • *****
  • Posts: 1046
  • Helpful: +93/-1
  • Lazzy Trucker
Re: PreDB Request
« Reply #5 on: 2013-06-18, 08:32:41 am »
Weird. With over 3.5 mil releases and 2.5 mil pre's, it runs in seconds for me. Did you mod the predb.php file?

Offline SCROFT

  • Junior Indexer
  • **
  • Posts: 14
  • Helpful: +0/-0
Re: PreDB Request
« Reply #6 on: 2013-06-18, 09:49:09 am »
nope all files are stock.

i do have 6,193,654 lines for releases.

Offline zombiehoffa

  • Junior Indexer
  • **
  • Posts: 17
  • Helpful: +5/-0
Re: PreDB Request
« Reply #7 on: 2013-06-18, 05:19:55 pm »
no idea if this is at all related, but did you mount your mysql temp dir as a ram disk? I was running into some problems after I did that on an unrelated thing... Probably not the problem, but figured I would ask...

Offline SCROFT

  • Junior Indexer
  • **
  • Posts: 14
  • Helpful: +0/-0
Re: PreDB Request
« Reply #8 on: 2013-06-18, 10:09:14 pm »
not as a ram disk... but i did put the tmpdir on a SSD.

Offline SCROFT

  • Junior Indexer
  • **
  • Posts: 14
  • Helpful: +0/-0
Re: PreDB Request
« Reply #9 on: 2013-06-20, 07:57:02 am »
Monitor Running v0.1r2422:  2 days 17 hrs 48 mins 48 secs
Category                    In Process            In Database     
===============   ====================
predb                             0(0)         2,523,997(00%)
Total              7,571,427(-580,190)      6,331,563(35,334)

still says...
Retrieving titles from preDB sources.
Matching up predb titles with release search names.

Let me know if there is any additional info needed... or need me to test anything.

Offline jonnyboy

  • Epic Indexer
  • *****
  • Posts: 1046
  • Helpful: +93/-1
  • Lazzy Trucker
Re: PreDB Request
« Reply #10 on: 2013-06-20, 08:29:34 am »
Monitor Running v0.1r2422:  2 days 17 hrs 48 mins 48 secs
Category                    In Process            In Database     
===============   ====================
predb                             0(0)         2,523,997(00%)
Total              7,571,427(-580,190)      6,331,563(35,334)

still says...
Retrieving titles from preDB sources.
Matching up predb titles with release search names.

Let me know if there is any additional info needed... or need me to test anything.
Its on the list, just not the top. :)

Offline SCROFT

  • Junior Indexer
  • **
  • Posts: 14
  • Helpful: +0/-0
Re: PreDB Request
« Reply #11 on: 2013-06-20, 08:39:59 am »
k no problem then i can wait.

Is the list posted somewhere by any chance? just so I would know the known issues and it would reduce complaints about known issues.

Offline jonnyboy

  • Epic Indexer
  • *****
  • Posts: 1046
  • Helpful: +93/-1
  • Lazzy Trucker
Re: PreDB Request
« Reply #12 on: 2013-06-23, 11:34:31 am »
k no problem then i can wait.

Is the list posted somewhere by any chance? just so I would know the known issues and it would reduce complaints about known issues.
https://github.com/nZEDb/nZEDb/wiki

Offline SCROFT

  • Junior Indexer
  • **
  • Posts: 14
  • Helpful: +0/-0
Re: PreDB Request
« Reply #13 on: 2013-06-24, 07:36:56 am »
Thanks jonnyboy!