Author Topic: [SOLVED]tmux script stops updating binaries after X hours  (Read 201926 times)

Offline hydlide

  • Junior Indexer
  • **
  • Posts: 7
  • Helpful: +0/-0
I'm having a strange issue here. My tmux script runs perfect for a couple of hours but then it just randomly stops working. The update_binaries pane seems stuck. If I invoke a CTRL + C in that pane, it'll restart update_binaries and start working again.
I tested this on 2 machines, and I got exactly the same behaviour. It also doesn't matter if running sequential or not, update_binaries is stuck in both occassions.

Please let me know what extra information you need to debug this problem

Thanks!
« Last Edit: 2013-12-02, 09:02:47 AM by jonnyboy »

buzzie

  • Guest
Re: tmux script stops updating binaries after X hours
« Reply #1 on: 2013-07-03, 01:19:44 AM »
It looks that i am having the same problem.
I noticed yesterday that my nZEDb hadnt been updated for 2 days after updating.
After rebooting my box i removed some groups that i no longer need to have.
It started fetching the headers of the remaining groups and i detached my screen.
This morning i saw in the tmux session that it had updated over 2000 releases.
After that.....nothing.
It doesnt restart updating and checking for new releases as it should.
And i have no idea how to restart it all apart from rebooting my box.
pressing CTRL + C doesnt seem to do anything.

Any help is appreciated.

Thanks

Offline Saner

  • Decent Indexer
  • ***
  • Posts: 84
  • Helpful: +7/-0
Re: tmux script stops updating binaries after X hours
« Reply #2 on: 2013-07-03, 05:15:36 AM »

And i have no idea how to restart it all apart from rebooting my box.
pressing CTRL + C doesnt seem to do anything.

Any help is appreciated.

Thanks

Why would you reboot ?

Try

Quote
killall tmux
and than restart it.

Anyway, on my box I have not had any issues, are you sure its not something on the database side of things as opposed to the scripts ?

But If you really need to reboot to clear it, it sounds (this is pure guesswork) that it may be a table locking issue or something that is locking up the database, so a reboot seems to be your only solution (you could also restart mysql by itself of course)

when it hangs I would see what the mysql server is up too.


Offline scottia

  • Decent Indexer
  • ***
  • Posts: 53
  • Helpful: +3/-0
Re: tmux script stops updating binaries after X hours
« Reply #3 on: 2013-07-03, 05:25:24 AM »
I restart MySQL regularly and have no issues or stalling since the last GIT.  I think table locks can cause certain things to hang.

buzzie

  • Guest
Re: tmux script stops updating binaries after X hours
« Reply #4 on: 2013-07-03, 05:29:05 AM »

And i have no idea how to restart it all apart from rebooting my box.
pressing CTRL + C doesnt seem to do anything.

Any help is appreciated.

Thanks

Why would you reboot ?

Try

Quote
killall tmux
and than restart it.

Anyway, on my box I have not had any issues, are you sure its not something on the database side of things as opposed to the scripts ?

But If you really need to reboot to clear it, it sounds (this is pure guesswork) that it may be a table locking issue or something that is locking up the database, so a reboot seems to be your only solution (you could also restart mysql by itself of course)

when it hangs I would see what the mysql server is up too.

Hey,

Thanks for the killall command.
Didnt know of that command thats why i reboot my box if needed.

What can i do to check mysql?

I am pretty much a noob on those things as you might have guessed :)

Offline Saner

  • Decent Indexer
  • ***
  • Posts: 84
  • Helpful: +7/-0
Re: tmux script stops updating binaries after X hours
« Reply #5 on: 2013-07-03, 05:43:30 AM »
watch -n 1 "mysql -uroot nzedb -e 'show processlist;'"

Or you could install mytop

sudo apt-get install mytop

and then configure it.

nano ~/.mysqlconfig

Code: [Select]
user=MYSQLUSER
pass=MYSQLPASSWORD
host=localhost
db=NZEDBDATABASE
delay=5
port=3306
6socket=
batchmode=0
header=1
color=1
idle=1

then run "mytop" or get the tmux script to run it for you
« Last Edit: 2013-07-03, 05:52:37 AM by Saner »

Offline dose

  • Junior Indexer
  • **
  • Posts: 24
  • Helpful: +1/-0
Re: tmux script stops updating binaries after X hours
« Reply #6 on: 2013-07-03, 11:20:43 AM »
I am also having that same issue since cymysql stops at least 10 times a day and nothing besides postprocess shows in show full processlist

Running on funtoo

Offline jonnyboy

  • Epic Indexer
  • *****
  • Posts: 1046
  • Helpful: +93/-1
  • Lazzy Trucker
    • nZEDb
Re: tmux script stops updating binaries after X hours
« Reply #7 on: 2013-07-03, 12:03:42 PM »
I am also having that same issue since cymysql stops at least 10 times a day and nothing besides postprocess shows in show full processlist

Running on funtoo
How can cymysql "stop" it never runs. It is only a connector to mysql.

Offline scottia

  • Decent Indexer
  • ***
  • Posts: 53
  • Helpful: +3/-0
Re: tmux script stops updating binaries after X hours
« Reply #8 on: 2013-07-03, 12:23:18 PM »
The issue started with Cymysql.  Cymysql causes MySQL to stop.  This results in a restart of MySQL regularly to prevent stops.

I am no expert but I don't have issues if I regularly restart MySQL.

buzzie

  • Guest
Re: tmux script stops updating binaries after X hours
« Reply #9 on: 2013-07-03, 01:12:55 PM »
watch -n 1 "mysql -uroot nzedb -e 'show processlist;'"


thanks.
i did that and i got the following:
root    localhost       nzedb   Query   0       NULL    show processlist
in front of root there is an ID # and it was growing every second by 1
it was over 15000
I have no idea what it means.

i just restarted mysql and killall tmux and restarted the script.
curious how long it will stay this time

Offline dose

  • Junior Indexer
  • **
  • Posts: 24
  • Helpful: +1/-0
Re: tmux script stops updating binaries after X hours
« Reply #10 on: 2013-07-03, 01:41:49 PM »
cymysql is not the one stopping. is the update binaries process. it just stops right in the middle until i ctrl c it. I was just saying that it started happening right after changing from pymysql to cymysql

Offline jonnyboy

  • Epic Indexer
  • *****
  • Posts: 1046
  • Helpful: +93/-1
  • Lazzy Trucker
    • nZEDb
Re: tmux script stops updating binaries after X hours
« Reply #11 on: 2013-07-03, 03:33:20 PM »
The reason pymysql was replaced is it has not been updated in a couple of years and was not working for everyone.

Offline dose

  • Junior Indexer
  • **
  • Posts: 24
  • Helpful: +1/-0
Re: tmux script stops updating binaries after X hours
« Reply #12 on: 2013-07-03, 05:19:24 PM »
I am just saying about cymysql to indicate a point in time when it started happening.
right now its stuck and this is what i see in processlist
+--------+-------+-----------+-------+---------+-------+--------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+----------+
| Id     | User  | Host      | db    | Command | Time  | State        | Info                                                                                                                                                      | Progress |
+--------+-------+-----------+-------+---------+-------+--------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+----------+
|  40805 | nzsvc | localhost | nzedb | Sleep   | 28332 |              | NULL                                                                                                                                                      |    0.000 |
|  40806 | nzsvc | localhost | nzedb | Sleep   |     1 |              | NULL                                                                                                                                                      |    0.000 |
| 105655 | nzsvc | localhost | nzedb | Sleep   |  9640 |              | NULL                                                                                                                                                      |    0.000 |
| 105664 | nzsvc | localhost | nzedb | Sleep   |  9640 |              | NULL                                                                                                                                                      |    0.000 |
| 135822 | root  | localhost | NULL  | Query   |     0 | NULL         | show full processlist                                                                                                                                     |    0.000 |
| 135866 | nzsvc | localhost | nzedb | Query   |     1 | Sending data | SELECT p.ID, p.category, r.ID as releaseID from predb p inner join releases r on p.title = r.searchname where p.releaseID is null and p.source = 'womble' |    0.000 |
| 135872 | nzsvc | localhost | nzedb | Sleep   |     0 |              | NULL                                                                                                                                                      |    0.000 |
| 135873 | nzsvc | localhost | nzedb | Sleep   |     0 |              | NULL                                                                                                                                                      |    0.000 |
+--------+-------+-----------+-------+---------+-------+--------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+----------+

Offline jonnyboy

  • Epic Indexer
  • *****
  • Posts: 1046
  • Helpful: +93/-1
  • Lazzy Trucker
    • nZEDb
Re: tmux script stops updating binaries after X hours
« Reply #13 on: 2013-07-03, 05:22:04 PM »
I don't see anything stuck in there.

Offline dose

  • Junior Indexer
  • **
  • Posts: 24
  • Helpful: +1/-0
Re: tmux script stops updating binaries after X hours
« Reply #14 on: 2013-07-03, 05:26:24 PM »
but the update binaries was not moving anymore. was just sitting there at no new articles in one of the groups and would not do anything until i ctrl c that window.