Recent Posts

Pages: [1] 2 3 ... 10
1
Just to answer my own question, i needed to run "php populate_rt_indexes.php releases_rt"  to update the sphinx search index.
2
Modifications / bash script to properly rename music releases
« Last post by hanshansen on 2017-02-26, 01:45:35 PM »
Hi,

I solved my problem where headphones didn't find a lot of music releases with the bash script below. The problem occurs when releases are renamed to their preDB name and don't get renamed again when getAudioInfo was successful.

The script works as following:

- gets all id's from the releases table where musicinfo_id AND a predb_id exist (>0)
- loops through all id's and gets categories_id, musicinfo_id and searchname from the releases table and artist, title and year from the musicinfo table

Since the musicinfo is far from 100% correct, the script then checks if all words from the artist and title and the year appear in the search name (= the predb name). Before that, I remove things like everything in brackets [] () from the title.  If all words match, it renames the release to "artist - title (year) category", sets predb_id=0 (so it won't be processed again) and saves the old searchname to the fromname, in case I might need it in the future. You could remove that if you don't see the need of that.

After I ran the script I update the sphinx search index with "php populate_rt_indexes.php releases_rt"


Code: [Select]
#!/bin/bash
nzb_dir="/var/www/nZEDb"
config="${nzb_dir}/nzedb/config/config.php"

db_host=$(grep "DB_HOST" $config|cut -d"'" -f 4)
db_user=$(grep "DB_USER" $config|cut -d"'" -f 4)
db_password=$(grep "DB_PASSWORD" $config|cut -d"'" -f 4)
db_name=$(grep "DB_NAME" $config|cut -d"'" -f 4)

function mysqlcommand {
        mysql --skip-column-names -u $db_user -p$db_password -h $db_host $db_name -e "$1"
}


ids=$(mysqlcommand "SELECT id FROM releases WHERE musicinfo_id > 0 AND predb_id > 0;")

for id in $ids;do
        IFS=$'\n'
        data=$(mysqlcommand "SELECT categories_id,musicinfo_id,searchname FROM releases WHERE id = '$id';")
        categories_id=$(echo $data|cut -d$'\t' -f1)
        musicinfo_id=$(echo $data|cut -d$'\t' -f2)
        searchname=$(echo $data|cut -d$'\t' -f3)

        case $categories_id in
                3010)
                        category="MP3"
                        ;;
                3040)
                        category="FLAC"
                        ;;
                *)
                        category=""
                        ;;
        esac

        data=$(mysqlcommand "SELECT artist,title,year FROM musicinfo WHERE id = '$musicinfo_id';")
        artist=$(echo $data|cut -d$'\t' -f1)
        title=$(echo $data|cut -d$'\t' -f2|sed 's/\[.*\]//g;s/([^()]*)//g')
        year=$(echo $data|cut -d$'\t' -f3)
        IFS=" "
        match=1
        for i in $artist $title $year;do
                i=$(echo $i|sed 's/[][()!,&:\x27]/\ /g')
                echo "$searchname"|grep -i -e "$i" 2>&1 > /dev/null||match="0"
        done
        if [ $match -eq 1 ];then
                mysqlcommand "UPDATE releases SET searchname='$artist - $title ($year) $category',predb_id='0',fromname='$searchname' WHERE id='$id';"
                echo "$artist - $title ($year) $category                      $searchname"
        fi
done


As always, please do a backup before using this script. I don't take responsibility, if something goes wrong.
3
Guides / Re: nZEDb install script for Ubuntu Server 16.04.1
« Last post by archer on 2017-02-23, 03:20:56 PM »
Thanks for this post, it has a number of useful insights which I will be sure to incorporate into the WIP docs.

Feel free to contact me (Archer and niel are my nicks) in IRC with further comments or if you would like to help improve the docs.
4
Guides / Re: nZEDb install script for Ubuntu Server 16.04.1
« Last post by archer on 2017-02-23, 03:18:03 PM »
In step two 'Database Setup' the username was only set to 'nzedbindexer'.  I changed this to 'nzedbindexer@localhost' but did not correct this problem.  If you have followed this guide and did not skip any steps, the file system security should be set correctly.

Ultimately after a bit of research I read an article that instructed me to use 0.0.0.0 in place of "localhost". After discussing this with other users on the forum, I found out that 0.0.0.0 instructs the db to listen on any address.

0.0.0.0 does apply to all TCP/IP addresses, but it is preferable to use a unix socket file when your Db is local as (in theory) it has less overhead and is faster.
5
General Talk / Re: Munin Plugin?
« Last post by ThePeePs on 2017-02-23, 03:15:39 PM »
You're welcome.  BTW, I just updated it again, to include CBP.  I'll make sure to update this thread anytime I make improvements it it.  It still needs a lot of cleaning up, and I might pull the "bigger" category graph, cause it's a longer DB query, and the graph is kinda hard to read.

Any other suggestions comments are welcome.
6
Guides / Re: nZEDb install script for Ubuntu Server 16.04.1
« Last post by archer on 2017-02-23, 03:06:19 PM »
10. Configure the PHP settings

Next press CTRL-W and enter the following:
Code: [Select]
memory_limit
Here comes another problem I had with the install guide.  PHP 7.0 comes preset to the following:
Code: [Select]
memory_limit = -1What this simply means is use any available memory. 

The install guide suggested that this be set to

Quote
memory_limit = 1024M

The problem with setting this memory limit too low is PHP will run out of memory very quickly.  A lot of times you wont know anything is wrong with the exception that pages will not display correctly or settings will not stay set, or worse, the website will crash for no reason. 

After poking around in the log files and then researching what this setting actually does, I feel its safe to leave this at -1.  The setting -1 allows PHP to allocate memory as needed and does not restrict is ability to run correctly.  No this does not mean that PHP uses all available server memory.  It means that it CAN use all available server memory if it needs to.  I have found that PHP is very frugal with the RAM it uses.  If I find this installation is using too much RAM, then I will update this post.  I do positively know 1024M is not enough RAM.

You're correct that 1024 is too little on current machines. The WIP has this set to 2048M. We actually want a concrete setting here, because it is easier to catch problems when they exhaust RAM early instead of when something else uses the remaining RAM and we have to backtrack to PHP causing it.
7
Guides / Re: nZEDb install script for Ubuntu Server 16.04.1
« Last post by archer on 2017-02-23, 02:52:14 PM »
5. Install MySQL

This next command will install MySQL.  The document (in the first post) details out 3 different installs.  I will be selecting the regular MySQL install for my server.

Just paste this code into your putty terminal and answer Y to install all software packages.  This will install roughly 24 packages.

Code: [Select]
sudo apt-get install mysql-server mysql-client libmysqlclient-dev
During the install, you will be prompted to input the new root password for the MySQL user 'root'.  DO NOT skip this step.  Enter a password that you will remember.  You will need this later.

This install should take about 5-10 minutes.

We've changed this to recommend MariaDb and only provide instructions for it. There are a number of reasons for this, not list is just one set of option for beginners to follow.
8
Guides / Re: nZEDb install script for Ubuntu Server 16.04.1
« Last post by archer on 2017-02-23, 02:44:53 PM »
2.  Decide on what to do with Apparmor

I have learned by trial and error that you must make this decision now.  The reason is if you decide to remove Apparmor and wait until step 5 of this document, when you purge Apparmor, you pretty much gut the server and have to start over. 

If you are going to keep Apparmor, then you can skip this step and wait to configure apparmor later.

I have decided to purge Apparmor from my server.

I have also found that the order of purging Apparmor is out of order but probably does not hurt anything at this point.  For clarity sake, here is how Apparmor should be removed.

Remove Apparmor:
Code: [Select]
sudo update-rc.d apparmor disable
Code: [Select]
sudo apt-get purge apparmor
Then I rebooted the server before continuing the software install.
Code: [Select]
sudo reboot

Hi. In my original docs that is how it is. i.e. second step and as you show the instructions. This was changed for the readthedocs documentation, but I have reverted it in my dev branch of those docs.
9
General Talk / Re: Munin Plugin?
« Last post by schlobjob on 2017-02-23, 12:05:09 PM »
Perfect  :D got it working! thank you so much!
10
General Talk / Re: Munin Plugin?
« Last post by ThePeePs on 2017-02-23, 07:41:49 AM »
Oh, sorry. should have included some doc, when you setup the symlink, you need to set them up as follows:

Code: [Select]
/etc/munin/plugins# ls -l | grep nze
lrwxrwxrwx 1 root root   31 Feb 21 23:48 nzedb_api -> /usr/share/munin/plugins/nzedb_
lrwxrwxrwx 1 root root   31 Feb 21 23:48 nzedb_category -> /usr/share/munin/plugins/nzedb_
lrwxrwxrwx 1 root root   31 Feb 22 13:10 nzedb_users -> /usr/share/munin/plugins/nzedb_
lrwxrwxrwx 1 root root   31 Feb 21 23:48 nzedb_releases -> /usr/share/munin/plugins/nzedb_

Each one will generate a different graph.  I am also working on adding a few others, like a parent only category graph, cause the "big" one is a bit hard to read, and CBP, predb, and maybe what's in post-processing.

The main problem is, to get any of this data, requires querying the db, which on larger sites, might cause issues.  That being said, I might look at modding the tmux monitor script to generate a txt file when it updates, and have munin hit that for stats. If not that, then just working off the information_schema since those queries are much lighter.
Pages: [1] 2 3 ... 10