[OUTDATED]Configuring IRCScarper w/Latest ZNC - Update from Github Guides UBUNTU

[hernando]

I had a bit of a hard time configuring ZNC and IRCScrapper to properly work for me with current versions of ZNC. With the great help of Wally73 and Kevin 123, I was able to get it working. Some of these thing do not appear in any of the guides, so I thought of updating the process to get this to work. This will be a big copy paste from all guides out there.

EDIT:
You may first want to follow the top part of this tutorial to merge the dumps of Predb releases.
http://forums.nzedb.com/index.php?topic=1860.0
Just come back here, when the tutorials start talking about installing ZNC.

Setting Up ZNC Did this with version 1.6.0

Install the required app for adding PPA's:

Code: [Select]

sudo apt-get install python-software-properties
Add the PPA:

Code: [Select]

sudo add-apt-repository ppa:teward/znc
Update your sources:

Code: [Select]

sudo apt-get update
Install ZNC:

Code: [Select]

sudo apt-get install znc znc-dbg znc-dev znc-perl znc-python znc-tcl
Run the ZNC config creation wizard:

Code: [Select]

znc --makeconf
Here is where it is different from what the guides explain:

Listen on Port: 6664
Listen Using SSL: no
Listen using both IPv4 and IPv6: yes

Username: (use a unique name; that is not close to your real name). For this example, I will use: moviegoer

Password: something unique and somewhat secure - I will use mypass. Confirm the password.

Nick: same as username
Alternate nick: I used only letters: moviegoers
Ident: same as username

Real name: something fake like: Mister Tee
Bind host: 0.0.0.0
Setup Network: yes

Network Settings:
Name: synirc
Server host: contego.ny.us.synirc.net
Server Uses SSL: yes
Server port: 6697
Password: leave empty
Initial Channels: #nZEDbPRE

Launch ZNC: yes

Go to your web browser and enter the ip of your server like this:

Code: [Select]

http://192.168.x.x:6664replace x's with the correct ip of your server. Log in with your username and password from above.

Go to Global Settings
Change Max Buffer Size to: 1000

Go to Your Settings
Scroll to Networks and you should see one with name synirc. Click on Edit.
You should see all you nicknames, ident and real names that you typed earlier. Make sure "Active" button is checked.

Under Servers of this IRC network, have the following 2 items in the box:

Code: [Select]

contego.ny.us.synirc.net +6697
toronto.on.ca.synirc.net +6697
Arrow to bottom under modules, and check on:
chansaver
keepnick
kickrejoin
nickserv
perform
simple_away

Click on Save and return button.

Arrow to bottom section Modules and check the following:
chansaver
controlpanel
perform

Under Default Settings:
Channel Modes: +stn

Click on Save and return button.

For good measure, I would reboot your server. Not sure if this is necessary.

Setting Up ZNC

Once your server has rebooted, launch ZNC by typing this on the terminal:

Code: [Select]

znc
Copy settings_example.php to settings.php:

Code: [Select]

cp misc/testing/IRCScraper/settings_example.php misc/testing/IRCScraper/settings.php
Edit your settings.php:

Code: [Select]

nano /var/www/nZEDb/misc/testing/IRCScraper/settings.php
Change values per below (watch the single and double quotes ' and ":

in $username set to your username (in my example I used 'moviegoer'
'SCRAPE_IRC_SERVER', '127.0.0.1'
'SCRAPE_IRC_PORT', '6664'
'SCRAPE_IRC_TLS', false
'SCRAPE_IRC_NICKNAME', "$username"
'SCRAPE_IRC_REALNAME', "$username"
'SCRAPE_IRC_USERNAME', "$username"
'SCRAPE_IRC_PASSWORD' use the one you use to log into ZNC, in my case it was 'mypass'

Optional - this uses Regex - in my example, it filters out releases that are in German and porn.
'SCRAPE_IRC_TITLE_IGNORE', '/\.(German|XXX)\./i'

Save and exit the file.

Go to your IRCScraper folder

Code: [Select]

cd /var/www/nZEDb/misc/testing/IRCScraper
Start the scrape.php by typing the following:

Code: [Select]

php scrape.php true false true
If it all goes well, you should be getting a lot of info on the screen... If you see something like what is in the image below:


We have to change a few things. In the screen capture above, you need to copy both of the signature that we see on the screen... its the long line of numbers... DO NOT use the ones in the image above, use the ones you see on your terminal screen.

Go back to the ZNC URL link. Click on
Your Settings. Go to the Networks and Edit the synirc network. On this screen, paste your numbers in a similar fashion to screen capture below:



EDIT: Thanks to kevin123, you may want to also paste the numbers below. Just make sure you do not repeat any:

Code: [Select]

60:84:3f:6a:02:c3:3f:87:be:7e:f7:85:2b:38:c2:f6:78:fc:27:02:76:03:f0:f7:48:09:da:14:d3:87:38:64
67:7b:f5:25:0d:c0:2d:06:b8:57:2a:ef:9f:5c:2f:c9:48:e9:17:f0:43:22:2e:67:0a:56:ca:f8:ee:98:79:71
72:26:00:f7:f0:7f:1d:13:a6:20:88:73:ba:42:6c:8b:5e:ef:fd:04:b3:98:90:f7:23:63:bf:08:46:6d:2e:41
b3:75:8a:5a:a7:ed:5e:ef:22:45:d4:07:bd:06:32:1e:b8:92:07:49:72:cc:7e:7a:63:fb:3f:e1:92:c3:9a:5a
Once you have pasted the numbers, back on terminal Ctrl-c to exit the scrape.php script. Back on the ZNC, then click the Save and return button.

Go back to terminal and relaunch the scrape.php script:

Code: [Select]

php scrape.php true false true
If it all goes well, you should see release names scrolling by. You are in business. It is my understanding that these fingerprints may change over time, and they will require changing using above procedure.

PLEASE NOTE: I am NOT an expert. I got this working with the help of guys in the IRC chat... I am not sure I can be of much help if you run into trouble.

I have NOT been able to get ZNC to launch at boot time. Every time I reboot I manually restart it with the znc command. I cannot get the tutorials that help you set up a daemon to run it at startup. I know that znc.conf file is stored in my /home folder. If someone knows how to do this, please feel free to add it on this thread.

Good luck.

[johnnyops]

Fantastic write up!!

[kevin123]

Check here for getting it on boot: http://wiki.znc.in/Running_ZNC_as_a_system_daemon

Also this has a on boot guide towards the middle: https://shellfish.io/tutorial/1/how-to-install-znc-on-debian-7/

[Islandhitman]

Good job hernando !

[Wally73]

for the ssl fingerprint stuff
log in as your znc user for ircscraper in your favorite client and do what it says in /znc status window (you probably have to do /znc Addetc a few times )

[david_ritterhous]

my contribution since your guide just helped me.
For start up on Ubuntu, add this to crontab

Code: [Select]

@reboot screen php /var/www/nZEDb/misc/testing/IRCScraper/scrape.php true false true


[yeahbut]

Awesome write up mate! I have been struggling with getting this to work, and thanks to you, it's all good now.

[xeddog]

Hi all -

I know this thread has been idle for a long time, but I was using this process to get IRCScraper working for me.  I found one thing that seems to need changing.  Near the middle somewhere there is this line:

cp misc/testing/IRCScraper/settings_example.php misc/testing/IRCScraper/settings.php

For my installation of nZEDb master branch it was:

cp nzedb/config/ircscraper_settings_example.php  nzedb/config/ircscraper_settings.php

Other than that it is still a very good write-up.

Wayne

[dertbv]

when trying to save the file I get
Writing config [/home/badman/.znc/configs/znc.conf]... [ Unable to open file ]

[AlienResidents]

I'm not sure if this is covered elsewhere, and I couldn't find it, so here is some info for generating the SSL fingerprints for irc.synirc.org (and for that matter probably other SSL sites as well).
I use Linux also, and this is what I run to grab the fingerprints.

Code: [Select]

port="6697"
sig_hash="sha256"
for server in "$(curl -s https://www.synirc.net/servers | grep '/server/' | sed 's/^.*">//g; s/<.*$//g')"
do
  openssl s_client -showcerts -connect "$server":"$port" < /dev/null 2> /dev/null | openssl x509 -"$sig_hash" -fingerprint -noout | awk -F'=' '/Fingerprint/ {print $2}'
done

Once you get that output, you can copy/paste that into your znc trusted fingerprints webamin textarea as shown by hernando.

I'm considering to create a scheduled job using irssi to connect to znc directly to add / update / remove trusted fingerprints for irc.synirc.org if / when they change.