Request ID processing

[Miatrix]

I have added a new feature (In the dev branch now) that looks for releases with a name like [123456] some name.  It will look up the name from the database I have accumulated so far.  Mostly a.b.erotica since I was able to get my hands on a good number of requests from that group.

If anyone has a list of request ids and the release name for any groups please let me know.

Here is how it works if your interested....
In update_releases.php there is a new stage (currently 5b) that does this processing. 

• The 1st step looks for any releases where it hasn't looked up the status and it doesn't have a [<some numbers>] at the start of the release name.  It will set these records to a -1 in the reqidstatus field.
• Now it will look for all releases where reqidstatus = 0 and the relnamestatus = 1 (Hasn't been named by another process).  it will read each release and try to parse the number.  If it works it will call the url in your settings to get the release name.  If it finds it it will set the reqidstatus = 1 otherwise it sets it to -2.
  

reqidstatus values (In the releases table)

• 1 = Request ID Found and searchname updated
• 0 = Record hasn't been processed yet
• -1 = Release not in the right format to look up the request ID
• -2 = Request ID Not Found though it appeared to be a valid release

The 1st time you run this it will take a long time to run since it will have to make a web lookup for each release that starts with a [<some numbers>]

Let me know if you have any questions, issues, suggestions....

[Miatrix]

Jonnyboy wrote a threaded script to update things faster (dev branch only atm).

from misc/update_scripts/threaded_scripts
run: python3 requestid_threaded.py

You will want to stop tmux if you are running this until it's done.

[Saner]

This seems to work well, but you are not exaggerating when you say

The 1st time you run this it will take a long time to run

I dont really want to switch to the dev branch, and I was wondering if there is any way to run the script for the first run separately.

I have 900 odd days backfilled, and as you can imagine it was taking a while, after an hour or so I killed it and disabled it for now, but I do want to enable it


Thanks for the work.

S.


//

Edit, i misunderstood :d running threaded now,  will revert back to master when done

[jonnyboy]

Been in master since a few minutes after testing.

[Saner]

Yes, it must have been a lack of coffee

I checked out dev and when i got the changed file list, i thought "Where is it, I cant see the threaded script"

Then I saw it was there, so I figured I missed it.

Glad I am only going half mad, and it was already done when I updated this morning.

I really thought I was going mad, lesson learnt, have more coffee before playing in the morning :S

It looks to be a really good addition though once you are through the initial run.

[jonnyboy]

Mine has taken nearly 10 hours, 4k left.

[Saner]

Blimey!, I am glad I left mine to finish now before I backfilled further.

[jonnyboy]

The threaded script has been integrated into releases.php. Just select it in admin-edit.