| 17:13:51 | | thuban quits [Read error: Connection reset by peer] |
| 17:15:06 | | thuban joins |
| 19:03:48 | <rewby> | I've grab-site'd the userlist by wessel. HCross has double checked my warcs to make sense and I'm uploading them to the IA. |
| 20:13:45 | <rewby> | wessel1512: I was able to get some info out of a contact of mine. Your idea for dictionaries isn't gonne work according to them. |
| 20:20:04 | <rewby> | The best suggestion he had for discovery was to use site:home.xs4all.nl in google. |
| 20:20:10 | <rewby> | But google's a bit of a rate-limiting problem |
| 20:20:30 | <rewby> | JAA floated the idea of bing. But that'd be a complicated query since they like to start repeating results. |
| 20:22:16 | <@OrIdow6> | What have you already done for discovery? |
| 20:22:22 | <@OrIdow6> | https://wiki.archiveteam.org/index.php/Site_exploration |
| 20:25:27 | <rewby> | I've just used wessel's list for scraping so far. |
| 21:29:19 | <wessel1512> | i have been using google bing and duckduckgo with the site:home.xs4all.nl and some dutch wordlist |
| 21:30:12 | <wessel1512> | but i cant seem to find a good scrape for them |
| 21:31:45 | <wessel1512> | the bing scraping scripts for jaa work soft of but isn't really usebel with wordlist |
| 21:37:47 | <wessel1512> | last time i with ziggo i used scrapestorm and google custom search only scrapestorm can't handle the huge amount of urls. and its payware with a price per month of 100 us dollar. |
| 21:48:14 | <wessel1512> | i cued hammer google custom search for 4 hour straight back then |