| 00:39:48 | <pabs> | arkiver: would the DPoS be able to handle wikis that require a custom JS-based cookie, maybe by hard-coding it? like verified=1 for https://www.exotica.org.uk/wiki/Main_Page |
| 00:39:55 | <pabs> | klea++ |
| 00:39:56 | <eggdrop> | [karma] 'klea' now has 5 karma! |
| 00:41:04 | <pabs> | klea: see the AT wiki page about Cloudflare, it has details |
| 00:43:10 | <klea> | pabs: thx |
| 00:43:50 | <klea> | pabs: i suppose by hardcoding it would work |
| 03:07:52 | <@arkiver> | pabs: i think we can do that yes, how often is this required? |
| 04:52:43 | | DogsRNice quits [Read error: Connection reset by peer] |
| 12:28:19 | <klea> | yzqzss: could you make the wikiteam3uploader have a flag to delete after upload like the dokuwiki one has |
| 12:40:40 | | @imer quits [Ping timeout: 256 seconds] |
| 12:50:17 | | imer (imer) joins |
| 12:50:17 | | @ChanServ sets mode: +o imer |
| 14:51:34 | <justauser> | klea: https://www.wiki.balug.org/robots.txt while you're on it. |
| 14:52:56 | <klea> | what type of wiki is that? |
| 14:53:08 | <justauser> | Dokuwiki. |
| 14:53:35 | <klea> | ack |
| 14:54:04 | <justauser> | !dw --url https://www.wiki.balug.org/wiki/doku.php?id=start --auto --queue bulk --silent-mode done --user-agent curl |
| 14:54:13 | <justauser> | was what failed for pabs. |
| 14:54:25 | <klea> | it fails also |
| 14:54:37 | <klea> | > IndexError: list index out of range |
| 14:56:19 | <klea> | oh wiki is at https://www.wiki.balug.org/wiki/doku.php |
| 14:59:08 | <klea> | aaa |
| 14:59:10 | <klea> | > requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://www.wiki.balug.org/wiki/doku.php?do=export_xhtml&id=balug%3Abooks_and_publications |
| 14:59:47 | <justauser> | WFM. Ratelimiting? |
| 15:00:28 | <klea> | potentially |
| 15:00:44 | <klea> | i've set delay 1.5 and 3 threads |
| 15:03:10 | <klea> | > dokuWikiDumper.exceptions.RevisionListNotFound: Revision list not found for [[balug:linuxworld]] < i wonder if i should run it with the dangerous ignore errors flag |
| 15:03:27 | <justauser> | Probably, if that's the only page where it happens. |
| 15:03:49 | <justauser> | If it happens everywhere, --curonly is better. |
| 15:03:57 | <klea> | ack |
| 15:04:06 | <klea> | how do i set the UA? |
| 15:05:24 | <justauser> | --user-agent 'Whatever' |
| 15:05:35 | <justauser> | --help works well. |
| 15:06:30 | <klea> | --user-agent not in helptext |
| 15:09:20 | <justauser> | Huh, fun. |
| 15:24:58 | <klea> | justauser: can you check www.wiki.balug.org_wiki-20251229, please. |
| 15:25:02 | <klea> | it's uploading still |
| 15:40:49 | <klea> | done |
| 15:43:10 | <justauser> | Item not found. |
| 15:43:32 | <justauser> | You missed "wiki-". |
| 15:44:57 | <justauser> | Definitely too small. |
| 15:45:57 | <justauser> | https://www.wiki.balug.org/wiki/doku.php?id=sf-lug:build_new_sf-lug_box is missing even from the title list. |
| 15:50:58 | <klea> | so i should delete that item? |
| 15:52:13 | <justauser> | Definitely keep: it's better than nothing. |
| 15:52:31 | <justauser> | I'll try to investigate why did it behave that badly. |
| 15:53:48 | <klea> | ack |
| 15:54:07 | <klea> | i'll clean it up and redo a new dump because i touched the options but didn't clean the data directory |
| 16:03:21 | <klea> | i'm making an asciinema recording this time. |
| 16:18:14 | | DogsRNice joins |
| 16:58:20 | <klea> | justauser: https://archive.org/details/wiki-www.wiki.balug.org_wiki-202512290001 -> https://ia600105.us.archive.org/view_archive.php?archive=/13/items/wiki-www.wiki.balug.org_wiki-202512290001/www.wiki.balug.org_wiki-202512290001-pages.7z&file=pages%2Fsf-lug%2Fbuild_new_sf-lug_box.txt |
| 17:02:22 | <justauser> | Now, this is reasonable. |
| 17:10:08 | <klea> | thanks |
| 19:51:19 | <klea> | https://transfer.archivete.am/qPHFD/dokuWikiDumper-2025-12-29T15:56:47Z.cast |
| 19:51:21 | <eggdrop> | inline (for browser viewing): https://transfer.archivete.am/inline/qPHFD/dokuWikiDumper-2025-12-29T15:56:47Z.cast |
| 19:51:22 | <klea> | funky .cast |
| 21:31:01 | | c3manu quits [Ping timeout: 272 seconds] |
| 21:32:12 | | jacksonchen666 quits [Ping timeout: 256 seconds] |
| 21:34:19 | | c3manu (c3manu) joins |
| 21:34:58 | | jacksonchen666 (jacksonchen666) joins |
| 21:39:14 | <@JAA> | klea: With the next run a bit after midnight, a dump without history or files should be produced. |
| 21:39:36 | <klea> | JAA: thanks |