HomeSearchPrevious dayNext day

00:00:52<fireonlive>skeletor wins
00:03:04etnguyen03 (etnguyen03) joins
00:05:52<vokunal|m>I'm not sure where the number comes from, but one source stated 2,398,412 posts
00:18:21<@JAA>At the bottom on https://www.he-man.org/forums/boards/forum.php
00:32:50<@JAA>(And 'the source' is https://old.reddit.com/r/Archiveteam/comments/17rps3j/hemanorg_forums_shutting_down_after_over_20_years/ I guess.)
00:38:50<vokunal|m>ahhhhhh. thanks
00:48:34etnguyen03 quits [Ping timeout: 265 seconds]
01:25:14etnguyen03 (etnguyen03) joins
01:31:49mcint joins
01:50:55HP_Archivist quits [Client Quit]
01:51:04<Arcorann>Anyone know what's going on with HikariNoAkari? I heard there was some drama and it was shutting down
01:52:09<thuban>it's been discussed, but not with any particular insight
01:52:12<@JAA>Yeah, apparently: https://i.imgur.com/jeJSEu6.jpg
02:05:25Pedrosso quits [Ping timeout: 265 seconds]
02:11:21Pedrosso joins
02:56:10Island quits [Remote host closed the connection]
02:56:10Island joins
02:56:10DogsRNice_ joins
02:56:12DogsRNice quits [Remote host closed the connection]
02:56:12kdqep__ quits [Remote host closed the connection]
02:56:12ScenarioPlanet quits [Remote host closed the connection]
02:56:13parfait_ joins
02:56:18ScenarioPlanet (ScenarioPlanet) joins
03:06:48BearFortress_ joins
03:06:48Island_ joins
03:06:48Scen joins
03:09:15pabs quits [Ping timeout: 272 seconds]
03:10:12Pedrosso quits [Ping timeout: 266 seconds]
03:10:12BearFortress quits [Ping timeout: 266 seconds]
03:11:09ScenarioPlanet quits [Ping timeout: 265 seconds]
03:11:09Island quits [Ping timeout: 265 seconds]
03:11:19Doran is now known as Doranwen
03:20:43pabs (pabs) joins
03:42:11etnguyen03 quits [Ping timeout: 272 seconds]
03:52:51pabs quits [Client Quit]
03:55:28pabs (pabs) joins
04:01:53etnguyen03 (etnguyen03) joins
04:07:27kiryu quits [Remote host closed the connection]
04:08:59kiryu (kiryu) joins
04:12:05<@JAA>So the AB job for Hikari No Akari finished, but it timed out on almost all sitemaps, i.e. I'm not sure it's complete. It does appear to have gone through the pagination, but new posts being added could still have led to some things getting missed.
04:26:31dumbgoy quits [Ping timeout: 272 seconds]
05:09:35apache2 quits [Ping timeout: 272 seconds]
05:15:26DogsRNice_ quits [Read error: Connection reset by peer]
06:00:53etnguyen03 quits [Ping timeout: 272 seconds]
06:10:31nicolas17 quits [Client Quit]
06:12:03etnguyen03 (etnguyen03) joins
06:15:01Dango360_ joins
06:18:37Dango360 quits [Ping timeout: 272 seconds]
06:31:06etnguyen03 quits [Client Quit]
06:40:18_Dango360 joins
06:42:02_Dango360 quits [Client Quit]
06:42:21Dango360 (Dango360) joins
06:44:35Dango360_ quits [Ping timeout: 272 seconds]
06:45:29Dango360_ joins
06:49:08Dango360 quits [Ping timeout: 265 seconds]
06:53:11Island_ quits [Read error: Connection reset by peer]
07:03:37Perk8 joins
07:03:53kdqep__ joins
07:05:29Perk quits [Ping timeout: 272 seconds]
07:05:29Perk8 is now known as Perk
07:07:30parfait_ quits [Ping timeout: 265 seconds]
07:17:06parfait_ joins
07:21:02kdqep__ quits [Ping timeout: 265 seconds]
07:56:05<h2ibot>PaulWise edited Bugzilla (+78, updates): https://wiki.archiveteam.org/?diff=51119&oldid=50954
08:04:52jacksonchen666 (jacksonchen666) joins
08:32:39Dango360_ quits [Client Quit]
08:58:50onkel joins
09:00:08onkel quits [Remote host closed the connection]
09:02:06jacksonchen666 quits [Ping timeout: 245 seconds]
09:12:17jacksonchen666 (jacksonchen666) joins
09:15:33parfait_ quits [Client Quit]
09:36:13lukash9 quits [Ping timeout: 272 seconds]
09:45:26jacksonchen666 quits [Ping timeout: 245 seconds]
09:52:13jacksonchen666 (jacksonchen666) joins
09:57:53jacksonchen666 quits [Client Quit]
10:00:03Bleo1 quits [Client Quit]
10:01:20Bleo1 joins
10:02:36JohnnyJ joins
10:14:26JohnnyJ quits [Client Quit]
10:17:11JohnnyJ joins
10:23:50JohnnyJ quits [Client Quit]
10:55:09SF quits [Ping timeout: 265 seconds]
10:55:25SF joins
11:08:41SF quits [Ping timeout: 272 seconds]
11:21:52SF joins
11:28:29sec^nd quits [Remote host closed the connection]
11:30:27sec^nd (second) joins
11:45:15Scen is now known as ScenarioPlanet
12:09:34T31M_ joins
12:10:13Perk6 joins
12:10:13TheTechRobo quits [Client Quit]
12:10:13T31M quits [Client Quit]
12:10:13mattx433 quits [Client Quit]
12:10:13katocala quits [Remote host closed the connection]
12:10:13Perk quits [Client Quit]
12:10:13nulldata quits [Client Quit]
12:10:13Bleo1 quits [Client Quit]
12:10:13T31M_ is now known as T31M
12:10:14Perk6 is now known as Perk
12:10:14Bleo1 joins
12:10:16katocala joins
12:10:18mattx433 (mattx433) joins
12:10:23nulldata (nulldata) joins
12:10:41TheTechRobo (TheTechRobo) joins
12:16:58Wohlstand (Wohlstand) joins
12:29:47dumbgoy joins
12:34:14BearFortress_ quits [Ping timeout: 265 seconds]
12:39:31BearFortress joins
13:01:25Arcorann quits [Ping timeout: 272 seconds]
13:03:43Wohlstand quits [Ping timeout: 265 seconds]
13:04:37HP_Archivist (HP_Archivist) joins
13:18:13<betamax>high chance it's already been done, but Minnesota is / has run a flag-design contest, all* 2123 designs are on the site: https://serc.mnhs.org/flags
13:18:37<betamax>*all => I'm pretty sure that they're no longer accepting submissions. not 100%, though
13:19:22<betamax>I'm not going to put it in AB because I have only glanced at the site and am not sure if extra work is needed to capture the "click each submission to view the larger-size image"
14:20:48etnguyen03 (etnguyen03) joins
14:50:01sec^nd quits [Ping timeout: 245 seconds]
14:52:50sec^nd (second) joins
14:54:43Megame (Megame) joins
15:09:21etnguyen03 quits [Ping timeout: 272 seconds]
15:41:23icedice (icedice) joins
15:43:29etnguyen03 (etnguyen03) joins
15:52:39DogsRNice joins
16:07:27icedice quits [Client Quit]
16:13:37guest joins
16:13:43<guest>hello.
16:13:54<guest>like using archive bot
16:18:27<fireonlive>i’m glad you do!
16:18:39<fireonlive>it’s a nice bit
16:18:42<fireonlive>bot*
16:27:10<nulldata>Bit Bot
16:29:55Wohlstand (Wohlstand) joins
16:29:59Wohlstand quits [Client Quit]
16:33:34<DogsRNice>beep boop
16:34:17atphoenix_ (atphoenix) joins
16:35:12<@JAA>betamax: Looks like the large images work just fine without JS. They're standard links. I've thrown it into AB.
16:37:23atphoenix__ quits [Ping timeout: 272 seconds]
16:37:59guest quits [Remote host closed the connection]
16:43:54lennier2 joins
16:46:53lennier1 quits [Ping timeout: 272 seconds]
17:12:13etnguyen03 quits [Ping timeout: 272 seconds]
17:20:26<ScenarioPlanet>https://transfer.archivete.am/l5gdO/static.spore.com-ids-2016.txt.zst - Full (?) list of Spore.com creation IDs including INVALID/PURGED/BANNED statuses, as of 2016.
17:21:21<ScenarioPlanet>20741764 entries ^
17:26:08etnguyen03 (etnguyen03) joins
17:30:04Wohlstand (Wohlstand) joins
17:34:59Naruyoko quits [Remote host closed the connection]
17:35:17Naruyoko joins
17:43:35sd quits [Quit: sd]
17:43:57sd (sd) joins
17:46:34Mateon1 joins
17:54:48<pokechu22>ScenarioPlanet: what speed and concurrency can that be ran at?
17:55:13Pedrosso joins
17:55:18<pokechu22>oh, wait, those are just numeric IDs, so it can't be ran directly
17:55:33icedice (icedice) joins
17:55:44<pokechu22>17:20 <ScenarioPlanet> https://transfer.archivete.am/l5gdO/static.spore.com-ids-2016.txt.zst - Full (?) list of Spore.com creation IDs including INVALID/PURGED/BANNED statuses, as of 2016.
17:55:47<pokechu22>17:21 <ScenarioPlanet> 20741764 entries ^
17:56:00<pokechu22>Pedrosso: might find that interesting
17:56:11<Pedrosso>Hey uh pokechu22, I've been looking over archivebots logs of "https://davoonline.com/phpBB3?archiveteam". It's been archiving a lot of the same login page with different "?*" things
17:56:23<Pedrosso>Also yes, I find it very interesting
17:56:36<pokechu22>Yeah, that doesn't look great :|
17:56:52<pokechu22>well, it makes sense for viewtopic, but https://davoonline.com/phpBB3/ucp.php?style=17&mode=login&redirect=search.php%3Fauthor_id%3D6234%26sd%3Dd%26sk%3Dt%26sr%3Dposts%26st%3D0%26start%3D40%26style%3D17 isn't useful
17:56:56parfait (kdqep) joins
17:57:11<Pedrosso>idk too much about the archivebot. I've seen ignorelists used. Idk how they work but can a "just ignore all mode=login lol" work?
17:58:01<pokechu22>Yeah, mode=login would ignore any URLs with the text mode=login in it, while ^https://davoonline.com/phpBB3/ucp\.php.*[?&]mode=login ignores anything starting with https://davoonline.com/phpBB3/ucp.php and containing either ?mode=login or &mode=login
17:58:37<pokechu22>Now, https://davoonline.com/phpBB3/viewtopic.php?style=17&p=18852 is a bit weird too since I'm not sure where the style=17 is coming from - I haven't seen other styles though so maybe it's fine?
17:59:48<pokechu22>hmm, no, URLs from https://davoonline.com/phpBB3/ don't have style=17 but once a URL with style=17 is retrieved that same parameter is added to everything else... and it looks identical to without it
18:00:18<@JAA>pokechu22: https://transfer.archivete.am/inline/68759/2y5iu7ey1kzbspqay7vlkbkuq-trace
18:01:04<pokechu22>... ah, it came from one link that has style=17 on it in https://davoonline.com/phpBB3/viewtopic.php?p=41535#p41535 it looks like
18:01:10benjins joins
18:01:12<pokechu22>Probably best to just nuke that then
18:01:18<pokechu22>we don't need to save everything twice
18:01:46<Pedrosso>Is it able to be modified live with these ignores?
18:02:05<Pedrosso>Oh, nice
18:02:10<pokechu22>Yep, you can add and remove ignores as needed (and adjust the speed and concurrency at which it runs too)
18:03:17Island joins
18:03:39<Pedrosso>As for the spore.com archive, I didn't know it would be able to archive users, but I'm glad it found its way, hah
18:04:10<pokechu22>Looking at http://archivebot.com/ignores/2y5iu7ey1kzbspqay7vlkbkuq we have /ucp\.php\?mode=(login|delete_cookies|pm) in the forums ignoreset - which doesn't work with the random style=17 in the middle
18:04:47benjinsm quits [Ping timeout: 272 seconds]
18:07:08<@JAA>Yeah, that should probably be /ucp\.php\?(.*&)?mode=(login|delete_cookies|pm)(&|$) instead.
18:30:22Mateon2 joins
18:30:26atphoenix_ quits [Remote host closed the connection]
18:30:26Naruyoko quits [Remote host closed the connection]
18:30:26Island quits [Remote host closed the connection]
18:30:26benjins quits [Remote host closed the connection]
18:30:26parfait quits [Remote host closed the connection]
18:30:26icedice quits [Remote host closed the connection]
18:30:29mattx433 quits [Client Quit]
18:30:29Mateon1 quits [Remote host closed the connection]
18:30:29Mateon2 is now known as Mateon1
18:30:29benjins joins
18:30:31Naruyoko joins
18:30:33mattx433 (mattx433) joins
18:30:46Island joins
18:30:48icedice (icedice) joins
18:30:49atphoenix_ (atphoenix) joins
18:30:51parfait (kdqep) joins
18:46:51benjinsm joins
18:50:45benjins quits [Ping timeout: 265 seconds]
18:52:52<vokunal|m>Since AB grabs a lot more than just the urls on a site, is there a way to determine whether a job will finish in time or not? Based on the current rate, he-man.org could grab ~3.3m if all goes well, but those aren't nessesarily all the urls on the site and probably external links
18:56:44<pokechu22>The easiest approach is to use --no-offsite if it seems like it'll be close to not finishing, and then manually run the offsite links afterwards (but that requires having the job's database manually saved since links skipped by --no-offsite don't end up in the log)
19:13:41rktk (rktk) joins
19:16:33<Pedrosso>What's the deal with all the 600,001-600,001 ms delays?
19:17:37null quits [Ping timeout: 272 seconds]
19:18:23<that_lurker>most of those are in a pipeline that has been offline for about a year
19:19:09<@JAA>s/most of //
19:19:51<that_lurker>JAA: Didn't you plan to remove the pipeline? Or is it on the todo list
19:20:04<@JAA>Yeah, the latter.
19:21:00nicolas17 joins
19:30:32Pedrosso quits [Remote host closed the connection]
19:50:49lunik173 quits [Quit: :x]
19:51:14lunik173 joins
19:56:12benjinsm quits [Read error: Connection reset by peer]
20:05:55benjins joins
20:54:05BlueMaxima joins
21:12:45itachi1706 quits [Quit: Bye :P]
21:13:15itachi1706 (itachi1706) joins
21:18:33hitgrr8 quits [Quit: away]
21:36:44Island quits [Remote host closed the connection]
21:36:44qwertyasdfuiopghjkl quits [Remote host closed the connection]
21:36:59Island joins
21:41:13<fireonlive>https://t.me/zlibrary_official/41 "Sad news! Yesterday a large number of our domains were seized again. We should highlight that the majority of the seized domains were not mirrors of the Z-Library website, but they were separate sub-projects, containing only books in rare languages of the world, and their blocking is confusing. For instance, these
21:41:13<fireonlive>domains included books in Tamil, Mongolian, Catalan, Urdu, Pashto, and other languages."
21:47:11icedice2 (icedice) joins
21:47:16BlueMaxima_ joins
21:47:26TheTechRobo quits [Client Quit]
21:47:26nulldata quits [Client Quit]
21:47:26parfait quits [Remote host closed the connection]
21:47:26atphoenix_ quits [Remote host closed the connection]
21:47:26icedice quits [Remote host closed the connection]
21:47:26Naruyoko quits [Remote host closed the connection]
21:47:26BlueMaxima quits [Remote host closed the connection]
21:47:27Naruyoko joins
21:47:31atphoenix__ (atphoenix) joins
21:47:32parfait (kdqep) joins
21:47:43nulldata (nulldata) joins
21:47:52TheTechRobo (TheTechRobo) joins
21:48:15benjinsm joins
21:52:09benjins quits [Ping timeout: 272 seconds]
21:52:51Naruyoko5 joins
21:53:04parfait quits [Remote host closed the connection]
21:53:04Naruyoko quits [Remote host closed the connection]
21:53:04mattx433 quits [Client Quit]
21:53:09mattx433 (mattx433) joins
21:53:18parfait (kdqep) joins
21:55:31Pedrosso joins
21:55:42<Pedrosso>Lost connection, did I miss anything?
21:57:27<Pedrosso>Thanks to whoever added esporo to the archivebot :]
21:59:07Barto quits [Ping timeout: 272 seconds]
22:06:05pabs quits [Ping timeout: 272 seconds]
22:18:23Barto (Barto) joins
22:36:08pabs (pabs) joins
22:47:15etnguyen03 quits [Ping timeout: 272 seconds]
22:50:43Pedrosso quits [Remote host closed the connection]
22:53:05Pedrosso joins
23:18:04dumbgoy_ joins
23:22:05dumbgoy quits [Ping timeout: 272 seconds]
23:32:05mindstrut joins
23:32:20mindstrut quits [Client Quit]
23:33:44Arcorann (Arcorann) joins
23:59:34<h2ibot>Vokunal edited Deathwatch (+11): https://wiki.archiveteam.org/?diff=51120&oldid=51118

HomeSearchPrevious dayNext day