HomeSearchPrevious dayNext day

00:05:11Wohlstand quits [Client Quit]
00:05:15tekulvw (tekulvw) joins
00:07:11APOLLO03 quits [Client Quit]
00:07:25APOLLO03 joins
00:09:59tekulvw quits [Ping timeout: 272 seconds]
00:17:35nicolas17 quits [Ping timeout: 272 seconds]
00:18:35APOLLO03a joins
00:19:00APOLLO03 quits [Read error: Connection reset by peer]
00:23:17ThreeHM quits [Ping timeout: 272 seconds]
00:23:38ThreeHM (ThreeHeadedMonkey) joins
00:35:50nicolas17 (nicolas17) joins
00:52:34tekulvw (tekulvw) joins
00:52:48etnguyen03 quits [Client Quit]
00:57:26tekulvw quits [Ping timeout: 268 seconds]
01:02:21APOLLO03a quits [Read error: Connection reset by peer]
01:04:08APOLLO03 joins
01:11:22cyanbox quits [Read error: Connection reset by peer]
01:12:37tekulvw (tekulvw) joins
01:16:22etnguyen03 (etnguyen03) joins
01:28:31tekulvw quits [Ping timeout: 272 seconds]
01:30:22tekulvw (tekulvw) joins
01:37:49APOLLO03a joins
01:38:39tekulvw quits [Ping timeout: 272 seconds]
01:38:45APOLLO03 quits [Ping timeout: 268 seconds]
02:06:08etnguyen03 quits [Client Quit]
02:09:22sec^nd quits [Ping timeout: 245 seconds]
02:09:47SootBector quits [Ping timeout: 245 seconds]
02:12:49SootBector (SootBector) joins
02:14:40sec^nd (second) joins
02:24:23ljcool2006 joins
02:28:51s-crypt quits [Quit: Ping timeout (120 seconds)]
02:29:07s-crypt (s-crypt) joins
02:31:59Webuser181299 joins
02:37:21etnguyen03 (etnguyen03) joins
02:38:37Flashfire42 quits [Quit: Ping timeout (120 seconds)]
02:39:09Flashfire42 (flashfire42) joins
02:40:28Webuser181299 quits [Client Quit]
02:41:21cyanbox joins
02:47:40APOLLO03 joins
02:47:41APOLLO03a quits [Ping timeout: 272 seconds]
02:49:03cyanbox quits [Ping timeout: 268 seconds]
02:54:41<pabs>ZoeB: looks like the highest auction number is https://spheremusic.com/Bargaindtl.asp?Item=30514
02:55:05<pabs>ZoeB: anything over 30514 redirects to /Session.asp for me
02:59:25Arcorann__ (Arcorann) joins
03:04:01<pabs>also, looks like /Bargainqa.asp needs an artificially added ?Item= too, otherwise you just get the one from your session cookie
03:06:07<pabs>oh, and there is some JS to contend with
03:10:18<pabs>oh, and the News page uses POST
03:10:37<nicolas17>/o\
03:10:53<pabs>can't fake it with GET params either
03:13:51<pabs>the news pagination is JS + POST...
03:14:16<pabs>at least the news items themselves are GET based
03:15:02<pabs>oh noes, found an ODBC error https://spheremusic.com/Modeldtl.asp?Model=738
03:20:28<pabs>ugh, some broken links like https://spheremusic.com/%3Chttp://www.dynaudioacoustics.com
03:23:26<pabs>oh, www. is the canonical domain, but some parts link to non-www urls
03:24:02<pabs>and the FAQ is JS/POST too :/
03:26:30APOLLO03a joins
03:27:35APOLLO03 quits [Ping timeout: 272 seconds]
03:31:57APOLLO03a quits [Read error: Connection reset by peer]
03:32:02APOLLO03 joins
03:35:47<pabs>gah, the gallery items are JS based
03:41:24<pabs>oh, ?Lg=F sets a cookie to make the UI French
03:43:43pedantic-darwin quits [Quit: The Lounge - https://thelounge.chat]
03:54:18APOLLO03a joins
03:55:12APOLLO03 quits [Read error: Connection reset by peer]
04:04:30cyanbox joins
04:11:04etnguyen03 quits [Client Quit]
04:12:04APOLLO03a quits [Client Quit]
04:13:03APOLLO03 joins
04:18:02<pabs>ZoeB: https://transfer.archivete.am/p557w/spheremusic.com-archivebot-plan.txt
04:18:02<eggdrop>inline (for browser viewing): https://transfer.archivete.am/inline/p557w/spheremusic.com-archivebot-plan.txt
04:23:58tekulvw (tekulvw) joins
04:25:00ducky_ (ducky) joins
04:27:43ducky quits [Ping timeout: 268 seconds]
04:28:33etnguyen03 (etnguyen03) joins
04:29:01tekulvw quits [Ping timeout: 272 seconds]
04:29:28etnguyen03 quits [Remote host closed the connection]
04:30:17ducky_ quits [Ping timeout: 272 seconds]
04:38:12HP_Archivist quits [Ping timeout: 268 seconds]
04:41:51ducky (ducky) joins
04:57:46Starchives quits [Read error: Connection reset by peer]
05:00:55igloo22225 quits [Read error: Connection reset by peer]
05:01:01igloo222256 (igloo22225) joins
05:01:18<h2ibot>PaulWise edited Obstacles (+158, add HTTP POST issue, smolnet protocols): https://wiki.archiveteam.org/?diff=60549&oldid=60474
05:04:43n9nes quits [Ping timeout: 268 seconds]
05:05:17n9nes joins
05:06:09Island quits [Read error: Connection reset by peer]
05:07:10APOLLO03 quits [Client Quit]
05:07:23APOLLO03 joins
05:24:49tekulvw (tekulvw) joins
05:29:49tekulvw quits [Ping timeout: 272 seconds]
05:34:12APOLLO03 quits [Client Quit]
05:34:21APOLLO03 joins
05:47:43APOLLO03 quits [Client Quit]
05:48:56APOLLO03 joins
05:52:43SootBector quits [Remote host closed the connection]
05:53:49SootBector (SootBector) joins
05:55:49Starchives joins
05:56:25APOLLO03 quits [Ping timeout: 272 seconds]
06:06:54nexussfan quits [Quit: Konversation terminated!]
06:12:09HP_Archivist (HP_Archivist) joins
06:21:10Justin[home] joins
06:23:01DopefishJustin quits [Ping timeout: 272 seconds]
06:25:43archiveDrill quits [Quit: The Lounge - https://thelounge.chat]
06:26:15archiveDrill joins
06:50:10^ quits [Ping timeout: 268 seconds]
06:50:16^ (^) joins
07:01:15tekulvw (tekulvw) joins
07:09:17tekulvw quits [Ping timeout: 268 seconds]
07:22:42<h2ibot>Hans5958 edited Main Page/Current Projects (+203, Complete Open Diary and add Roblox Groups): https://wiki.archiveteam.org/?diff=60550&oldid=60380
07:23:21<ZoeB>@pabs That sounds very likely about the highest auction number. And wow, thank you, that file looks about right, and far more thorough than mine would have been!
07:27:03tekulvw (tekulvw) joins
07:32:06tekulvw quits [Ping timeout: 268 seconds]
07:41:14lennier2 joins
07:43:35tekulvw (tekulvw) joins
07:44:43lennier2_ quits [Ping timeout: 272 seconds]
07:48:31tekulvw quits [Ping timeout: 272 seconds]
08:02:23<skankhunt42>hey there! Is it worth to spin up more containers for the opendiary grab or would I just hit the tracker limit? I am using a /64 ipv6 subnet and a single ip for each container. I can spin up like 1000 containers easily, just wondering if it is worth it
08:04:41<skankhunt42>just wanted to help to get this done before they shut down
08:43:53tekulvw (tekulvw) joins
08:48:41tekulvw quits [Ping timeout: 272 seconds]
08:51:35APOLLO03 joins
09:10:17<pabs>do we have anything other than Mnbot that can save HTTP POST requests to WARC?
09:13:00<h2ibot>Manu edited Discourse/archived (+100, Queued discourse.pi-hole.net): https://wiki.archiveteam.org/?diff=60551&oldid=60544
09:14:00<h2ibot>Manu edited Distributed recursive crawls (+80, Candidates: Add www.tsp.gob.cu): https://wiki.archiveteam.org/?diff=60552&oldid=60539
09:22:44<pabs>ZoeB: running now http://archivebot.com/?initialFilter=spheremusic
09:22:58<ZoeB>Wonderful, thank you so much!
09:23:12<pabs>shit, forgot to add the ignores. restarting
09:27:18chunkynutz601 joins
09:28:35chunkynutz60 quits [Ping timeout: 272 seconds]
09:28:36chunkynutz601 is now known as chunkynutz60
09:30:11<ZoeB>Is it possible to slow it down a bit without restarting it? I don't want to get them slapped with a bandwidth bill or anything.
09:31:01<pabs>sure
09:31:18<pabs>how slow do you want it?
09:34:49nathang2184 quits [Ping timeout: 268 seconds]
09:37:28<ZoeB>Well it's 30,000 items and we have at least 30 days, so we don't need more than 1,000 items a day. With 86400 seconds in a day, we can wait up to a minute per item. Maybe something like 10 seconds per item would be sensible?
09:39:34<ZoeB>Ah, I forgot each item can have multiple images... though I think most are less than 6, so somewhere in the region of 5-10 seconds per item should work, I think...
09:41:55nathang2184 joins
09:45:13Webuser213772 joins
09:56:09tekulvw (tekulvw) joins
10:00:35cyanbox quits [Read error: Connection reset by peer]
10:00:53tekulvw quits [Ping timeout: 272 seconds]
10:05:37<ZoeB>Much better, thanks again!
10:25:25APOLLO03 quits [Client Quit]
10:25:58APOLLO03 joins
11:10:57APOLLO03 quits [Client Quit]
11:11:21APOLLO03 joins
11:30:34tekulvw (tekulvw) joins
11:35:41tekulvw quits [Ping timeout: 268 seconds]
11:37:14APOLLO03a joins
11:40:06APOLLO03 quits [Read error: Connection reset by peer]
11:46:37Dada joins
11:48:02APOLLO03a quits [Client Quit]
11:49:06APOLLO03 joins
12:00:03Bleo1826007227196234552220 quits [Quit: The Lounge - https://thelounge.chat]
12:02:48Bleo1826007227196234552220 joins
12:26:36BitByBit quits [Quit: The Lounge - https://thelounge.chat]
12:28:07BitByBit (BitByBit) joins
12:33:55eythian quits [Quit: http://quassel-irc.org - Chat comfortabel. Waar dan ook.]
12:34:58eythian joins
12:37:55APOLLO03a joins
12:41:07APOLLO03 quits [Ping timeout: 272 seconds]
12:43:44APOLLO03 joins
12:43:51APOLLO03a quits [Read error: Connection reset by peer]
12:58:07APOLLO03a joins
12:58:16APOLLO03 quits [Read error: Connection reset by peer]
12:59:33Arcorann__ quits [Ping timeout: 268 seconds]
13:02:08APOLLO03a quits [Client Quit]
13:04:05APOLLO03 joins
13:50:14APOLLO03a joins
13:51:21APOLLO03 quits [Ping timeout: 268 seconds]
14:00:38BitByBit quits [Read error: Connection reset by peer]
14:00:54BitByBit (BitByBit) joins
14:04:49APOLLO03a quits [Client Quit]
14:05:38APOLLO03 joins
14:08:11FiTheArchiver joins
14:08:13FiTheArchiver quits [Remote host closed the connection]
14:14:44<hexagonwin>this seems really unrelated to the topic, but I'm unable to find out myself.. does anyone know how archiveteam warrior does concurrent uploading using rsync?
14:15:00APOLLO03 quits [Read error: Connection reset by peer]
14:17:54APOLLO03 joins
14:29:34<TheTechRobo>pabs: qwarc can do it, I had a spec file that could take an input file with URLs + post data and write a WARC with all of it
14:31:33<justauser>skankhunt42: OpenDiary is currently limited on tracker, too easy to overload. Might be welcome at other projects, though.
14:32:21<skankhunt42>justauser: got it, will scale down then and shift the resources to another project :)
14:32:59<justauser>I suspect Wget-at can do POST, but I don't have it running (yet?).
14:34:27Dada quits [Remote host closed the connection]
14:49:48<h2ibot>Justauser edited Obstacles (+261, /* Protocol choices */ Reworded a bit,…): https://wiki.archiveteam.org/?diff=60553&oldid=60549
15:08:26<TheTechRobo>justauser: For your [citation needed], see https://github.com/iipc/warc-specifications/issues/106
15:09:48<justauser>Oh, nice.
15:11:24chrismeller3 quits [Quit: chrismeller3]
15:12:01chrismeller3 (chrismeller) joins
15:24:07tekulvw (tekulvw) joins
15:28:20APOLLO03 quits [Client Quit]
15:28:57tekulvw quits [Ping timeout: 272 seconds]
15:29:21APOLLO03 joins
15:41:55<h2ibot>Justauser edited Obstacles (-5, /* Protocol choices */ Citation provided by…): https://wiki.archiveteam.org/?diff=60554&oldid=60553
15:48:57<justauser>Looking for tools similar to !ao < but with LIFO queue, ideally deployed at AT.
15:49:50<justauser>I want to grab Plague Inc. custom scenarios, but their download is:
15:50:00<justauser>https://s.ndemiccreations.com/plague/scenarios_play?id=10000
15:50:20<justauser>You make a request and get a link that expires in a few seconds.
16:26:07APOLLO03 quits [Client Quit]
16:26:21APOLLO03 joins
16:37:20Ryz2 quits [Quit: Ping timeout (120 seconds)]
16:37:33Ryz2 (Ryz) joins
16:41:20APOLLO03 quits [Client Quit]
16:42:25APOLLO03 joins
16:43:45ZoeB leaves
16:46:24pedantic-darwin joins
16:51:41tekulvw (tekulvw) joins
16:57:37GodzFire joins
16:59:25<GodzFire>pokechu22 sorry for not following up sooner. Did Cloudflare ever cooperate again so you could backup that sound effects Wikia?
17:00:03tekulvw quits [Ping timeout: 268 seconds]
17:02:27APOLLO03 quits [Client Quit]
17:02:53APOLLO03 joins
17:03:24<@arkiver>i remember some information about tenor was posted before, did we have a rough size estimate?
17:06:22tekulvw (tekulvw) joins
17:07:35Island joins
17:10:27<justauser>GodzFire: Doesn't look like it.
17:11:09tekulvw quits [Ping timeout: 268 seconds]
17:11:15<justauser>arkiver: Someone made a dump of images with old-style IDs.
17:11:42<@arkiver>yeah, the one on IA
17:11:44<@arkiver>justauser: ^
17:12:28<justauser>2 TB of images and it's expected to be over 90% of total Tenor.
17:12:41<@arkiver>2 TB seems very small
17:12:43<justauser>Only JSON descriptions were uploaded for now IIUC.
17:13:59<justauser>Actually, I probably invented that 90% number.
17:14:18<justauser>But it's probably that order of magnitude.
17:16:10<justauser>I read the chat as "I got 10 of the 11 million images", but it's actually not like that.
17:18:18tekulvw (tekulvw) joins
17:22:52tekulvw quits [Ping timeout: 268 seconds]
17:31:38tekulvw (tekulvw) joins
17:36:26tekulvw quits [Ping timeout: 268 seconds]
17:43:19APOLLO03 quits [Client Quit]
17:43:37APOLLO03 joins
17:56:10APOLLO03 quits [Ping timeout: 268 seconds]
18:00:37tekulvw (tekulvw) joins
18:01:58Zaxoosh joins
18:02:33Zaxoosh leaves
18:02:57Zaxoosh joins
18:03:10Zaxoosh leaves
18:04:39Zaxoosh joins
18:08:07applebeechandpie joins
18:08:18<applebeechandpie>yo guys
18:09:06applebeechandpie quits [Remote host closed the connection]
18:09:11tekulvw quits [Ping timeout: 272 seconds]
18:13:22<justauser>Hello?
18:14:39kiska quits [Quit: Ping timeout (120 seconds)]
18:15:03kiska (kiska) joins
18:22:34tekulvw (tekulvw) joins
18:31:19tekulvw quits [Ping timeout: 268 seconds]

HomeSearchPrevious dayNext day