HomeSearchPrevious dayNext day

00:02:36etnguyen03 (etnguyen03) joins
00:11:53tekulvw (tekulvw) joins
00:14:07spark3 joins
00:14:21spark74568 joins
00:15:38spark74568 leaves
00:15:43spark3 leaves
00:16:07nepeat quits [Ping timeout: 272 seconds]
00:16:45tekulvw quits [Ping timeout: 272 seconds]
00:18:50nepeat (nepeat) joins
00:38:36Wohlstand quits [Quit: Wohlstand]
00:51:07etnguyen03 quits [Client Quit]
01:11:13mls quits [Ping timeout: 272 seconds]
01:13:37etnguyen03 (etnguyen03) joins
01:15:14ljcool2006 joins
01:15:19tekulvw (tekulvw) joins
01:20:05tekulvw quits [Ping timeout: 272 seconds]
01:23:51roverinexile joins
01:26:37tekulvw (tekulvw) joins
01:27:03rover quits [Ping timeout: 272 seconds]
01:31:26tekulvw quits [Ping timeout: 268 seconds]
01:32:42mls (mls) joins
01:37:49tekulvw (tekulvw) joins
01:42:53tekulvw quits [Ping timeout: 272 seconds]
02:17:49tekulvw (tekulvw) joins
02:26:19tekulvw quits [Ping timeout: 268 seconds]
02:42:25ducky quits [Ping timeout: 272 seconds]
02:43:17ducky (ducky) joins
02:43:18tekulvw (tekulvw) joins
02:46:04hackbug quits [Remote host closed the connection]
02:52:13tekulvw quits [Ping timeout: 268 seconds]
02:52:55hackbug (hackbug) joins
02:57:33tekulvw (tekulvw) joins
03:04:42<Cupping1285>nicolas17 you can use the internet archive extension to archive all of it.
03:06:33fireatseaparks (fireatseaparks) joins
03:08:23tekulvw quits [Ping timeout: 272 seconds]
03:12:27tekulvw (tekulvw) joins
03:13:04fireatseaparks quits [Client Quit]
03:16:17fireatseaparks (fireatseaparks) joins
03:17:15tekulvw quits [Ping timeout: 272 seconds]
03:22:31tekulvw (tekulvw) joins
03:30:33tekulvw quits [Ping timeout: 272 seconds]
03:36:15Juest quits [Ping timeout: 272 seconds]
03:41:36<Yakov>#archiveteam RE: https://tirespy.ca/ is shutting down: Looks like the search functionality is all clientsided and all the items are already preloaded before search at https://storage.googleapis.com/winged-record-376000.appspot.com/json/en_CA/index.json and
03:41:36<Yakov>https://storage.googleapis.com/winged-record-376000.appspot.com/json/en_CA/categories_flattened.json
03:42:26Juest (Juest) joins
03:44:25<Yakov>and for preserving all product images, you just need to generate a list of https://storage.googleapis.com/winged-record-376000.appspot.com/images/products/[LOWERCASEDPRODUCTIDHERE].png from the index.json
03:44:52<Yakov>I'll get a list going now.
03:54:37<Yakov>Waiting for list to be queued by an OP in #archivebot :)
03:58:58etnguyen03 quits [Remote host closed the connection]
04:01:17Juest quits [Ping timeout: 268 seconds]
04:10:31Juest (Juest) joins
04:18:41datechnoman quits [Ping timeout: 272 seconds]
04:19:11Wohlstand (Wohlstand) joins
04:37:37Island quits [Read error: Connection reset by peer]
04:43:02<h2ibot>Zen edited List of websites excluded from the Wayback Machine (+25, add https://www.2chan.net/): https://wiki.archiveteam.org/?diff=60543&oldid=60511
04:43:56Wohlstand quits [Client Quit]
04:45:43DogsRNice_ quits [Read error: Connection reset by peer]
04:52:16nicolas17 quits [Quit: Konversation terminated!]
04:55:05tekulvw (tekulvw) joins
04:59:52tekulvw quits [Ping timeout: 268 seconds]
05:04:48n9nes quits [Ping timeout: 268 seconds]
05:05:22n9nes joins
05:06:35beastbg8_ joins
05:09:21beastbg8 quits [Ping timeout: 272 seconds]
06:03:31datechnoman (datechnoman) joins
06:16:45<hexagonwin>OrIdow6 thanks for the suggestion, but it's a custom python based scraper with automated chrome browser
06:17:37nexussfan quits [Quit: Konversation terminated!]
06:17:56tekulvw (tekulvw) joins
06:19:51<hexagonwin>for now i'll just do it the semi-manual way..
06:22:49tekulvw quits [Ping timeout: 272 seconds]
06:45:55tekulvw (tekulvw) joins
06:50:41tekulvw quits [Ping timeout: 272 seconds]
07:14:54pokechu22 quits [Quit: System maintenance]
07:30:03ljcool2006_ joins
07:31:34ljcool2006 quits [Ping timeout: 268 seconds]
07:48:02pokechu22 (pokechu22) joins
07:55:17tekulvw (tekulvw) joins
07:57:17ducky_ (ducky) joins
07:59:19ducky quits [Ping timeout: 268 seconds]
07:59:20ducky_ is now known as ducky
08:11:02emphie quits [Ping timeout: 268 seconds]
08:34:22emphie joins
08:53:14APOLLO03 joins
08:54:12CraftByte quits [Quit: Ping timeout (120 seconds)]
08:54:39CraftByte (DragonSec|CraftByte) joins
09:09:51APOLLO03 quits [Client Quit]
09:10:06APOLLO03 joins
09:23:19tekulvw quits [Ping timeout: 272 seconds]
09:35:42ZoeB joins
09:38:46<ZoeB>Hi! There's a website I'd like to nominate for (slow, careful) archiving, but it's tricky, as it involves getting sequentially numbered pages that aren't linked to (which curl is generally best at, to my understanding) then getting their contents (which seems more of a wget thing). Does anyone know how to do this?
09:40:59<ZoeB>I guess I can automatically generate an --input-file for wget with the necessary thousands of sequentialy numbered pages..?
09:42:56APOLLO03 quits [Client Quit]
09:50:08APOLLO03 joins
09:59:56TheEnbyperor_ quits [Remote host closed the connection]
09:59:56TheEnbyperor quits [Remote host closed the connection]
10:01:12<pabs>ZoeB: ArchiveBot can do that, and the results will go to web.archive.org. whats the site?
10:02:34<ZoeB>http://spheremusic.com , an auction site for synthesiser gear that's sold off equipment by many notable musicians. Most of the pages aren't linked anywhere, but starts at https://spheremusic.com/Bargaindtl.asp?Item=1 , with pictures starting at https://spheremusic.com/Bargaindtl.asp?Item=3495 and http://spheremusic.com/userimages/Img3495.jpg .
10:03:11<ZoeB>I don't want to DDOS the poor site, but it's worth noting there's talk of their next batch of auctions possibly being the last one as the original founder's retiring.
10:04:19<ZoeB>That's in April.
10:05:30<pabs>should we wait until after that perhaps?
10:09:12<pabs>do you know what the largest item number is?
10:09:30tekulvw (tekulvw) joins
10:10:11petrichor (petrichor) joins
10:14:18TheEnbyperor joins
10:14:37tekulvw quits [Ping timeout: 272 seconds]
10:16:11TheEnbyperor_ (TheEnbyperor) joins
10:25:14APOLLO03 quits [Client Quit]
10:25:59APOLLO03 joins
10:51:36tekulvw (tekulvw) joins
10:55:09@imer quits [Quit: Oh no]
10:56:25tekulvw quits [Ping timeout: 272 seconds]
11:00:34APOLLO03 quits [Client Quit]
11:01:26APOLLO03 joins
11:15:52<h2ibot>Manu edited Discourse/archived (+96, Queued devforum.play.date): https://wiki.archiveteam.org/?diff=60544&oldid=60542
11:20:08<ZoeB>I'm not sure how much longer the site's going to stick around after that... Certainly it might be worth waiting until it starts.
11:21:34APOLLO03 quits [Client Quit]
11:21:47APOLLO03 joins
11:22:29<ZoeB>I'm not sure how high it goes... The highest I looked at during their last auction was https://spheremusic.com/Bargaindtl.asp?Item=30407
11:23:27<ZoeB>This is spanning 25 years of seasonal auctions.
11:35:32<pabs>and the item numbers are sequential?
11:35:48<ZoeB>As far as I know, it certainly looks that way very much.
11:36:13<ZoeB>I don't think they've changed the code since the millennium.
11:37:24<pabs>so /userimages/ is linked from each auction page, so will be auto-discovered
11:38:25<ZoeB>It should be, yes. The only tricky part as far as regular wget is concerned is finding the pages themselves, which although sequential don't seem to be linked to from anywhere anymore once the season ends... even though other websites link to noteworthy pages.
11:38:59<ZoeB>As you've got that covered, it should hopefully be pretty straightforward, I'd imagine.
11:39:06<pabs>that should be easy, just generate the full list and add it to the ArchiveBot URL list
11:39:09APOLLO03 quits [Client Quit]
11:39:41APOLLO03 joins
11:39:46<pabs>fun, the img src URLs use the wrong slashes, but AB seems to cope with that
11:40:54<pabs>do we have a date for adding it to https://wiki.archiveteam.org/index.php/Deathwatch
11:41:14TheEnbyperor_ quits [Read error: Connection reset by peer]
11:42:10linuxgemini7 (linuxgemini) joins
11:43:10linuxgemini quits [Ping timeout: 268 seconds]
11:43:11linuxgemini7 is now known as linuxgemini
11:45:42TheEnbyperor_ (TheEnbyperor) joins
11:58:02APOLLO03 quits [Client Quit]
12:00:01Bleo1826007227196234552220 quits [Quit: The Lounge - https://thelounge.chat]
12:02:35APOLLO03 joins
12:02:43Bleo1826007227196234552220 joins
12:03:37imer (imer) joins
12:03:37@ChanServ sets mode: +o imer
12:05:35APOLLO03 quits [Client Quit]
12:09:22APOLLO03 joins
12:13:08SootBect1 quits [Remote host closed the connection]
12:14:18SootBector (SootBector) joins
12:16:39@imer quits [Client Quit]
12:16:43APOLLO03a joins
12:16:51APOLLO03 quits [Ping timeout: 272 seconds]
12:17:05imer (imer) joins
12:17:06@ChanServ sets mode: +o imer
12:18:50<ZoeB>Let's see... the final auctions start on April 4 and end on April 11.
12:19:28<ZoeB>I have no idea if they'll pull the plug right there and then, or a month later, or a year.
12:20:31<ZoeB>They might add more items halfway through that active week, though I'm mostly concerned about the 25 years of history.
12:22:20<ZoeB>e.g. if it's possible to spread out the first 30,000 items to, say, 1,000 a day before then, that might be better..?
12:28:00<ZoeB>Thank you for your help, by the way! I'd have done a worse job of it by myself, I'm sure.
12:28:07<ZoeB>It's nice to know it'll be done right.
12:31:51APOLLO03a quits [Client Quit]
12:32:34APOLLO03 joins
12:47:56@imer quits [Client Quit]
12:48:25imer (imer) joins
12:48:25@ChanServ sets mode: +o imer
12:53:06SootBector quits [Remote host closed the connection]
12:54:15SootBector (SootBector) joins
12:56:15notSokar quits [Remote host closed the connection]
12:57:33Sokar joins
13:10:44Arcorann__ quits [Ping timeout: 268 seconds]
13:21:27roverinexile quits [Ping timeout: 272 seconds]
13:21:46rover joins
13:50:41<pabs>ZoeB: maybe its best to do the whole thing now, and then do the few missing pages after the last auctions
13:50:55<pabs>I'll look at it tomorrow if I get a chance
13:52:13<ZoeB>That sounds good. Thank you!
14:13:46sec^nd quits [Remote host closed the connection]
14:14:04sec^nd (second) joins
14:22:16<h2ibot>Sanqui edited Deathwatch (+299, Add Deutsch für dich): https://wiki.archiveteam.org/?diff=60545&oldid=60537
14:22:22SootBector quits [Remote host closed the connection]
14:23:16<h2ibot>Sanqui edited Deathwatch (-3, typo): https://wiki.archiveteam.org/?diff=60546&oldid=60545
14:23:34SootBector (SootBector) joins
14:23:49tekulvw (tekulvw) joins
14:28:35tekulvw quits [Ping timeout: 272 seconds]
14:30:23APOLLO03 quits [Client Quit]
14:31:36APOLLO03 joins
14:56:27APOLLO03 quits [Ping timeout: 272 seconds]
15:04:39SootBector quits [Remote host closed the connection]
15:06:08SootBector (SootBector) joins
15:08:29datechnoman quits [Ping timeout: 272 seconds]
15:13:24tekulvw (tekulvw) joins
15:18:23tekulvw quits [Ping timeout: 268 seconds]
16:07:03datechnoman (datechnoman) joins
16:09:45ljcool2006__ joins
16:13:53ljcool2006_ quits [Ping timeout: 268 seconds]
16:18:21Freiner quits [Quit: Ooops, wrong browser tab.]
16:24:29ljcool2006__ quits [Ping timeout: 272 seconds]
16:34:16petrichor quits [Quit: ZNC 1.10.1 - https://znc.in]
16:47:17Juest quits [Ping timeout: 272 seconds]
16:48:29Juest (Juest) joins
16:50:44TunaLobster quits [Quit: Ping timeout (120 seconds)]
16:50:58TunaLobster joins
17:21:00petrichor (petrichor) joins
17:25:56cipherrot (petrichor) joins
17:27:49petrichor quits [Ping timeout: 272 seconds]
17:30:34Island joins
17:31:49itachi1706 (itachi1706) joins
17:35:28tekulvw (tekulvw) joins
17:40:29tekulvw quits [Ping timeout: 272 seconds]
17:56:45cipherrot quits [Client Quit]
17:58:31petrichor (petrichor) joins
18:06:33tekulvw (tekulvw) joins
18:11:31tekulvw quits [Ping timeout: 272 seconds]
18:32:16nicolas17 (nicolas17) joins
18:34:11twiswist quits [Quit: twiswist]
18:34:57ramsey quits [Ping timeout: 604 seconds]
18:37:19ramsey (ramsey) joins
18:39:26tekulvw (tekulvw) joins
18:44:27tekulvw quits [Ping timeout: 272 seconds]
18:45:27Sk1d joins
18:47:57Webuser114962 joins
18:48:14Webuser114962 quits [Client Quit]
18:49:55tekulvw (tekulvw) joins
18:52:10SootBector quits [Remote host closed the connection]
18:53:17SootBector (SootBector) joins
18:53:21SootBector quits [Remote host closed the connection]
18:54:28SootBector (SootBector) joins
18:58:32tekulvw quits [Ping timeout: 268 seconds]
19:11:44Webuser200070 joins
19:12:30Webuser200070 quits [Client Quit]
19:22:35tekulvw (tekulvw) joins
19:30:03tekulvw quits [Ping timeout: 272 seconds]
19:39:40tekulvw (tekulvw) joins
19:44:47tekulvw quits [Ping timeout: 268 seconds]
19:47:32<klea>huh, #nohost was closed.
19:47:36<klea>> As of January 16 these are being uploaded, but slowly as the IA is generally backlogged, and it may take months before they are all in the WBM
19:47:41<klea>What's the status?
20:05:39APOLLO03 joins
20:07:40petrichor quits [Client Quit]
20:15:06<klea>How do we archive this? https://opendata.aemet.es/centrodedescargas/inicio
20:16:53tekulvw (tekulvw) joins
20:21:47tekulvw quits [Ping timeout: 268 seconds]
20:28:36tekulvw (tekulvw) joins
20:33:23tekulvw quits [Ping timeout: 272 seconds]
20:49:16tekulvw (tekulvw) joins
20:54:45cyanbox joins
20:54:52<@imer>klea: those should be all uploaded, target certainly doesn't exist anymore
20:55:44petrichor (petrichor) joins
20:58:00<@imer>last ones uploaded june 15th apparently (looking at ia)
20:59:41Juest quits [Read error: Connection reset by peer]
21:02:39Juest (Juest) joins
21:05:09<h2ibot>Klea edited Cohost (+147, Update wording, make use of [[Template:Datetime]]): https://wiki.archiveteam.org/?diff=60547&oldid=59433
21:05:10<klea>Updated wiki page, thanks.
21:05:41tekulvw quits [Ping timeout: 272 seconds]
21:11:57Juesto (Juest) joins
21:15:11Juest quits [Ping timeout: 272 seconds]
21:15:12Juesto is now known as Juest
21:18:04tekulvw (tekulvw) joins
21:22:47tekulvw quits [Ping timeout: 272 seconds]
21:29:02Starchives_ joins
21:33:33Starchives quits [Ping timeout: 272 seconds]
21:45:54APOLLO03 quits [Client Quit]
21:46:51APOLLO03 joins
21:47:09Starchives joins
21:47:30Starchives_ quits [Ping timeout: 268 seconds]
22:00:32<kline>ls
22:11:09<klea>.config/ .ssh/ archiveteam/ little-things/ at-tayich/ dox/ copyparty.conf
22:11:28<BlankEclair>ls little-things
22:12:10petrichor quits [Ping timeout: 268 seconds]
22:14:16<klea>too long for irc lol.
22:14:39<klea>source is at https://gitea.arpa.li/JustAnotherArchivist/little-things if you want to take a look.
22:16:57JayEmbee (JayEmbee) joins
22:17:35<kline>whoops
22:19:16petrichor (petrichor) joins
22:19:25<kline>is there anyone who can moderate my edit to https://wiki.archiveteam.org/index.php/ArchiveTeam_Chain_Gang ?
22:19:34<kline>I think it's been ~1w now
22:19:45<kline>(no rush, im just saying i promise im being patient!)
22:22:43tekulvw (tekulvw) joins
22:23:32klea meows a link for whomever to go directly to mod screen. https://wiki.archiveteam.org/index.php/Special:Moderation
22:23:45<klea>I wonder how stupid it'd be to make more people from AT mods on the wiki.
22:26:19<h2ibot>BlankEclair edited ArchiveTeam Chain Gang (-3, /* Main Project Objective */ Fix list): https://wiki.archiveteam.org/?diff=60548&oldid=47511
22:27:23tekulvw quits [Ping timeout: 272 seconds]
22:29:21Webuser661707 joins
22:29:34<klea>oh no, if that was kline's edit, BlankEclair stole it :p
22:29:42<BlankEclair>?
22:29:46Webuser661707 quits [Client Quit]
22:29:51<BlankEclair>ah
22:29:53<BlankEclair>:3c
22:30:35<kline>if i go to the edit screen, it says im still awaiting moderation
22:30:40<kline>so it's not gone!
22:30:56<klea>Yeah
22:31:21<klea>you're still awaiting moderation, at the technical level, but the moderator now has to deal with approving or rejecting a change that may not change anything :p
22:31:28<kline>oh, no, that wasnt the change ive made. ive started doing the work proposed on that page, ive added links to the code and to the uploaded items
22:31:44<kline>sorry, i thought you meant my change had be overwritten
22:35:17<klea>no, it will just require the moderator to manually merge it.
22:39:43tekulvw (tekulvw) joins
22:42:02etnguyen03 (etnguyen03) joins
22:44:51tekulvw quits [Ping timeout: 268 seconds]
22:51:22Wohlstand1 (Wohlstand) joins
22:53:40Wohlstand1 is now known as Wohlstand
23:03:11<klea>regarding aemet opendata, apparently gives urls which seem to work without auth, for example https://opendata.aemet.es/opendata/sh/86b9c7dd_202602242300_climat_targz https://opendata.aemet.es/opendata/sh/db0b84f0 for requests to https://opendata.aemet.es/opendata/api/observacion/convencional/mensajes/tipomensaje/climat/?api_key=[snip], the urls seemed to not have changed even
23:03:11<klea>when I sent a few requests so maybe that url is the same for everybody?
23:04:41<klea>based on the metadata thing it'd even be correct since "© AEMET. Autorizado el uso de la información y su reproducción citando a AEMET como autora de la misma.", any warc would contain the url that the data came from, so citing aemet as author (f).
23:06:08APOLLO03 quits [Client Quit]
23:06:14<klea>however, if we grab that, we should probably also get the ui, since that could be more usefull for people looking trough WBM.
23:07:29APOLLO03 joins
23:14:18<klea>also, idk why the docs for curl told it to use cache-control: no-cache :p
23:17:24Goofybally quits [Killed (NickServ (GHOST command used by Goofybally1!~Goofyball@2.89.157.56))]
23:17:29Goofybally joins
23:25:23<klea>Apparently climate data averages are made once a day with a 4 day delay https://opendata.aemet.es/opendata/sh/b3aa9d28
23:27:06<klea>There's also a list of all weather stations updated once a day. https://opendata.aemet.es/opendata/sh/0b48e183 https://opendata.aemet.es/opendata/sh/0556af7a
23:27:20<klea>And well, the weather predictions thingy
23:27:29nexussfan (nexussfan) joins
23:28:06APOLLO03 quits [Client Quit]
23:28:21APOLLO03 joins
23:30:04<klea>smh, data in text format. https://opendata.aemet.es/opendata/sh/c1b69d25 https://opendata.aemet.es/opendata/sh/9d0f3ac4
23:31:05<klea>oh im stupid/silly, if i take data from the data as text section...
23:32:19<klea>https://opendata.aemet.es/opendata/sh/dfd88b22 also data predictions, which I suppose wouldn't be as desirable to archive?
23:38:30etnguyen03 quits [Client Quit]
23:39:47Sk1d quits [Quit: Leaving]
23:42:05etnguyen03 (etnguyen03) joins

HomeSearchPrevious dayNext day