HomeSearchPrevious dayNext day

00:06:22hooway_ quits [Client Quit]
00:15:22<purplebot>Current Projects edited by Wickedplayer494 (+183, Google Poly to proposed) just now -- https://www.archiveteam.org/?diff=46036&oldid=46034
00:25:07Arcorann (Arcorann) joins
00:27:22<purplebot>Running Archive Team Projects with Docker edited by Atphoenix (+1691, /* Instructions for using Docker …) just now -- https://www.archiveteam.org/?diff=46037&oldid=46009
00:37:22<purplebot>File:Google Poly - 1-1-21.png uploaded by Wickedplayer494 (+0) just now -- https://www.archiveteam.org/?diff=46039&oldid=0
00:37:23<purplebot>File:Google Poly logo.png uploaded by Wickedplayer494 (+0) just now -- https://www.archiveteam.org/?diff=46040&oldid=0
00:38:22<purplebot>Google Poly edited by Wickedplayer494 (+85, Imagery and a navbox) just now -- https://www.archiveteam.org/?diff=46041&oldid=45835
00:45:24nerdguy1138 quits [Ping timeout: 240 seconds]
00:54:22<purplebot>Running Archive Team Projects with Docker edited by Atphoenix (+625, new FAQ entry) 24 minutes ago -- https://www.archiveteam.org/?diff=46038&oldid=46037
00:59:54nerdguy1138 (nerdguy1138) joins
01:05:49tzt quits [Ping timeout: 252 seconds]
01:08:21<purplebot>Wikimedia Incubator created by Nintendofan885 (+745, start) just now -- https://www.archiveteam.org/?diff=46042&oldid=0
01:08:22<purplebot>Template:Wikis edited by Nintendofan885 (+29, +incubator) just now -- https://www.archiveteam.org/?diff=46043&oldid=30799
01:08:22<purplebot>Wikibooks edited by Nintendofan885 (+10, {{online}}) just now -- https://www.archiveteam.org/?diff=46044&oldid=45028
01:08:22<purplebot>Wikinews edited by Nintendofan885 (+10, {{online}}) just now -- https://www.archiveteam.org/?diff=46045&oldid=45023
01:08:22<purplebot>Wikiquote edited by Nintendofan885 (+10, {{online}}) just now -- https://www.archiveteam.org/?diff=46046&oldid=45032
01:09:22<purplebot>2 edits not shown
01:09:22<purplebot>Wikiversity edited by Nintendofan885 (+10, {{online}}) just now -- https://www.archiveteam.org/?diff=46050&oldid=45040
01:09:22<purplebot>Wikivoyage edited by Nintendofan885 (+10, {{online}}) just now -- https://www.archiveteam.org/?diff=46051&oldid=45038
01:09:22<purplebot>Deathwatch edited by Themadprogramer (+258, /* 2020 */) just now -- https://www.archiveteam.org/?diff=46052&oldid=45982
01:09:22<purplebot>Wikispecies deleted by Nintendofan885 (+0, Deleted to make way for move from …) just now -- https://www.archiveteam.org/?title=Wikispecies
01:09:22<purplebot>WikiSpecies edited by Nintendofan885 (+10, {{online}}), moved by Nintendofan885 (+0) just now -- https://www.archiveteam.org/?diff=46053&oldid=45013
01:09:45<@Kaz>"Notice(purplebot): 2 edits not shown"
01:09:47<@Kaz>hmm
01:40:31taka joins
01:53:51<taka>Hi. I've added this to the wiki, but in the spring of 2021, a change in niconico's specifications will remove the ability to display comments and other metadata for deleted videos, as well as comments from earlier dates, including videos that have not been deleted. However, niconico requires an interval for API and web page display (or else an
01:53:51<taka>error will be displayed), and it will be difficult to acquire archives in a short period of time.
01:56:14<OrIdow6>taka: Does this interval apply to each IP address individually, or does it apply to all of them at once?
01:58:03<taka>May be each IP adress.
01:58:04tzt joins
01:58:10<taka>A similar discussion took place before, but no mention was made of past comments.
02:00:01Dallas quits [Client Quit]
02:00:26<taka>OrIdow6 The API reference says, "If the API response status is 503, the server is under heavy load, so please wait at least five minutes before retrying.
02:03:25<OrIdow6>You (or whoever came here last) didn't mention comments, but it looks like someone in ArchiveTeam read the page and figured it out anyway.
02:03:55Dallas (Dallas) joins
02:04:53<OrIdow6>Have they announced any more specific dates than "spring" taka?
02:05:25<OrIdow6>And given that "spring" may have a slightly different meaning between English and Japanese, if they are still vauge, which months do you think it will happen?
02:06:42<OrIdow6>In any case the end-of-the-year workload is letting up, so we might be able to get to it fairly soon; but, for example, if it is shutting down January 10, that would necessitate that ArchiveTeam rearrange its priorities
02:08:14<taka>OrIdow6 Maybe "Spring" means April, or later.
02:08:24<taka>However, at the time, there was no mention of previous comments on videos that had not been deleted, so I would like to request if it is possible to archive those as well.
02:09:08<taka>Their announcement is still Spring.
02:15:29<OrIdow6>taka: Can you clarify what you mean by "previous comments"?
02:16:13<taka>On niconico, comments are displayed by flowing from right to left on the video. However, when the number of comments reaches a certain level, the old comments disappear, but there is a function that allows you to display the comments from that day by specifying the date in the calendar. However, this is scheduled to change in the spring.For this
02:16:13<taka>archive, it is necessary to get all the comments for all the days of all the videos, which I think is difficult, but is it possible?
02:16:58<OrIdow6>Thank you for explaining.
02:18:15<OrIdow6>I haven't looked at the website in detail yet, so I don't know for certain that there is a way; but because of how you have explained it, it sounds like there is a way
02:20:27<taka>Understood, thank you.
02:20:48<OrIdow6>I'll look into it, thank you.
02:22:00<taka>Note that niconico has recently discontinued its old API.
02:30:08taka leaves
02:36:18<OrIdow6>Two notes on that site: I suspect that some comments (NSFW ones?) may be hidden to logged-out users; and https://www.nicovideo.jp/watch/sm125732 is a good test for downloading comments (with about 2.6m of them)
02:40:22<OrIdow6>Also, sorting by number of comments does not seem to completely work; this may be related to the first
02:46:46<OrIdow6>https://qa.nicovideo.jp/faq/show/941?site_domain=default - documentation on what the English UI calls "Comment no-no level"
02:46:57taka joins
02:51:36<atphoenix>For those who are running AT docker images...does 'docker logs -n 0 -f archiveteam' work for you? If it does, what Docker version are you running, and on which OS? Docker 19.03.11 on my Ubuntu 20.04 install doesn't support the '-n 0' parameter. That parameter doesn't show up in the local help, but it does appear on Docker's documentation website.
02:57:47<@JAA>atphoenix: The shorthand was only added in v20.10: https://github.com/docker/cli/commit/04ab71457a4224136cf53fbaf4c8fadc7185a54b Try --tail instead.
02:57:48<fuzzy8021>try --tail instead
02:58:07<@JAA>fuzzy8021: Get ninja'd. :-)
02:58:26<fuzzy8021>lol and with a better answer ;)
02:58:29<atphoenix>thanks :) I just found that too. https://github.com/docker/cli/commit/0b7147a2a53d62e64ed2806245ba099e9f19548d#diff-a78856c01fd03c944d7d7e9a279e63cb5a661109d89d329b136d3dcc8406f01b
03:10:22<purplebot>Running Archive Team Projects with Docker edited by Atphoenix (+29, /* How can I see the status of …) just now -- https://www.archiveteam.org/?diff=46055&oldid=46038
03:45:22<purplebot>Running Archive Team Projects with Docker edited by Atphoenix (+32, /* How much disk space will the …), Atphoenix (+226, /* How can I see the status of …) 21 minutes ago -- https://www.archiveteam.org/?diff=46057&oldid=46055
03:51:58@dxrt quits [Ping timeout: 265 seconds]
04:15:59dxrt joins
04:16:02dxrt quits [Changing host]
04:16:02dxrt (dxrt) joins
04:16:02@ChanServ sets mode: +o dxrt
04:16:44rsn quits [Ping timeout: 240 seconds]
04:28:35@dxrt quits [Client Quit]
04:35:13qw3rty_ joins
04:39:20qw3rty__ quits [Ping timeout: 265 seconds]
05:52:19dxrt joins
05:52:21dxrt quits [Changing host]
05:52:22dxrt (dxrt) joins
05:52:22@ChanServ sets mode: +o dxrt
06:57:52@dxrt quits [Client Quit]
06:59:31dxrt joins
06:59:33dxrt quits [Changing host]
06:59:33dxrt (dxrt) joins
06:59:33@ChanServ sets mode: +o dxrt
07:28:01taka quits [Remote host closed the connection]
09:49:27sec^nd quits [Ping timeout: 240 seconds]
09:54:45sec^nd (second) joins
10:10:45hooway joins
11:44:55LutherZane joins
12:58:04Aoede quits [Quit: ZNC - https://znc.in]
13:17:19Aoede (Aoede) joins
13:27:22<purplebot>Coronavirus/Notable deaths edited by Ka (+846, making this somewhat less overly …) just now -- https://www.archiveteam.org/?diff=46058&oldid=45917
13:27:23<purplebot>Wikimedia Incubator edited by Ka (+24, describing the purpose) just now -- https://www.archiveteam.org/?diff=46059&oldid=46042
13:27:23<purplebot>Deathwatch edited by Taka (+284, /* 2021 */) just now -- https://www.archiveteam.org/?diff=46060&oldid=46052
13:36:22<purplebot>Webzdarma edited by Sanqui (+25, /* ArchiveBot jobs */ job 18) just now -- https://www.archiveteam.org/?diff=46061&oldid=46002
14:20:04Arcorann quits [Ping timeout: 240 seconds]
14:23:38katocala quits [Remote host closed the connection]
14:30:54katocala joins
14:33:20<betamax>how long does it take for an archivebot job to show up in the wayback machine? and is there a way to see when this has actually happened, aside from just checking one of the URLs in web.archive.org and seeing when it appears?
14:33:40<@kiska>upto 72h I think
14:35:31<betamax>that's good - I noticed some of the sploder forum pages had started appearing but on the whole it was very incomplete, so those must be "save page now" grabs done by individuals, rather than the archivebot job
14:35:46<betamax>(I was kinda panicking since the forums are now offline)
14:35:58<@kiska>JAA: ^^
15:41:44lukash7 quits [Ping timeout: 240 seconds]
15:52:22lukash7 joins
16:13:53<thuban>does anyone object to me feeding a list of 44k ~3K images to archivebot
16:18:11<thuban>i lied, average is probably more like 10-15K
16:18:44mutantmonkey quits [Ping timeout: 240 seconds]
16:21:29<betamax>I doubt anyone would have a problem - unless they're NSFW
16:22:44<thuban>no, nothing like that. it's just that i know we're in a bit of a crunch and i wanted to be on the safe side
16:24:07<EggplantN_>Orldow6 you available?
16:26:10<betamax>thuban: in my mind 125MB is very minimal, I'd say go for it
16:27:11<betamax>hmm, although the topic is "urgent jobs only"...
16:27:51<betamax>if it was me I would do it as I have a terrible memory and would 100% forget if I didn't do it now
16:31:26mutantmonkey (mutantmonkey) joins
16:43:01<@JAA>betamax: Looks like the upload backlog is 16-17 hours currently, plus however long IA takes to derive and then integrate into the WBM index. But yeah, 1-2 days usually if everything's running smoothly, sometimes longer.
16:44:06<@JAA>There's only one file from the still-running members page job pending upload currently.
16:44:17<@JAA>So all the other Sploder stuff should be on IA already.
16:45:05<@JAA>By the time it shows up on the viewer, it should usually be in the WBM I think. Or at least it shouldn't take much longer after that.
16:50:45<betamax>> By the time it shows up on the viewer
16:50:49<betamax>what viewer is this?
16:51:04britmob quits [Ping timeout: 240 seconds]
16:51:17<atphoenix>I guess AB Viewer
16:51:24<atphoenix>https://archive.fart.website/archivebot/viewer/
16:52:25<@JAA>Yep
16:52:31<@JAA>If it shows up there, that is, since it's buggy.
16:53:26<atphoenix>FWIW, we have a consolidated list of links to various AB mgmt/control/view tools on the AB wiki article: https://archiveteam.org/index.php?title=ArchiveBot#Usage.2C_Dashboards.2C_and_Completed_Job_Viewer
17:23:08<betamax>thanks, I've found a page in the CDX that isn't in the wayback yet so it must still be being processed / derived by the WBM
17:56:22<purplebot>Mediafire created by Kiska (+640, Init) just now -- https://www.archiveteam.org/?diff=46062&oldid=0
18:06:03HackMii quits [Client Quit]
18:06:25HackMii (hacktheplanet) joins
18:24:22<purplebot>Mediafire moved by JustAnotherArchivist (+0, Fix capitalisation) 23 minutes ago -- https://www.archiveteam.org/?diff=46063&oldid=0
18:37:40@dxrt quits [Client Quit]
18:45:28dxrt joins
18:45:30dxrt quits [Changing host]
18:45:30dxrt (dxrt) joins
18:45:30@ChanServ sets mode: +o dxrt
19:14:37britmob joins
19:33:28@dxrt quits [Client Quit]
19:42:10Ka (Ka) joins
19:43:42anarcat quits [Remote host closed the connection]
20:01:04dxrt joins
20:01:06dxrt quits [Changing host]
20:01:06dxrt (dxrt) joins
20:01:06@ChanServ sets mode: +o dxrt
20:23:06rsn joins
20:32:36<@Kaz>Ka: grr
20:51:16<Ka>Huh?
20:51:28<Ka>Oh right, the name I assume
22:29:31tzt quits [Ping timeout: 252 seconds]
23:01:03<brad>Okay, I now have installed grab-site via the instructions at https://github.com/ArchiveTeam/grab-site on an Ubuntu 20.10 instance, and I've got that running with two threads archiving https://community.fantasyflightgames.com/. I'm now going to start working on what it would take to do that as a container.
23:04:12<brad>Once I've got my container method up and running, I'll file a PR to provide that Dockerfile to the grab-site repo, and a PR for the instructions on how to use it.
23:05:58<brad>Then I would be interested to get a DPoS worker deployed via a container.
23:39:56<brad>Hmm. And I was playing arund with trying to increase the number of threads, and got myself banned.
23:40:20<brad>I'll have to drop that public IP address and try another one.
23:40:50<brad>And I'm going back to the defaults, which don't seem to result in banning.

HomeSearchPrevious dayNext day